Skill Routing from a Frozen LLM: +13.4 Pts, 0 Extra Params

Every agent harness with a skill library faces the same trade: preload every skill's metadata into context and watch attention disperse as the library grows, or retrieve skills outside the model's sight and hope an external reranker picks better than the agent could. A new preprint — The Router Within: Eliciting Native Skill Routing from a Frozen LLM — claims you don't need either. The routing signal is already inside the frozen model's forward passes. You just need two linear maps to read it out.

What problem does this solve?

Deployed skill systems route by stuffing skill descriptions into the prompt. The cost isn't just tokens — it's structural. Metadata scales linearly with library size, so a few hundred skills at ~150 description tokens each is 30k+ tokens of distraction before the task even starts, and the approach hard-caps how big a library can get. Retrieval pipelines (retrieve-and-rerank) fix the context bloat but move selection into a bolted-on model the agent can't inspect or reason with. Gavel's bet: a frozen LLM already represents "which capability does this task need" in its mid-layer states — you can decode it with a projection instead of a reranker.

What's the method?

Gavel (Glance And Verdict from a frozen LLM) works in two stages. At installation, one forward pass per skill builds a compact "bank" of the model's states on that skill — the library fingerprint. At runtime, the glance projects the task's mid-layer states and each skill's bank through two trained linear maps — the only trained parameters in the whole system — and scores the full library cheaply. The verdict then resumes the forward pass for the shortlisted skills and reads the model's own likelihood and yes/no judgment, fused with the glance score as a product of experts. No skill text ever enters the context. The authors also release SkillTraj, a benchmark of 372 simulated agent trajectories capturing the harder case: skills needed mid-rollout, not announced upfront.

What are the results?

• On Qwen3-32B, Gavel beats progressive disclosure and retrieve-and-rerank pipelines that carry 1.2B to 16B external reranker parameters — by up to 13.4 points on written tasks and up to 21.9 points when the skill need arises mid-rollout.
• It transfers zero-shot to three public benchmarks plus SkillTraj after training once — no per-benchmark tuning.
• In a bash-agent harness, the 32B backbone with Gavel triggered the correct skill on Skill-Use more often than far larger frontier models running in Codex. The router signal, apparently, beats raw scale at this specific task.
• Routing accuracy improves monotonically as the backbone improves — the capability is a property of the model, not the probe.

flowchart LR
    A[Skill install] --> B[One forward pass per skill
compact state bank]
    T[Task arrives] --> C[Glance: linear maps score
library vs mid-layer states]
    B --> C
    C --> D[Shortlist]
    D --> E[Verdict: resume forward pass,
likelihood + yes/no judgment]
    C --> F[Product-of-experts fusion]
    E --> F
    F --> G[Skill activated —
zero tokens of metadata in context]
            

What are the limitations?

Linear maps mean linear separability. Two projections that cleanly separate 50 skills may blur at 5,000 — the paper doesn't stress library scale, and the "caps the library size" criticism cuts both ways until someone probes Gavel at four digits. SkillTraj is simulated trajectories, which risks the usual sim-to-real gap: mid-rollout skill needs in production arrive noisier than benchmark rollouts. The zero-shot transfer claim covers three benchmarks; whether the maps transfer across model families (not just sizes) is untested here. And routing accuracy isn't execution quality — Gavel picks the skill; it can't fix a skill that's badly written once loaded. The Codex comparison is also a harness comparison as much as a router comparison, which muddies the frontier-model headline.

Why should someone building things care?

Two things matter here beyond the leaderboard. First, the economics: a 16B reranker is a second model to host, latency-budget, and keep in sync with the backbone. Two linear maps are a few megabytes. If the frozen backbone already encodes the choice, every skill library that ships a dedicated selection model is paying for signal it already has. Second, the decoupling: when routing happens inside the model's own states rather than its context, library size stops being a prompt-engineering constraint. You can grow a skill library the way you grow an index — it stops being visible to the agent until invoked. For anyone running skill-based agent frameworks (and every serious harness is converging on skills), the actionable read is: instrument your model's internal states at skill-install time now, so the routing probe is a fine-tune away rather than an architecture change. The catch to watch: this couples your router to your backbone version. Swap the model, retrain the maps — cheap, but not free.

The broader frame: 2026's agent stack has been accreting external modules — rerankers, selectors, planners — each buying capability with parameters. Gavel is evidence that at least one of those modules was always redundant, and the receipts say the redundant one was the expensive one.