A router that learns from behavior
The preprint asks a practical question: can an LLM router be learned as a reusable capability that transfers when tasks, candidate models or deployment conditions change? RouteFM is built to inspect behavioral context, meaning records of how anonymous candidate models performed on earlier queries, and infer which candidate is most promising for a new target. The router remains frozen at deployment, so adaptation comes through context rather than parameter updates.
On held-out RouterEval queries, RouteFM had the highest routing quality at every tested observation budget. At the smallest budget, eight behavioral observations for each candidate, its advantage over the strongest baseline was 0.84 quality points. Here, routing quality means the mean observed quality of the selected model. Methods received matched pools, contexts and targets, and target quality was withheld until after routing.
That lead also appeared on MMR-Bench, a seven-dataset collection with 10,370 queries and nine complete candidate models that was excluded from pretraining. RouteFM again had the highest quality at every tested budget. Its margin was 2.23 quality points with eight observations per candidate, but only 0.07 points in the large-observation regime.
Testing the sparse-evidence case
The router was pretrained on 212,470 query-task instances and 1,727,809 valid query-model observations from four sources. Those observations record how candidate models behaved on queries, giving RouteFM behavioral evidence rather than relying on candidate names alone.
Training started from random initialization and used episodes with six to 24 anonymous candidates and 16 to 32 disjoint target queries. Each candidate had 8 to 1,024 behavioral observations available, and training ran for 10,000 AdamW steps with an effective batch size of 16. In the limited-context evaluation, budgets were 8, 16, 32 and 64 observations per candidate; MMR-Bench used five-fold disjoint context-target splits and a regime with 40% of observations for context and 60% for evaluation.
What happened when the pool changed
In a new-candidate test, frozen RouteFM surpassed all refitted baselines after eight observations from the new candidate. When 16 observations were available, the expanded pool outperformed the original incumbent pool. The test examined whether a model could be added while keeping the router itself fixed.
RouteFM also matched the full-context performance of comparison methods using only a fraction of their available observations. The matching fractions were 1% for Weighted kNN, 3% for MLP and 26% for EmbedLLM.
Internal tests gave clues about what the system was using. Removing the candidate-pool Transformer produced the largest reported architectural drop, 1.33 quality points at eight observations per candidate. Removing target-to-context attention also reduced performance when observations were limited. Under a separate support-controlled test, training from scratch with 20% labeled target-domain data trailed frozen RouteFM by 1.08 points at the same budget, while fine-tuning made only small changes.
The breadth of pretraining data mattered in the mixture comparison. The default heterogeneous mixture performed best across the evaluated context sizes. Removing LLMRouterBench was associated with a 4.09-point reduction at eight observations per candidate and a 2.09-point reduction at 64. These comparisons do not establish that one source caused the difference.
A result with clear boundaries
These results answer a bounded benchmark question. The manuscript is a preprint, and its primary endpoint is benchmark routing quality, defined as the mean observed quality of the selected candidate. The reported tests span held-out RouterEval queries, MMR-Bench, architecture and adaptation comparisons, new-candidate incorporation and different context budgets. They do not by themselves establish performance on tasks, candidate pools, modalities or deployment conditions outside those tested.
Paper data and sources
Original title: Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing
Authors: Guannan Lai, Han-Jia Ye
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text