A preprint proposes an AI routing system that decides when a more accurate but costly estimate of a specialist model’s value is worth paying for. Across three benchmark settings—Math inference-time reasoning, retrieval-augmented generation (RAG) and EmbedLLM model selection—Pandora’s Router achieved the best or tied-best average result among the listed centralized methods on a measure combining routing regret with inspection cost.
The cost of better information
The Router works with two value estimates. f is cheap and always available; g is more accurate but costs more to query. It first uses f and invokes g only when a reservation-price rule says the expected value of the extra information clears its cost. Under a Gaussian, or bell-shaped, signal model, the paper derives a closed-form version of that rule.
Three benchmark settings
The evaluation used separate training, calibration and test splits for the three tasks. Reported counts were 9,504 training and 7,008 test Math examples; 18,661 training, 400 calibration and 1,600 test RAG questions; and 16,756 training, 609 calibration and 2,436 test EmbedLLM prompts.
On held-out calibration data, g had lower mean-squared error (MSE), the study’s measure of estimation error, in every domain: 0.096 versus 0.154 for f in Math, 0.109 versus 0.175 in RAG, and 0.198 versus 0.266 in EmbedLLM. The reported ratio of g’s cost to f’s cost was 1.6 in EmbedLLM, 5.8 in Math and more than 7,000 in RAG.
A system that follows the price
Averaged across the tested costly-estimator costs, the Router’s total regret plus inspection cost was 0.105 in Math, 0.118 in RAG and 0.386 in EmbedLLM. It tied Margin-Npr in Math and was lower than that baseline in RAG and EmbedLLM.
The cost sensitivity was visible in frontier tests. At a costly-estimator cost of 0.001, Pandora’s Router nearly matched g-only routing while incurring lower inspection cost. At 0.1, it made no g queries and matched f-only in routing performance and cost.
The auction version exposed a different trade-off
The paper applies the same value-of-information logic to a decentralized posted-price setting. In a leave-one-out evaluation, one specialist acted strategically while competitors supplied the posted price; the strategic bidder could refine its self-assessment before accepting. Results were compared with an oracle that had free refined information.
In the standard setting, Pandora’s Bidder stayed close to whichever of two static policies had lower regret: using f only or always using g. It refined frequently when the extra estimate was cheap, while defaulting or declining more often as the cost rose.
The trade-off became clearer when competing specialists used the noisy cheap estimator f to set prices. Individual surplus regret improved, but efficiency regret increased in RAG and EmbedLLM. In practical terms, the strategic specialist’s own payoff could improve while the overall allocation became less efficient.
What the tests do not settle
The evidence came from benchmark data in the three tasks and a leave-one-out posted-price auction. That design shows how the rules behaved in the tested settings, but it does not establish causal superiority in a live deployment.
The paper flags three main constraints: its Gaussian signal assumption, its restriction to two estimators—f and g—and the Bidder’s myopic leave-one-out calculation, which does not anticipate future strategic bids.
The work is listed as arXiv:2608.20316v1, dated 20 August 2026.
Paper data and sources
Original title: Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation
Authors: Adam Fisch, Shubhendu Trivedi, Fantine Huot et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text