A system designed to choose different large language models at different points in a task reported strong results while staying within long-term operating-cost budgets across four benchmark streams. Called ProgRouter, it routes work step by step, using progress signals to balance progress gain against time and cost. The findings are reported in an arXiv version 2 preprint dated 30 Aug 2026.
Choosing a model as the task changes
The paper tackles a central question for multi-agent LLM workflows: how to route agents dynamically across workflow steps while preserving task-solving quality under operating-cost budgets. It characterizes fixed single-model policies as offering an unfavorable quality-cost tradeoff. ProgRouter combines multi-view progress scoring, dual-path progress prediction with adaptive meta-gating, and online step-wise routing.
The progress predictor was trained online from realized task-progress feedback collected during workflow execution. The router uses multiple progress views and its dual-path predictor to guide step-wise LLM selection while balancing progress gain with time and cost constraints.
Evaluation covered 164 HumanEval Plus tasks, the complete set, plus randomly sampled subsets of 200 MBPP tasks and 200 MATH-500 problems, along with 100 randomly sampled ASQA questions. Each benchmark task set was shuffled with a fixed random seed, and each task was processed once without replay.
What the benchmarks showed
GPU energy was measured with NVIDIA's NVML system at 100-millisecond intervals. The study paired that energy measure with task scores and execution time in its reported benchmark results.
On HumanEval Plus, ProgRouter reported a 93.0% pass rate, 4,796 joules of energy use, and a 13.7-second execution time. On MBPP, it reported a 79.4% pass rate, 3,376 joules, and 10.3 seconds. On MATH-500, the reported pass rate was 84.3%, with 6,112 joules of energy use and a 19.0-second execution time.
On ASQA, ProgRouter reported 92.1% citation precision, 18,373 joules of energy use, and a 61.6-second execution time. The quality endpoint in that benchmark was citation precision.
An offline comparison of predictor variants reported the lowest mean absolute error, or MAE, for the dual-path predictor with a meta-learner: 0.0720. Lower MAE indicates more accurate progress prediction in the paper's measure. The other reported values were 0.0967 for the structured-only path, 0.0788 for the semantic-only path, and 0.0843 for the mean-combined paths.
The online routing ablation reported a 93.0% pass rate and 4,796 joules for the full system. Removing the predictor, multi-view scorer, budget penalty or queue penalty produced reported pass rates of 89.0%, 90.2%, 87.8% and 92.7%, respectively. A naive progress-per-cost strategy reported a 17.7% pass rate while using 7,797 joules.
The evidence has clear boundaries
The results come with a measurement caveat. No inferential tests, confidence intervals or repeated-run variability were reported, so the supplied analysis does not quantify statistical uncertainty or run-to-run stability.
The scope is also limited. The authors note that generalization to open-ended web navigation and tool-augmented question answering remains empirically unverified. The evidence therefore speaks to the four benchmark settings used here, rather than establishing how the router would perform in those broader settings.
For now, the preprint presents ProgRouter as a way to manage the quality-cost tradeoff in multi-agent LLM workflows. Its reported combination of progress-aware routing, cost constraints and benchmark performance is the paper's main result, while broader performance remains an open question.
Paper data and sources
Original title: ProgRouter: Online Progress-Guided Orchestration for Multi-Agent LLM Workflows under Quality-Cost Tradeoffs
Authors: Somgyuan Li, Ahmed M. Abdelmoniem, Shiqiang Wang
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text