An AI controller designed to decide when a reasoning system should keep working came close to the accuracy of a much larger fixed budget while using a fraction of the generated work. In a frozen-threshold online test on 300 untouched GSM8K questions, AERA scored 92.61%, compared with 93.01% for a fixed 128-response run. It averaged 4.44 responses per question, with a reported 96.53% saving in response count and 95.99% saving in completion tokens.
The gap was small but uncertain: AERA's paired score difference was -0.41 percentage points, and the 95% bootstrap interval ran from -1.79 to +0.80 points. The threshold was set at 0.9 after a 50-question calibration set, with the least-token threshold meeting the stated accuracy constraint selected before generation. The result points to a close accuracy and compute trade-off in that reported protocol, rather than a settled claim of higher accuracy.
The problem with the current answer
The study asks a practical question: when is another unit of reasoning useful, rather than simply how to reduce computation? AERA summarizes the cumulative prefix of generated responses using answer-distribution, timing, re-solving, semantic and compute features. At each checkpoint, meaning a point after a block of responses, it chooses whether to stop or allocate the next block, while excluding future correctness from the information available at decision time.
Its continuation label defines a potentially recoverable checkpoint as one where the current answer is incorrect but at least one later checkpoint becomes correct. That distinction matters because correctness did not move in one direction as reasoning continued. In the GPQA analysis, there were 42 recoveries and 14 collapses. Across six-checkpoint GSM8K trajectories, the 1,319 questions included 33 monotonic recoveries, 11 oscillatory recovered cases, one monotonic collapse and six oscillatory failures.
The evidence signals used to judge the current response could also move against the eventual answer. Among the 14 GPQA collapses, six showed a larger majority ratio, seven showed lower entropy and five showed higher semantic consensus. One case improved on all three measures while becoming incorrect. Conversely, four recoveries came with a falling majority ratio, six with rising entropy and 33 with falling semantic consensus. The paper presents this as an empirical distinction, not a causal claim.
The comparisons are more mixed
The main evaluation used offline sequential replay. Complete pools of responses were generated before the controller was evaluated, and the replay policy was allowed to see only the current checkpoint state when making each decision. This produced a common set of pre-generated reasoning trajectories for comparing different stopping and allocation policies.
In the descriptive offline comparisons, checkpoint-aligned ESC used 20.30 responses per question on GSM8K and 115.80 on GPQA, reaching 94.31% and 38.85% accuracy. ASC used 9.40 and 81.82 responses, with 94.43% and 38.99% accuracy. AERA reached a similar accuracy range with fewer responses than ESC, but its aggregate accuracy was not uniformly better than ASC.
AERA's stronger comparison came when random checkpoint assignments were matched to the same empirical stopping distribution. It exceeded the matched permutation by 1.18 percentage points on GSM8K, scoring 94.35% at 8.67 responses per question versus 93.17%, and by 4.50 points on GPQA, scoring 38.44% at 34.02 responses versus 33.94%. One-sided randomization tests reported a probability below 10 to the minus 4. The authors attribute the gain to question-specific routing rather than the stopping pattern alone.
A nested calibrated replay on GSM8K produced 94.28% accuracy at 7.88 responses per question, compared with 94.12% for Fixed-128. The paired difference was +0.17 points, with a 95% confidence interval from -0.17 to +0.50. Under the same constrained calibration, AERA and ASC were effectively indistinguishable: 93.72% versus 93.70% accuracy, at 4.48 versus 4.22 responses per question.
Where extra reasoning looked useful
A retrospective look at the stored trajectories offered a clue about what the gate was finding. At a threshold of 0.3, among questions AERA sent beyond checkpoint 4, 42.4% on GSM8K and 47.0% on GPQA had a higher-scoring later checkpoint. Among questions stopped at checkpoint 4, the corresponding figures were 0.9% and 33.3%. Mean best-future gains were 14.14 versus 0.09 points on GSM8K and 24.58 versus 1.78 points on GPQA. This was a hindsight diagnostic, not an untouched prospective endpoint.
The sharpest subgroup contrast appeared among 16 GPQA collapse questions. A nested out-of-fold gate using 55 features at a threshold of 0.3 reached 69.24% accuracy at 17.25 responses, compared with 36.98% at 110.0 responses for ASC. The paired difference was +32.25 points, with a 95% bootstrap interval from +15.89 to +49.59. On 16 GSM8K collapse questions, the difference was +2.91 points, with an interval from -0.53 to +7.91, at 82.5 versus 92.5 responses. These small subsets do not establish global superiority.
A one-shot version of the predictor, which made its allocation decision all at once, collapsed toward checkpoint 4 under severe target imbalance and was substantially less accurate than the sequential design. The ablation was consistent with the value of repeatedly re-observing the reasoning trajectory before deciding whether to stop.
The strongest result is still narrow
The document is an arXiv preprint, version 1, dated 28 August 2026. Its clearest result is specific: on the reported GSM8K online test, AERA came close to Fixed-128 accuracy while using far fewer responses and completion tokens. The mixed replay comparisons, including the near tie with ASC under constrained calibration, leave open how broadly that advantage would hold.
Paper data and sources
Original title: AERA: Adaptive Evidence Residual Allocation for Efficient Test-Time Reasoning
Authors: Ziming Wang, Ivor Tsang, Hangwei Qian
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-28
DOI: Not available
Original paper · Full text