A way of deciding when to intervene in a language model's reasoning recorded 90.1% accuracy on questions with enough evidence in one hosted-model test, compared with 82.9% when reflection was imposed at a fixed interval. It used 0.60 interventions per item instead of 1.00. But across six open-weight models, the white-box version had higher accuracy on ambiguous questions in all six and lower accuracy on questions with enough evidence in five.
The work is an arXiv preprint dated 26 August 2026.
The timing problem
The study's central question was when a developing reasoning path contains enough evidence to justify a bias-correction intervention. Its method, Adaptive Triggering, tracks a per-step bias-risk signal and accumulates the evidence with CUSUM, a running statistical measure. It inserts targeted in-context reflection only after the accumulated evidence crosses a calibrated threshold.
The monitor can use a white-box probability signal from the model or a black-box signal supplied by an LLM judge. The experiment compared four conditions: no intervention, generic reflection at a fixed interval, and adaptive triggering with either signal.
Testing the idea on BBQ
BBQ, the Bias Benchmark for QA, pairs ambiguous contexts that lack enough information with disambiguated versions that supply it. The authors found four undocumented schema inconsistencies that could bias the benchmark calculation, and state that corrected scoring logic has been released.
The analysis covered seven open-weight models, with six producing verified results across all nine BBQ categories, plus a stratified 504-item subsample from hosted gpt-4o-mini. For the open-weight tests, decoding was greedy at temperature zero. Non-completions were excluded from accuracy calculations, while paired accuracy comparisons used McNemar's exact test and category-stratified 95% bootstrap confidence intervals.
A selective black-box check held up better
In the hosted model's self-judge evaluation, disambiguated accuracy was 92.1% with no intervention, 82.9% under fixed-interval reflection and 90.1% under adaptive black-box triggering. The adaptive condition averaged 0.60 interventions per disambiguated item, versus 1.00 under the fixed schedule.
The adaptive-versus-fixed comparison was statistically significant, with p = 0.0003. The adaptive-versus-no-intervention comparison was not statistically significant, with p = 0.13.
The disambiguated result also appeared with an independent judge and 252 items per split. Adaptive black-box triggering recorded 89.7% disambiguated accuracy, 93.7% ambiguous accuracy, a 5.2% bias score and 0.163 interventions per item. Its reported difference from fixed-interval reflection on disambiguated items was 6.75 points, with a 95% confidence interval from 2.78 to 10.71 and p = 0.0015. The ambiguous-item improvement did not replicate with the independent judge.
The signal mattered as much as the schedule
Across six open-weight models, adaptive white-box triggering had higher ambiguous-item accuracy in all six but lower disambiguated-item accuracy in five. Ten of the 12 paired comparisons were significant at p < 0.001; the disambiguated change for DS-Llama-8B was the exception.
Firing behavior varied sharply across models even under the same N = 5 fixed schedule. Rates ranged from 0% on Gemma-9B to 99.0% on DS-Llama-8B. At roughly the N = 5 firing rate, adaptive black-box triggering had higher accuracy and lower ambiguous-item bias; only N = 2 reached higher accuracy at roughly twice the intervention rate.
The white-box signal's problem was a mismatch between what it detected and what the correction was meant to fix. It aligned with unsupported stereotype reliance on ambiguous items, but could also flag an answer that matched a stereotype because the question's evidence supported it. In the full analysis, white-box intervention changed 3,112 correct disambiguated answers into incorrect ones, compared with 761 repairs of incorrect answers, a 4.09-to-one ratio.
Detector-specific calibration did not remove the white-box pattern. Selected and default white-box settings produced identical held-out traces, trigger points and answers. White-box firing ranged from 21.8% to 26.6%, compared with 8.7% to 34.9% for the black-box signal.
Accuracy was not the only trade-off
Intervention also carried completion and compute costs. On DS-Llama-8B disambiguated items, non-completion was 8.5% when adaptive white-box triggering fired and 4.5% when it did not. Counting those non-completions as incorrect would have changed the apparent accuracy loss from 0.1 to 2.3 percentage points. With adaptive black-box triggering, total generation calls were 10.1 rather than 5.1 per Qwen item and 4.5 rather than 2.7 per Gemma item.
The hosted result came from a self-judge evaluation, and the ambiguous-item improvement did not replicate with an independent judge. Fired and non-fired items were not randomly assigned, so the non-completion contrast cannot be interpreted as a causal effect of triggering.
At full nine-category scale, complete-case analyses for the six verified open-weight models used 13,070 to 15,686 items per split. The authors state that code and evaluation resources are available with the corrected scoring logic. A remaining question is whether a detector can distinguish unsupported stereotype reliance from correct evidence-supported reasoning.
Paper data and sources
Original title: Adaptive Triggering for Bias Correction in LLM Reasoning
Authors: Nayoung Kim, Mickey Mancenido, Huan Liu
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text