A branch-based AI training method reported higher scores than ARPO on the paper's mathematical and search tests. Under matched rollout budgets, its five-task mathematical macro-average Pass@1 - the paper's one-answer accuracy measure - was 66.0% with the 1.7B model scale and 69.3% with the 4B scale, 1.7 and 2.2 percentage points above ARPO, respectively.
On the five search tasks, the corresponding averages were 53.2 and 57.5 at the two model scales, 1.8 points above ARPO at both scales. No confidence intervals or inferential tests are reported.
The method separates two jobs
The paper focuses on two choices in training a tool-integrated language model: how to spend a fixed rollout budget - in plain terms, how many alternative response paths to try - and how to turn the results of those paths into token-level credit. Its research question is whether handling those choices separately can improve policy optimization.
CBPO identifies candidate points across a full response and allocates its branch budget with path-level and node-level decay. It compares branches that share an exact prefix, meaning the same text up to a point, and uses variation in their outcomes to construct token-level credit through Contrastive Branch Value, or CBV.
Training uses bounded outcome rewards. Mathematical tasks use binary correctness rewards. Search tasks use normalized token-level F1 or an LLM-as-a-Judge score, and every reward is bounded between 0 and 1.
A focused set of comparisons
The evaluation covered five tool-augmented mathematical benchmarks and five knowledge-intensive search benchmarks at two model scales. The paper does not report the number of individual benchmark instances or the total number of training rollouts.
CBPO ranked first or tied for first on all five mathematical benchmarks at the 1.7B scale and on four of five at 4B. It led four of five search benchmarks at each scale, although it trailed EAPO by 0.5 points on MuSiQue.
A second check using Pass@1, Pass@3 and Pass@5 used five independent samples per problem. At the smaller model scale, the macro-average scores were 66.0%, 74.6% and 77.9%; at the larger scale they were 69.3%, 76.9% and 80.2%. CBPO exceeded ARPO at every reported value.
What changed inside the method
In component ablations under matched configurations, the mathematical macro average was 1.5 points lower when either decay term was omitted, at both scales; 2.2 points lower when both were omitted; and 3.0 and 2.9 points lower when CBV was omitted at 1.7B and 4B. The paper presents these as comparisons among the configurations tested.
During training, CBPO attained a higher final mean reward than GIGPO and ARPO while averaging fewer tool calls per trajectory.
Configuration tests used total rollout budgets from 4 to 16. The macro average was higher at the larger reported budget, and the highest reported average came with six initial trajectories and 10 branches for both backbones. The analysis does not establish that this setting is generally optimal beyond the budgets tested.
The boundaries of the result
The full-response analysis examined three Qwen3-1.7B MATH trajectories and was a qualitative mechanism check, not population-level or causal evidence. A controlled ablation limited to tool boundaries is still needed to isolate the contribution of scanning the full response.
The reported rank comparisons apply only to the listed benchmarks and model scales, while the configuration findings apply only to the rollout budgets tested. Benchmark accuracy does not establish source reliability or social neutrality.
The study used public benchmarks and collected no new personal or human-subject data. It also notes that open-web retrieval can expose models to inaccurate, biased, offensive or privacy-sensitive material.
The document is an arXiv preprint, version v1, dated 25 August 2026. No funding statement is reported.
Paper data and sources
Original title: Contrastive Branch Policy Optimization
Authors: Ying Wang, Changlin Qiu, Bang Lin et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: Not available
Original paper · Full text