A result that changed with model size
An AI training method that lets its teaching policy adapt during self-distillation was associated with substantially higher mathematics benchmark scores from Qwen3-8B than OPSD. DualOPSD's avg@12 was 23.61 percentage points higher than OPSD on AIME 2024, 13.89 points higher on AIME 2025 and 10.00 points higher on HMMT February 2025.
The result was not consistent across sizes. Relative to OPSD at 100 steps, DualOPSD was higher by 13.06, 5.00 and 6.67 points at 4B on the three benchmarks, but lower by 3.61, 1.94 and 1.94 points at 1.7B. The direction of the comparison therefore reversed at the smallest tested scale.
A separate comparison measured how much the 8B and 1.7B versions gained over Base. The paired differences were 31.11 points on AIME 2024, 22.78 on AIME 2025 and 16.39 on HMMT February 2025. Paired bootstrap 95% intervals ran from 20.00 to 42.22, 12.78 to 33.61 and 6.39 to 27.22 points, respectively; all excluded zero.
How the teacher works
The study's central question was whether the privileged teaching policy should learn during on-policy self-distillation. DualOPSD is built as a closed loop: the student first receives clipped forward-KL supervision from a privileged teacher, then the teacher is updated with full-vocabulary reverse KL toward the updated student on the same trajectory. The adapted teacher supplies targets for later updates without generating a separate teacher rollout or using a correctness verifier.
Each optimization step used a batch of 32 prompts, one student rollout per problem and a maximum of 1,024 tokens. Student and teacher were then updated in sequence using the same token IDs and mask.
An ablation tested a lower-clipped teacher update against the full reverse-KL version. At step 100, the full version had a mean avg@12 score 2.87 points higher, with differences of 1.39 on AIME 2024, 1.67 on AIME 2025 and 5.56 on HMMT February 2025.
A cleaner output, but a cost and a warning
Accuracy was not the only difference. Pooled truncation was lower with DualOPSD at each scale: 2.31% versus 9.63% for OPSD at 4B, 1.20% versus 3.24% at 1.7B, and 3.33% versus 8.43% at 8B.
The paper also reported a KL diagnostic comparing the models' token probabilities. At step 100, DualOPSD's teacher-to-student value was 0.171 nat/token, reported as 50.2% below OPSD; the student-to-teacher value was 0.064, or 45.9% below OPSD. The paper notes that lower KL is expected from the teacher objective, so this limited diagnostic alone is not a verdict on the method.
There was a measurable training-time trade-off. On one A100, the 4B run took about 1.00 hour for OPSD and 1.40 hours for DualOPSD. The paper attributes the gap to an extra teacher forward and backward pass, and notes that wall time on one GPU is not a hardware-independent measure of algorithmic FLOPs.
A narrow test, not a general verdict
The test covered one model family: Qwen3 models at 1.7 billion, 4 billion and 8 billion parameters, all in non-thinking mode, trained on 29,434 mathematics problems and solutions. Within each scale, every method started from the same base checkpoint.
Evaluation used 30 problems each from AIME 2024, AIME 2025 and HMMT February 2025, with 12 sampled responses per problem. The same 90 competition problems were used at every model scale, and the main score, avg@12, measured accuracy across those responses.
That narrow design matters. The evidence comes from one training seed, while the fixed-prefix KL check covered only eight 4B examples. The 8B result needs replication before it can support a broader claim about adaptive teaching.
The work is an arXiv version 1 preprint dated 26 August 2026. Its central message is conditional: adaptive teaching tracked with better benchmark results at the two larger tested scales, while the smallest model moved in the opposite direction.
Paper data and sources
Original title: DualOPSD: Adaptive Privileged Teachers for On-Policy Self-Distillation
Authors: Yutong Chen, Guangfu Guo, Zhichao Xu, Kunpeng Liu
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text