A preprint reports that LEGO-OPD, a method combining language-prior and visual-grounding signals, had the highest listed averages among the evaluated methods for both multimodal and text-only reasoning in Qwen3 experiments. At two tested calibration settings, it outperformed the evaluated comparison baselines on those averages. The result is based on benchmark accuracy averages, and no formal significance testing is reported.
Building the combined teacher
LEGO-OPD builds one teacher distribution, the combined guidance used by the student, from complementary factors supplied by independently selected experts. One factor provides a language prior and the other visual grounding. Its source-referenced calibration adjusts the visual update using the grounding expert’s source KL, a measure of update size, as a reference that changes with each generated prefix.
The main setup used Qwen3-VL-2B-Instruct as the student, Qwen3-4B-Instruct-2507 as the language expert, Qwen3-VL-4B-Instruct as the grounding expert, and a 54,840-example subset of MMFineReason-123K as multimodal training data. The reported result therefore comes from a defined model and data configuration.
The evaluation separated multimodal reasoning, text-only reasoning and visual perception. The pooled QBench1,2 measure used Group A questions from sets containing 1,495 and 1,000 questions. Except for perception, each question was sampled five times, and category averages were unweighted means across benchmarks.
What changed in the scores
At rho=1 and rho=2, LEGO-OPD reported multimodal averages of 45.84 and 46.28, respectively. Its text-only averages were 56.01 and 56.68. The paper reports that both settings exceeded H-OPD on all six multimodal reasoning benchmarks.
The two settings also differed on the visual-perception benchmarks. BLINK rose from 52.03 to 52.55, while QBench1,2 rose from 69.34 to 69.62 in the second setting. These are benchmark accuracies, not direct measurements of human visual perception.
Why the weighting matters
An ablation compared adaptive calibration with fixed coefficients. At rho=1, adaptive calibration recorded multimodal and text-only averages of 45.84 and 56.01, compared with 42.43 and 55.82 for fixed alpha=1, 43.22 and 55.02 for fixed alpha=2, and 41.60 and 54.12 for fixed alpha=4. Adaptive calibration had the highest reported average in both categories in this comparison.
The factor tests pointed in the same direction. With rho=1, the full Qwen3-4B and Qwen3-VL-4B composition scored 33.36 on MATH-Vision, 56.96 on MathVerse and 44.93 on OlympiadBench-MM. Replacing the visual residual with Student 2B produced scores of 33.03, 54.17 and 43.07; replacing the language prior produced 26.58, 52.69 and 37.20. The full composition had higher listed scores than either substitution variant.
An internal calibration diagnostic used 24,576 shared prefixes from 768 rollouts on 256 MMMU questions, checked at student steps 0, 100 and 200. The approximate calibration coefficient broadly tracked a numerical reference, but deviations were larger at larger source KL. That signal measures the magnitude of a grounding update, not whether the visual evidence is correct.
What the test cannot answer
The paper’s limitations narrow the claim. The visual residual is a conditional model contrast and may include information already present in generated prefixes. The composition also requires compatible models with shared tokenization and output vocabularies. The paper treats erroneous visual evidence as a failure mode.
The comparison also carries a reproducibility caveat: the paper says its paper-based attempt to reproduce Decomposed OPD did not reproduce that method’s reported improvements in the authors’ experimental setting.
The manuscript is labeled a preprint. Its reproducibility statement says code is available at an anonymous repository, while the supplied end matter does not report a funding source.
Paper data and sources
Original title: LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation
Authors: Jaeyun Shin, Hangeol Chang, Jong Chul Ye
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text