A rubric-based training variant recorded the highest reported average scores in two groups of tests among vision-language models built from a Qwen3-VL-8B-Instruct starting point, according to a preprint. The lead was not consistent across every benchmark: answer-level reinforcement learning remained stronger on several individual measures.
The study compares a supervised fine-tuning checkpoint, answer-level reinforcement learning and reinforcement learning with rubric-based feedback. Its question is whether rubric-based rewards are associated with stronger visually grounded reasoning while preserving the model's general capabilities.
How the comparison was set up
The rubric-based approach converts fine-grained visual rubrics into scores for each training item and feedback assigned to prefixes of the generated answer. That gives the training process information about earlier portions of a response, rather than relying only on one score for the completed answer.
The training stack began with Qwen3-VL-8B-Instruct fine-tuned on the public OpenMMReasoner-SFT-874K corpus. The answer-level and rubric-based GRPO variants shared that supervised starting point, the same 50,248 V-Rubrics examples, the same rollout group size and the same maximum training horizon. They differed in mini-batch sizes and in how rewards or credit were assigned.
The fixed training set drew on 17 canonical sources. It included 18,121 hard examples, 25,306 medium examples and 6,821 simple examples. Across the set, the researchers counted 352,938 rubric items: 209,436 for visual faithfulness, 101,369 for reasoning consistency and 42,133 for instruction following.
Every rubric annotation was generated by Gemini-3-Pro under one structured, image-conditioned protocol. During training, Qwen3-VL-235B-A22B served as the language-model judge, using separate prompts to check whether an answer was equivalent to the reference and to verify rubric criteria.
What the scores showed
Across the general vision-language and knowledge benchmarks, the rubric-based GRPO variant had a Knowledge Average of 61.88 and an Overall Average of 68.04. The answer-level variant recorded 59.35 and 66.25, while the supervised fine-tuning checkpoint recorded 58.31 and 64.93.
The rubric-based variant also had the highest reported Overall Average in the group covering visual mathematics, charts and logic. Its score was 62.45, compared with 61.94 for answer-level GRPO and 58.45 for the supervised checkpoint.
In a separate credit-assignment comparison, the reported Overall Average was 66.25 for answer-only sequence-level credit, 67.74 for answer-plus-rubric sequence-level credit and 68.04 for answer-plus-rubric component-plus-prefix credit. The comparison does not isolate prefix localization alone because the two rubric variants differ jointly in component standardization and localization.
The evaluation used VLMEvalKit across 10 benchmark families, with standard accuracy and unweighted averages over the displayed metrics and splits. The study reports point estimates without confidence intervals or significance tests, so the figures are score comparisons rather than statistical tests of reliability.
The lead varied by benchmark
Answer-level GRPO remained stronger on MMBench-Dev, MathVerse V/O and CharXiv reasoning. The reported advantage for the rubric-based variant was therefore not uniform across metrics.
Questions the results leave open
The authors report limitations involving the quality of automatically generated rubrics and judge decisions, the use of approximate fuzzy prefix localization, and possible bias when Qwen-family judges evaluate Qwen-family policies.
Prefix localization is practical local feedback rather than exact token-level supervision. The analysis leaves open how well the approach would transfer beyond the evaluated models and benchmarks.
The document is arXiv:2608.25580v1, dated 26 Aug 2026, and is identified as a preprint. The study reports support from Singapore's Ministry of Education through MOE AcRF Tier 2, grant MOE-T2EP20223-0002, plus cash and in-kind funding from NTU S-Lab and industry partner(s).
Paper data and sources
Original title: V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning
Authors: Shulin Tian, Minglun Li, Yuhao Dong et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text