An automated system that turns spatial question-and-answer examples into large synthetic datasets reported a 28.2% overall score on MMSI-Bench, including an 8.6-percentage-point gain on the benchmark's Cam.-Cam. measure over the base model, according to an arXiv preprint.
The framework is presented as an automated way to use spatial QA exemplars to synthesize large-scale, high-fidelity datasets without manual annotation or relying on a multimodal large language model's implicit spatial reasoning. For its main training runs, it used approximately 6,000 synthetic QA pairs for MMSI-Bench with Qwen2.5-VL-7B, and roughly 10,000 pairs for VSI-Bench with Qwen2.5-VL-3B.
The generation system uses two tracks and Qwen3-Coder-30B-A3B-Instruct for all of its agents. It separates their roles through prompts, contextual memory and track-specific tool access. The reported efficiency figures say code synthesis averaged three minutes for the camera track and two minutes for the QA track across two GPUs, after which data execution scaled linearly with simulated scenes without further language-model intervention.
Gains across several tests
On VSI-Bench, the 3B model scored 42.9% overall. The paper reports gains of 7.6 percentage points overall and 6.2 points on zero-shot Route Plan. These are reported differences between benchmarked model configurations, so they should not be read as proof that the pipeline alone caused the gains.
The paper reports gains on two other tests. On SpaCE-10, the overall improvement was 8.9 points, including 18.2 points for Size Assessment and 16.1 points for Function Reasoning. On ViewSpatial-Bench, the overall boost was 8.5 points, including 23.5 points for person-perspective Object View Orientation, 10.7 for Scene Simulation Relative Direction and 6.0 for Person-perspective Relative Direction.
The role of the agent layout was tested in a 100-example ablation. The one-agent configuration reached 34.0% QA-generation success and 60.0% camera-trajectory success, while the four-agent configuration reached 92.0% and 84.0%. That comparison shows a difference between the tested setups; it does not by itself isolate role separation as the cause.
The generator itself also figured in the comparison. Under the same four-agent pipeline, replacing Qwen3-Coder-30B with Qwen3.5-9B was associated with QA-generation success falling from 92.0% to 48.0% and camera-trajectory success from 84.0% to 56.0%. In a separate comparison using 10,000 examples, direct Gemini3.5-Flash annotation produced an average score of 28.2, versus 35.3 for the base model and 42.9 for Exemplar2VQA. On the report's Appr. Order measure, the corresponding scores were 6.3, 35.0 and 40.5.
An extended OSR-Bench test reported 36.5% overall for Exemplar2VQA, with 59.0% on Object Count and 41.1% on Relative Distance. The main benchmark figures are reported as point estimates. A separate MMSI-Bench rerun across three random seeds produced an overall mean of 29.33, with a standard deviation of 0.99. That check addresses variation across seeds, not every source of uncertainty in training or evaluation.
A simulator-based starting point
The approach has a narrow operating range. It is optimized for static, object-centric configurations and requires explicit simulator scene metadata. Raw real-world photographs would therefore need additional upstream models before the framework could use them. The method's absolute generation success is also bounded by the coding capabilities of its foundational language model and by edge cases in the physics engine.
The findings therefore concern synthetic data generation and benchmark performance under the stated settings. The checklist says the code will be open-sourced upon publication, while the supplied document is an arXiv preprint, version 1, dated 29 September 2026. The evidence remains bounded by the simulator-based design and static tasks described in the paper.
Paper data and sources
Original title: Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Coding
Authors: Jiayu Ying, Qijian Tian, Ruijie Xu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text