A tool-using multimodal AI system scored higher than a supervised-fine-tuning baseline when it had to navigate synthetic mazes under unseen rules. The method, called Disentangled Multimodal Planning, recorded an Exact Match score—the share of cases in which the full trajectory matched the shortest valid path—of 90.0% in RegularMaze, compared with 70.3% for the baseline. In QuestMaze, the figures were 88.0% and 56.3%.
A maze built around rules
RuleMaze was procedurally generated as pairs of maze layouts and natural-language rules, with executable validators checking whether a proposed route complied. The benchmark kept cases with at least one compliant route and a unique rule-consistent solution, even when distractor trajectories were present.
RegularMaze and QuestMaze each contained 3,600 training samples and 400 test samples. Each test set included 100 seen-rule cases and 300 unseen-rule cases, with the unseen cases divided evenly among easy, medium and hard levels.
The test rules were presented in natural language. Unseen rules were defined as rules whose logical formalizations were absent from training, while Precision Rate measured the proportion of consecutively correct steps.
Splitting the planning job
Disentangled Multimodal Planning divides the task among perception, execution and rule-verification tools, with a controller trained to orchestrate them. The design makes those components interpretable and replaceable, allowing the system to process the maze, carry out actions and check compliance as separate parts of the plan.
Training used teacher-forced supervised fine-tuning over complete reasoning and tool-call traces for 15 epochs, with batches of 32 examples.
On RegularMaze's unseen rules, Direct SFT reached 65.3% EM and 71.4% PR. Coordinate representations reached 69.0% and 73.7%, while ASCII representations reached 67.0% and 70.6%; DMP reached 90.0% EM and 92.7% PR.
The higher scores held on harder tests
In RegularMaze, DMP reached 98.0% EM and 98.4% PR on seen rules. On unseen rules, it reached 90.0% EM and 92.7% PR; its EM was 95.0% on easy rules, 88.0% on medium rules and 87.0% on hard rules.
In QuestMaze, DMP reached 91.4% EM and 94.3% PR on seen rules, and 88.0% EM and 91.7% PR on unseen rules. For the hardest rules, its EM was 84.0%, compared with 40.0% for the supervised-fine-tuned baseline.
What the component tests found
Versions of the system with components removed had lower unseen-rule averages than full DMP. Without execution, EM and PR were 38.0% and 41.2%; without verification, they were 80.6% and 82.4%; without perception plus verification, they were 77.0% and 79.1%, compared with 90.0% and 92.7% for full DMP.
The gap was also present at different training-data scales. With 25% of the training data, DMP reached 66.0% EM and 75.8% PR on unseen rules, versus 27.3% and 39.3% for Direct SFT. With 100% of the data, the corresponding scores were 90.0% and 92.7% for DMP, versus 65.3% and 71.4%.
A controlled test with clear limits
The benchmark depended on executable validators. They were reported as 93.3% correct before manual verification, and all validators were manually inspected and verified before deployment.
The work is an arXiv preprint, and its evidence is limited to a synthetic, controlled grid-maze abstraction. The authors note that the setup does not represent perceptual noise, continuous dynamics or partially observable environments. Validator-based scoring also assumes that rules can be formalized precisely, while iterative tool use may add computational overhead.
The reported figures are descriptive benchmark percentages; no confidence intervals or inferential tests were reported. The results therefore describe performance on the listed test sets without a formal uncertainty estimate around the differences.
Paper data and sources
Original title: Rule-Compliant Visual Spatial Planning for Multimodal Large Language Models
Authors: Yu Chen, Ting Lei, Yaoyi Li et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text