The idea is to give the model a spatial reference
The preprint describes InsightMap as a model built around top-down maps used as explicit spatial memory. It links historical views to labeled locations on the map, then jointly learns which navigation action to take and how the map should look after that action. The basic idea is to connect partial observations to a persistent spatial reference.
That map is not generated during the navigation test. During inference, InsightMap decodes only action text; future-map generation is auxiliary training supervision and is not performed during navigation evaluation. The navigation scores therefore reflect action prediction under the reported setup, rather than live map generation at each step.
Training used an 800K-example multimodal corpus, with 65% of examples devoted to navigation and the remaining 35% to visual question answering and grounding. The pipeline first ran one epoch of navigation-only training to collect expert-guided corrections, then jointly trained the full corpus while keeping the VAE frozen.
Strong results in simulated environments
Simulation tests used the validation-unseen splits of R2R-CE and RxR-CE in continuous Matterport3D environments through the Habitat simulator. On R2R-CE, the model reported 64.7 for oracle success rate, 56.9 for success rate, 4.89 metres for navigation error and 50.7 for success weighted by path length. The paper describes the first two as the best listed values and the last two as second-ranked.
On RxR-CE, it reported 5.58 metres for navigation error, 54.9 for success rate, 46.1 for success weighted by path length and 62.8 for normalized dynamic time warping. These are described as the strongest values among the listed methods.
Those rankings need context. The comparison covers methods with different sensor inputs, training data and waypoint predictors, and no confidence intervals or hypothesis tests are reported. The figures are point results under the stated benchmark setups, not a controlled guarantee that the same ordering would hold elsewhere.
The same representation carried into other tasks
The map-based system was also tested on static-scene reasoning. It reports a CIDEr score of 103.7 on ScanQA validation and 60.1% exact-match accuracy on the SQA3D test set.
For ScanRefer, a 3D object-grounding task, accuracy after proposal matching was 59.7% at an IoU overlap threshold of 0.25 and 53.1% at 0.5. Those results were 15.9 and 32.3 percentage points above raw predictions, respectively. Because the final figures use detected object proposals and matching, they do not isolate the model from the proposal generator.
In component comparisons, the map-supervision variant reached a success rate of 56.9 and a path-length-weighted score of 50.7, with differences of 4.3 and 3.2 points over the comparison variant when generation-module trainability was held the same. The full task mixture also reported 56.9 and 50.7, gains of 7.7 and 6.1 points over C1. These ablation differences are informative, but they do not by themselves prove that map supervision alone caused the improvement.
Robot trials were promising but small
Real-world testing used a Unitree Go2 robot in hallway, lab and office settings. Each scenario had 20 trials, and a trial counted as successful when the robot reached within 2 metres of the goal within 50 steps. InsightMap reported success rates of 95% in the hallway, 90% in the lab and 65% in the office, exceeding NaVILA by 20, 20 and 45 percentage points.
The office result was lower than the other two, and the evaluation remained limited to 20 trials in each of three settings. The results do not establish how the method will perform with other robots, environments, sensors or instruction distributions.
One further boundary is that generated maps were training-only supervision in the navigation tests. Whether using them during inference would change navigation performance remains unanswered by this evaluation.
Paper data and sources
Original title: InsightMap: Structured Spatial Modeling for Embodied Multimodal Reasoning
Authors: Hongpei Zheng, Hujun Yin
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text