A robot-planning approach that gathers visual evidence before task planning was associated with higher full-recipe completion when instructions left important details unstated, and with higher halt rates when a required object was missing, according to an arXiv preprint.
A stop signal for uncertainty
The approach, called EAFG, asks a vision-language model — an AI system that works from instructions and visual input — to generate exploratory subgoals. The robot executes those subgoals through task-and-motion planning, gathers new observations and only then plans the requested task.
A feasibility gate can then mark the task ready to plan, ask for more evidence or halt. Exploration is restricted to reversible operations, while direct task actions are excluded until feasibility has been assessed.
If a required object is still unobserved after plausible source locations have been inspected, EAFG halts rather than assuming that the object exists.
Mixed results when the task was explicit
The researchers compared VLM-TAMP with and without EAFG in the reported task settings, using GPT-5.5 and Gemini-3.5-Flash. Each condition used 20 robot execution runs.
When instructions were explicit and the needed items were present, GPT-5.5 completed the full recipe in 20% of runs without EAFG and 45% with it. Gemini-3.5-Flash went from 15% to 10%. With EAFG, pepper and salt step completion reached 100% for both models, although GPT-5.5's water-step rate was 80% without EAFG and 55% with it.
The pattern was therefore not a uniform gain: the full-recipe rate fell for Gemini-3.5-Flash, and the GPT-5.5 water step also performed worse with EAFG in this setting.
The difference was clearer when instructions were incomplete
An underspecified instruction leaves some task-relevant information unstated. In that setting, GPT-5.5 completed the full recipe in 5% of runs without EAFG and 40% with it. Gemini-3.5-Flash recorded 0% without EAFG and 20% with it.
The step-level results also favored EAFG in this setting. For GPT-5.5, the reported salt and pepper rates were 10% and 30% without EAFG, compared with 100% and 100% with it. For Gemini-3.5-Flash, the two reported seasoning rates were 5% and 5% without EAFG, compared with 95% and 85% with it.
When the carrot was not there
In the explicit-but-missing setting, the absent object was a carrot. GPT-5.5's halt rate was 45% without EAFG and 90% with it; its mean number of attempts involving the carrot was 4.00 and 0.55, respectively. For Gemini-3.5-Flash, the halt rate was 40% without EAFG and 100% with it, while mean attempts were 2.40 and 0.00.
That behavior follows the framework's decision rule: once plausible locations have been checked and the object remains unseen, the feasibility gate can stop the robot rather than treat the object as present.
What the test leaves unanswered
The supplied analysis describes these as descriptive comparisons, not causal estimates. It reports no inferential tests, confidence intervals or p-values, and every condition used 20 runs, so the percentages should be read as results from the tested runs rather than broad performance guarantees.
There is also a practical failure mode. EAFG does not explicitly recover when an exploratory manipulation fails; if TAMP cannot execute an evidence-acquisition subgoal, the feasibility gate may be left with too little evidence to judge the task.
Taken together, the results suggest that gathering task-relevant evidence before planning may help in the tested cases, particularly when instructions are incomplete or an object is absent. The preprint does not establish performance beyond the reported settings or uniform improvement across models and outcomes.
Paper data and sources
Original title: Evidence-Gated Task and Motion Planning with Vision-Language Models
Authors: Tsunehiko Tanaka, Matthew Stephenson, Alistair Macvicar, Edgar Simo-Serra
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text