A preprint reports that an AI assistant called REVOIR achieved higher answer correctness while asking fewer clarifying questions in benchmark tests, especially when users could correct it. The system was evaluated on ambiguous question answering and preference-aligned household task planning, where it had to decide whether to seek more information or act.
A policy built around the cost of asking
REVOIR is an inference-time hierarchical policy. It updates its beliefs about user intent, evaluates the value of asking and acting, then chooses a cost-adjusted ask-or-act branch. In plain terms, it weighs the possible value of a question against the option of acting immediately. The comparison included prompting, reasoning, fine-tuning and information-gain baselines.
The first benchmark, CondAmbigQA, contains 2,000 ambiguous queries. Each query is paired with conditions representing possible latent user intents. A rubric-based large language model judge assesses final-answer correctness, while questions and corrections are counted as clarification effort.
On CondAmbigQA, the authors report that REVOIR beat all ReAct variants in correctness when the agent controlled termination. In a GPT-5.4-mini comparison, REIGN and Entropy Thresholding required 1.3 to 2.8 times as many questions as REVOIR. When the user controlled termination, REVOIR outperformed all baselines in correctness and used the fewest non-ReAct clarifications.
The paper also compares accuracy and effort at different trade-off settings. It reports that REVOIR outperformed baselines across most settings once the effort-to-correctness trade-off ratio rose above 0.008 for agent termination and 0.002 for user termination. When clarification was treated as cost-free, represented by a ratio of 0, REVOIR ranked second.
The ability to respond to corrections was a notable part of the test. The paper reports that only REVOIR reduced both the number of questions and the total number of clarifications when user corrections were available. Other methods increased both measures.
The gains carried over to a second benchmark
On ADAPT, the benchmark for preference-aligned household task planning, the evaluation used 16 personas per split, including 12 seen and 4 held-out personas, across 8 breakfast tasks and 4-fold cross-validation. Results were reported as a preference satisfaction rate, or PSR.
With few-shot in-context learning, REVOIR reached a PSR of 58.8% on seen personas and 56.0% on unseen personas. It exceeded Always-Ask by 6 points and Reflection-DPO by 13 to 15 points, while asking five times fewer questions than Reflection-DPO.
In the zero-shot evaluation, REVOIR achieved a PSR of 51.5% on seen personas and 49.5% on unseen personas. It virtually matched Always-Ask while asking 11 times fewer questions.
A promising benchmark result, with a clear boundary
The paper also reports agreement between human and simulated-user verdicts. Overall agreement was 0.809, with a reported 95% confidence interval of 0.72 to 0.88. For episodes accepted by the simulator, agreement was 0.901, with a 95% interval of 0.81 to 0.95.
The findings do not yet establish how the method performs with real users. The authors call the one-step value-of-information estimate myopic, meaning it does not look far ahead, and note that REVOIR does not support dual-control settings or fully interleaved questions and non-terminal actions. Real-user evaluation is identified as future work.
The manuscript is an arXiv preprint dated 29 September 2026.
Paper data and sources
Original title: Rational Clarification by Assistive Agents via Value-of-Information Reasoning
Authors: T. Duy Nguyen-Hien, Yee Whye Teh, Wee Sun Lee, Tan Zhi-Xuan
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text