In PhotoChat tests, PURE-CORR, FIXED and MULTI-C had comparable image-retrieval performance. DCP-B was substantially worse on retrieval and showed very large shifts between consecutive grounding states during high-revision turns. The comparison therefore separates retrieval performance from the way a policy’s grounding state changes over a dialogue.
The study asks when a grounded dialogue model should preserve an established interpretation and when it should revise that interpretation as new evidence arrives. Here, grounding means the model’s running interpretation of what a conversation refers to. The question is especially difficult when an immediate mismatch conflicts with accumulated uncertainty and contextual signals.
A fixed test for changing interpretations
To make the comparison, the researchers kept the underlying architecture fixed while varying the revision policy across PURE-CORR, FIXED, DCP-B and MULTI-C. The CLIP backbone remained frozen throughout training, keeping perceptual representations fixed while the grounding dynamics were compared.
Evaluation covered image retrieval using R@1, R@5, R@10, median rank and MRR, alongside revision and coherence dynamics. Random served as a chance-level lower bound, while CLIP-only provided a frozen reference without grounding updates or task-specific training.
PhotoChat contains 21 training shards, split deterministically into 19 training shards and 2 test shards using split seed 42. The same split was shared across training seeds and policy regimes, and the reported results are means plus or minus standard deviation over three training seeds.
Similar rankings, different grounding dynamics
On R@1, the measure of how often the target image ranked first, PURE-CORR scored 0.118 plus or minus 0.005, FIXED scored 0.117 plus or minus 0.006, and MULTI-C scored 0.112 plus or minus 0.004. Their median rank was 16.0 plus or minus 1.0, 16.0 plus or minus 1.0, and 16.0 plus or minus 0.0, respectively. All three had a displayed median rank of 16.0, so MULTI-C did not distinguish itself through a large retrieval gain.
DCP-B was the retrieval outlier. Its R@1 was 0.009 plus or minus 0.006, R@5 was 0.052 plus or minus 0.032, R@10 was 0.086 plus or minus 0.048, MRR was 0.038 plus or minus 0.019, and median rank was 184.0 plus or minus 121.3. Its revision sensitivity was 0.900 plus or minus 0.015, while Switch KL, a measure of shifts between consecutive grounding states, was 12.31 plus or minus 0.88 during high-revision turns. The figures describe very weak retrieval alongside very large grounding shifts.
MULTI-C’s own grounding measures showed a different pattern. Its preservation score was 0.083 plus or minus 0.001, revision sensitivity was 0.598 plus or minus 0.097, and mean rho was 0.148. Alongside its near-peer retrieval, those values fit the study’s description of bounded, selective revision dynamics.
The revision signal pointed away from simple mismatch
Within MULTI-C, local divergence had a scaled contribution of 0.399 and a correlation of r = -0.935 with revision. Incoming uncertainty had a contribution of 0.103 and a correlation of r = +0.562. These figures describe associations, not evidence that either signal caused the observed revisions.
In practical terms, stronger local divergence was associated with less revision, while greater incoming uncertainty was associated with more. The authors interpret this as conservative, selective belief revision, with accumulated uncertainty and context carrying weight even when a single local mismatch was not followed by more revision.
The behavior appeared turn by turn
Turn-level traces from two randomly sampled pre-disclosure dialogues gave a qualitative view of the policies’ behavior. PURE-CORR and FIXED stayed stable. DCP-B showed a spike-then-freeze pattern, with coherence failing to recover, while MULTI-C showed temporary coherence decreases followed by recovery. The trajectory evidence is therefore based on two qualitative examples.
An inference-time ablation used zero-vector perturbations on MULTI-C. With the full model, R@1 was 0.112 plus or minus 0.004. With the image vector zeroed, R@1 was 0.000 plus or minus 0.000 and MRR was 0.002 plus or minus 0.000; zeroing the text vector produced the same values. Both conditions were associated with near-chance retrieval in this test.
What the comparison can say
The evaluation reports retrieval alongside revision and coherence dynamics, so retrieval numbers are only one part of the assessment. The fixed architecture and frozen perceptual representations make the reported differences a comparison of policy behavior under the same setup.
Taken together, MULTI-C remained near the PURE-CORR and FIXED retrieval range while showing bounded revision dynamics, whereas DCP-B was substantially worse on retrieval and showed very large high-revision shifts. That is a descriptive separation between retrieval and revision behavior, not evidence that one revision rule caused a better result. The open design question is how a grounded dialogue model should balance local mismatch with accumulated uncertainty and context when deciding whether to preserve or revise an interpretation.
Paper data and sources
Original title: Beyond Local Surprise: Grounded Dialogue as Selective Belief Revision under Referential Uncertainty
Authors: Ziming Liu, Bhanu Chaitanya Jasti, Ziyang Xu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text