A picture helps narrow the search
An arXiv preprint describes a video-search task in which a user supplies both a reference image and a text description. The aim is to locate the stretch of video where that particular instance carries out the described action, even when similar instances appear.
The paper constructs two benchmarks, IDVTG-Gym and IDVTG-InternVid. The comparison table lists 506.8 hours of video and 76.8K queries in total: 204.1 hours and 14.7K queries for Gym, and 302.7 hours and 62.1K queries for InternVid.
Higher reported scores across the tests
The proposed VGD-Agg system recorded 61.83, 56.53 and 54.17 on the overall IDVTG-Gym benchmark. In the table's reported order, those are Recall@1 at overlap thresholds of 0.5 and 0.7, followed by mean Intersection over Union, a measure of overlap between the predicted and labelled time spans. SnAG, the named comparison baseline, recorded 53.64, 48.82 and 46.73.
The Gym results varied by split. VGD-Agg reported 55.53, 44.91 and 43.29 on the Sub-act. split, and 66.58, 65.36 and 62.53 on the Holistic. split.
On IDVTG-InternVid, VGD-Agg reported 51.21, 41.24 and 48.30, compared with SnAG's 44.26, 35.46 and 41.70. On the Web Dataset, the figures were 21.99, 13.25 and 22.34 for VGD-Agg, against 16.48, 9.06 and 16.53 for SnAG. The Web test used the model trained on InternVid directly, without fine-tuning.
How the model separates similar instances
VGD-Agg uses two processing paths. A fast branch makes preliminary temporal proposals, while a slower branch compares the video with the reference image at frame level. The design also includes a Compare Token and a Depress Value, video-specific representations for disambiguation.
The benchmark data were assembled in two stages: first, the pipeline identifies ambiguous text events and their time boundaries; then it links the target instance to an image query that can help disambiguate the event.
The evidence has clear limits
For a manual check, the authors randomly sampled 10% of the data. The automatically labelled data reached a reported 76.8% on R1@0.5 against manual annotations. The supplied analysis does not report a confidence interval, an inter-annotator measure or sampling uncertainty for that check.
Removing parts of the architecture generally lowered or mixed the reported scores. Without the Slow Branch, the three figures were 53.64, 48.82 and 46.73; without the Fast Branch, 59.41, 54.40 and 51.32; and without Aggregation, 61.12, 56.76 and 53.50, compared with 61.83, 56.53 and 54.17 for the full model. The full configuration also had the highest scores among the listed Compare Token and Depress Value variants, and among the listed text-only and alternative image-text fusion settings.
Performance also fell when the reference image was degraded. On InternVid, the clean-image scores were 51.21, 41.24 and 48.30; they fell to 49.99, 39.79 and 47.20 with altered brightness, and to 49.10, 39.33 and 46.52 at low resolution.
The comparison is limited to the tested benchmark settings. The supplied analysis reports no confidence intervals, significance tests or run-to-run variability, so the tables cannot show whether the reported gaps would persist under those checks.
Paper data and sources
Original title: ID-VTG: Image-Disambiguated Video Temporal Grounding
Authors: Minghang Zheng, Jingli Wei, Hongyi Yang, Yang Liu
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text