A new arXiv preprint presents a six-million-pair collection for instruction- and reference-based image and video editing. In the paper's instruction-based comparison, the reported Overall Score reached 4.22 with GPT-5.5 and 4.56 with Gemini-3-Pro. The reference-conditioned model, which uses a visual example to guide an edit, was also reported to require 50% less training computation than the standard editing model, with comparable inference cost. These are benchmark point estimates, not a guarantee that the approach will lead every test.
A collection built from edited and original footage
The dataset contains 5 million video editing pairs and 1 million image editing pairs. The video pairs are 720p and run from 81 to 129 frames, and the video subset was constructed by 12 editing experts. The two counts describe separate video and image subsets.
For most editing tasks, the pipeline uses an edited video as the source and the original video as the target. The visual reference is meant to specify the location of an edit, its direction, or its appearance. The authors describe this reversed pairing and the reference information as the basis for more reliable supervision, although that interpretation is not an independently tested causal finding.
The source pool began with approximately 1 million permissioned Pexels videos. After duplicate and short-video filtering, around 700,000 videos were split into 2.5 million segments of 161 frames, producing approximately 40 million masks.
The dataset is reported to contain 6 million references spanning 10 types. Those references cover location, direction and appearance guidance, giving the reference-based part of the system several ways to describe what an edit should change.
Filtering the generated examples
Quality control combined machine checks with human-refined prompts. The prompts were refined by 3 human annotators and asked GPT-5.2 to look for visual artifacts, consistency between instructions and results, motion problems, hallucinated or unintended changes, and mismatches with the reference. DINO-V3 was also used to flag unchanged edits and motion inconsistency.
That process matters because the pipeline generates masks and checks for hallucinated or unintended changes, but it remains a screening system rather than a guarantee that every pair is artifact-free. The supplied analysis does not treat LLM filtering as equivalent to expert review.
Promising scores, with human evidence still limited
For its main comparison, the benchmark used recent Pexels videos excluded from RefVideo-6M. It assessed 100 instruction-based videos and 100 reference-based videos with 2 mainstream large language model evaluators. The paper reports average scores from those evaluations alongside CLIPScore, temporal consistency and forced-choice user preferences.
In instruction-based editing, the proposed system had the highest reported Overall Scores in both judge settings: 4.22 under GPT-5.5 and 4.56 under Gemini-3-Pro. It also had the best reported CLIPScore and temporal consistency, and drew 54.7% of the user-study preference in that setting.
Reference-based editing produced a similar pattern, though not a clean sweep. RefMoT's reported Overall Score was 4.03 under GPT-5.5 and 4.16 under Gemini-3-Pro, compared with 3.73 and 3.98 for UniVideo. UniVideo was slightly better on Attribute Alignment, showing that the proposed method did not lead every category.
In the web-based forced-choice study, 33 valid users answered 12 questions per setting. The proposed method received 54.7% preference for instruction-based editing and 52.5% for reference-based editing; the second-best values were 7.9% for instruction-based editing and 17.9% for reference-based editing. The study population and recruitment details were limited, and no confidence intervals were reported.
The paper also tested whether its automated judges agreed with human labels. In a 90-video alignment study, GPT-5.5 agreed on 85 videos, or 94.44%, while Gemini agreed on 83, or 92.22%. The check used a strict binary decision rule, so the result says more about agreement on those labels than about the full range of editing quality.
Efficiency claims and the next test
RefMoT is the paper's reference-conditioned model. The authors report that it cuts training computation by 50% while keeping inference cost comparable to the standard editing model. The comparison is a reported efficiency result from the paper; measurement procedures and uncertainty estimates were not supplied in the reviewed material.
The ablations point in the same direction within the paper's comparisons. In dataset ablations using up to 200,000 videos per dataset, reported Overall Scores rose from 3.32 to 3.88 under GPT-5.5 and from 3.21 to 3.90 under Gemini-3-Pro. In full RefMoT architecture ablations, the scores rose from 3.97 to 4.22 and from 4.08 to 4.56, respectively. These were descriptive comparisons without inferential uncertainty.
That leaves a central question about transfer. The corpus is primarily derived from Pexels, and the evaluation leans heavily on automated metrics and LLM-as-judge scores, while human validation is limited. The benchmark uses 100 videos in each setting and the user study has 33 valid users. The reported score differences come without confidence intervals or formal hypothesis tests, so they do not establish that the dataset or architecture will outperform alternatives across independently curated tasks and reference types.
RefVideo-6M is available for further checking. The paper states that the open-source dataset is available through Hugging Face and that source code with additional implementation details was uploaded to OpenReview. The document is an arXiv preprint, version 1, dated 26 August 2026.
RefVideo-6M therefore offers a large resource and an encouraging within-benchmark report, but the evidence remains tied to a Pexels-derived distribution, selected test cases and the paper's scoring setup. Human-rated comparisons on broader sources would be needed to see whether the reported ranking holds.
Paper data and sources
Original title: RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing
Authors: Bojia Zi, Xiaoyan Yang, Yu Zhou et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text