Video-editing systems can make a target object disappear while leaving parts of its physical footprint behind, according to a new preprint. In qualitative examples, many methods removed the visible object but retained shadows, reflections, illumination changes or other traces.
The study focuses on whether the whole scene has been restored after the object and its associated environmental changes are removed—not only whether the masked region looks filled in.
A paired test for what disappears
The researchers built BeyondMasks, a benchmark of 180 paired video sequences: 90 synthetic scenes and 90 real-world captures. Each pair includes an object-present video, a temporally aligned clean background reference, per-frame masks and an instruction prompt.
The synthetic pairs use Veo3-generated scenes in which an object was inserted into a clean background video while temporal alignment was preserved. The real-world set consists of 90 tripod-stabilized, back-to-back captures with a fixed camera position or settings; the masks cover only the target object, leaving after-effects unmasked.
The after-effect categories overlap rather than forming separate, balanced groups. The reported counts include 127 shadow cases, 61 reflection cases, 46 translucent-effect cases, 37 causal-physical-effect cases, 38 light-source-effect cases, 20 steam or scattering cases and six with no explicit after-effect.
Scoring the scene, not just the pixels
The proposed CORE evaluation compares three aligned videos: the original with the object, the clean background reference and the edited output. It separates how completely the object disappears and the background is restored from how well object-induced after-effects are removed.
Those dimensions are summarized as ObjectScore, or CORE-OS, and AfterEffectScore, or CORE-AES, on a 1-to-5 scale, with higher scores indicating better performance. The setup also compares the results with PSNR, SSIM, LPIPS and FVD, alongside nine mask-based, text-guided and hybrid methods stated in the experimental setup.
Human ratings favored the broader measure
In a user study, eight annotators rated five methods across 20 videos. CORE's ObjectScore had a Pearson correlation of 0.615 with the mean human rating, while AfterEffectScore reached 0.733. The corresponding correlations among raters were 0.693 and 0.785.
A supplementary comparison reported much weaker alignment for the conventional image measures. PSNR correlated at 0.15 for ObjectScore and 0.28 for AfterEffectScore; SSIM at -0.14 and -0.10; and LPIPS at -0.02 and 0.10. Two CORE variants produced correlations of 0.62/0.73 and 0.60/0.76 for the two scores, close to inter-rater correlations of 0.69/0.79.
The table also lists different performance profiles for reconstruction metrics and CORE scores. DiffuEraser is reported with PSNR 25.1880, SSIM 0.8769, LPIPS 0.1124 and FVD 164.26, while EffectErase is reported with CORE-OS 4.015 and CORE-AES 3.162.
Steam and causal effects were hardest to clear
Across the six top-performing models, average AfterEffectScore was lowest for steam, at 2.032, and causal interactions, at 2.141. The corresponding averages were higher for shadows, at 2.520, and reflections, at 2.446.
The split results were descriptive rather than randomized. DiffuEraser posted similar reconstruction figures in the synthetic and real subsets—PSNR 25.3617 and 25.1622, SSIM 0.8932 and 0.8621, and LPIPS 0.0942 and 0.1290—while ROSE's CORE-OS fell from 4.068 on synthetic scenes to 3.505 on real captures.
What the preprint leaves unresolved
The user check was small, and the reported correlations do not directly validate physical correctness. The study also does not establish that any evaluated method understands physical laws or that CORE is a ground-truth measure of correctness.
The benchmark is compact, with overlapping and imbalanced after-effect categories. Its synthetic scenes depend on Veo3, while the real pairs may retain uncontrolled differences in timing or lighting. The comparison does not establish performance beyond the benchmark scenes and evaluated methods.
The work is an arXiv preprint, version 2608.20107v1 dated 20 Aug 2026, and no journal publication is reported. Independent datasets, larger annotator pools and other vision-language-model judges would be needed to test whether the scoring approach generalizes.
Paper data and sources
Original title: BeyondMasks: Evaluating Causal and Physical Consistency in Video Object Removal
Authors: Yigit Ekin, Enes Sanli, Aykut Erdem et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text