A video-generation system that controls object paths, camera motion and depth through 3D point tracks reached 20.6 frames per second in a reported comparison, while keeping its motion error within 4% of a bidirectional teacher model. The figure was higher than the 16.7 and 10.4 FPS reported for two MotionStream causal baselines, and its end-point error, or EPE, was 5.48, below every causal baseline. The paper's stated target is precise, real-time control over object motion, camera behavior and depth through a unified interface.
The design pairs a 50-step bidirectional teacher with a four-step causal student, a model that generates the stream forward as new frames are produced. The teacher-to-student distillation is described as giving the student constant memory and arbitrarily long generation. That is the core engineering idea: use the teacher's control behavior as a reference while reducing the denoising steps needed for streaming.
The machinery behind the controls
To supply the geometric signals, the authors built OpenVidHD-Motion3D from OpenVid-1M with 3D tracks produced through SpatialTrackerV2. The pipeline queried a 32-by-32 grid of points in the first frame and retained roughly 0.4 million high-definition clips after filtering. Those tracks give the model a structured way to condition object motion, camera behavior and depth.
Teacher training used Wan2.2-TI2V-5B in two stages, moving from 49 frames at 256-by-256 pixels to 81 frames at 480-by-832 pixels. The setup used 256 fixed tracks per sample. The teacher was then distilled into the four-step streaming design.
What the benchmark measured
The main joint-control comparison used 30 in-the-wild DAVIS validation videos selected for challenging coupled object and camera motion. The methods shared the same first frame and text where applicable, but used their native conditioning interfaces. That means the test put the systems on a common starting point without making their control inputs identical.
On that set, the teacher had the best reported combination of motion and visual-quality scores. EPE was 5.29 and LPIPS was 0.404, both the lowest values, while SSIM was 0.479, the highest. PSNR was the only reported metric where it did not lead: the teacher scored 16.04 against a comparison value of 16.61. The authors say pixel-wise scoring can penalize plausible variation when the content is underspecified.
The speed comes with boundaries
The causal student retained near-teacher motion accuracy at higher throughput. It reached 20.6 FPS, versus 16.7 and 10.4 FPS for the two listed MotionStream causal baselines, while its EPE of 5.48 was within 4% of the teacher and lower than every causal baseline. The comparison was nonrandomized and included no confidence intervals.
The streaming result sustained more than 20 FPS at 480p on a single high-end GPU, with memory use independent of the length of the generated video. Extended demonstrations reached 350 frames. The authors present those measurements as evidence of engineering feasibility for long streaming rollouts on the stated hardware, based on benchmark and demonstration results.
Testing what the 3D signal adds
An ablation on 20 clean validation videos using 81-frame clips compared the full model with a version that used 2D tracks only. The full model reported LPIPS 0.200 and EPE 0.985, compared with 0.225 and 1.523 for the 2D-only version. In that test, the complete geometric conditioning design had better reported appearance and motion scores.
Other ablations examined track identity and the training schedule. The variant without track-ID embeddings reported EPE 2.455, compared with 0.985 for the full model, and LPIPS 0.283, compared with 0.200. Single-stage training reported EPE 3.217, compared with 0.985, and SSIM 0.613, compared with 0.661. These results associate the tested components with better benchmark values, but do not establish generalization beyond the settings examined.
Motion transfer is still a visual demonstration
In qualitative motion-transfer examples, the system used 3D tracks and camera parameters extracted from a source video, then paired them with a restyled first frame. The generated video reproduced the shown human motion, camera movement and background parallax while changing style and scene content. No quantitative motion-transfer evaluation is reported, so these examples illustrate rather than measure the behavior.
The paper's failure cases show the limits of that control. Small subjects or objects can become progressively blurry, and the model can break causality or object permanence. The examples described by the authors include a cake developing a missing chunk and a piping bag appearing from nowhere.
An early engineering result
The results read as an engineering demonstration of fast 3D control and long streaming generation, with scene consistency still unresolved by the reported failures. The document is a preprint identified as arXiv:2608.25479v1 and dated 26 August 2026. Its acknowledgment reports partial support for S. Li and Y. Zhu from the National Natural Science Foundation of China, grant 62376009, and the Beijing Nova program.
Paper data and sources
Original title: 4DStreamCtrl: Interactive Video Generation with Online 4D Control
Authors: Shiqian Li, Chenguo Lin, Zhiguang Liu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text