An arXiv preprint reports higher robot-task success for TrAct, a system that uses visual tracks to connect control with predictions of future video. Against the listed π0.5 baseline, reported success rose from 27% to 55% in simulation and from 49% to 76% on real-world tasks.
A shared interface for control and prediction
TrAct's three-part design uses visual tracks as the common interface between control and prediction. VLAT proposes action-track pairs; TWM, the track-conditioned world model, predicts future videos; and VLAC scores possible rollouts to choose an action. The system therefore links policy learning with world-model-based decision-making through the same visual representation.
The clearest gains appeared in prediction scores
That design was reflected in the video tests. TWM outperformed AWM on all five reported video metrics across both camera views. In the simulation agent view, TWM's PSNR score was 24.51 versus 15.12 for AWM, while LPIPS was 0.106 versus 0.438. The paper treats these as measures of prediction quality, not direct measures of task completion.
Shifted tasks separated the systems
On LIBERO-INTEGRAL, average success was 0.27 for π0.5, 0.44 for VLAT, 0.49 for VLAT+AWM and 0.55 for TrAct. The benchmark's 20 tasks are divided into 10 robustness tasks and 10 cross-embodiment tasks. TrAct also scored higher than VLAT+AWM in the four highlighted comparisons: 0.60 versus 0.50 for Object, 0.60 versus 0.45 for Camera, 0.65 versus 0.55 for RobotInit, and 0.50 versus 0.44 for Cross-Embodiment.
On standard LIBERO, the reported averages were 96.8% for π0.5, 98.0% for VLAT, 98.0% for VLAT+AWM and 98.3% for TrAct. That made TrAct the highest of the listed methods, although the differences there were small.
The pattern continued on a real robot
For the physical tests, the setup used a Franka Emika Panda and two RGB cameras. It included 400 demonstrations collected at 15 Hz for four training tasks, followed by five unseen evaluation tasks. Each physical task was run for 10 episodes under both background conditions. TrAct averaged 0.76 success, compared with 0.66 for VLAT+AWM and 0.55 for VLAT.
The background test exposed a sharper split. Under the unseen background condition, VLAT+AWM fell to 0.40, matching VLAT without reranking, while TrAct remained at 0.70. The result is specific to that visual shift and does not show how the system would perform across other environments, cameras or robot embodiments.
The comparison used shared pretraining
TWM and AWM were both pretrained on 76,000 DROID trajectories for 30,000 steps with a batch size of 64, then fine-tuned on the same LIBERO and real-world datasets. VLAT pretraining used 76,000 DROID trajectories and 150,000 EgoDex trajectories, sampled from about half of the full EgoDex dataset, with a 1:2 mixing ratio.
A three-seed check narrowed the uncertainty
A separate check tested variation across three independent evaluation seeds. TrAct's mean success was 0.547, with a standard error of 0.003 and a 95% confidence interval from 0.53 to 0.56. VLAT+AWM had a mean of 0.490, a standard error of 0.006 and a 95% confidence interval from 0.47 to 0.51. The reported intervals did not overlap. This check covered those two systems, so it strengthens that comparison without settling the reliability of every other reported contrast.
A benchmark result, not a deployment guarantee
The study's test base remains limited in size and variety. LIBERO-INTEGRAL contributes 20 simulation tasks, while the physical evaluation covers five unseen tasks on the Panda setup, with 400 demonstrations from four training tasks. Those experiments provide a simulation and physical proof of concept, but they do not establish that the reported gains will persist across additional robot embodiments, camera configurations, environments or longer-horizon tasks.
Taken together, the preprint reports a consistent pattern within its chosen tests: the visual-track system had higher reported task success than the listed baselines, while TWM had stronger reported video metrics than AWM. Whether that pattern survives broader testing remains open. The document is identified as arXiv version 3, dated 1 Sep 2026.
Paper data and sources
Original title: TrAct: Bridging Robot Control and Visual Prediction with Visual Tracks
Authors: Zhi Cao, Howard Ji, Kevin Zhang et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: Not available
Original paper · Full text