Across five benchmark systems, the paper’s Orthogonal JEPA recorded higher reported task scores than Standard JEPA in visual binding, single-cell modeling, health forecasting and continuous control. In molecular-dynamics comparisons, it recorded lower reported error measures than TrajCast-JEPA for water, quartz, paracetamol and benzene.
The study asks whether a JEPA can organize a predictable world state through several structured prediction factors instead of one monolithic prediction path. In plain language, it tests whether a model can make separate predictions about parts of an internal state and then combine them into a complete state.
A different way to split prediction
Orthogonal JEPA uses learned basis matrices and dedicated factor-specific prediction branches. It also regularizes orthogonality and factor activity, discourages coordinate collapse and synthesizes a complete latent state.
The evaluation spans five systems covering hidden-state completion, temporal forecasting, planning and autoregressive rollout. The comparisons held the adapter, encoder family, context–target sampler, data split, optimization budget and downstream readout fixed, with auxiliary terms fixed when used.
Results across several domains
On controlled visual-binding scenes, Orthogonal JEPA had higher INJ, the held-out-cell matching measure, and Rec., grid recovery, plus a lower Coll., collapse rate, than Standard JEPA with learned-grid supervision. With DINOv3, INJ was .581 versus .572, Coll. .417 versus .426 and Rec. .659 versus .645; with SigLIP2, the corresponding figures were .490 versus .483, .503 versus .514 and .688 versus .679.
The visual setup used image-disjoint splits, a leave-one-cell-out readout and injective Hungarian alignment. The three metrics were averaged over ten seeds, and collapse was measured at a threshold of 0.5.
Single-cell models were pretrained on approximately 800,000 human kidney cells and evaluated on PBMC-10K clustering plus Adamson and Norman perturbation-response data. Orthogonal JEPA reported AvgBIO scores of 0.8001 after PBMC fine-tuning and 0.7452 in the zero-shot setting, versus 0.7830 and 0.7194 for Cell-JEPA; its Pearson correlations were 0.798 on Norman and 0.942 on Adamson, versus 0.787 and 0.937.
On the health-forecasting benchmark, mean PRAUC across more than 1,000 future clinical events was 0.718 for Orthogonal JEPA and 0.711 for Standard JEPA.
Both JEPA conditions used 500 random-action trajectories. In planning tests, Orthogonal JEPA’s mean return was 45.1 ± 11.2 on Walker2d, −8.5 ± 0.6 on HalfCheetah and 30.6 ± 3.8 on InvertedPendulum, compared with 4.9 ± 12.6, −11.2 ± 0.8 and 18.1 ± 2.3 for Standard JEPA. Results were reported as mean ± standard deviation over three seeds; planning used 200 candidates, 20 elites, five iterations and an eight-step horizon.
For the two molecular error measures, MAE and RMSD, Orthogonal JEPA reported 0.00376 and 2.459 for water, 0.01011 and 1.877 for quartz, 0.00765 and 1.846 for paracetamol, and 2.05×10−5 and 0.0699 for benzene. The paper reports these values as lower than the corresponding TrajCast-JEPA values in all four systems.
What the comparisons leave open
The authors interpret the cross-domain comparisons as favoring orthogonal predictive factorization as a reusable design for latent world models.
The paper cautions that geometric orthogonality alone does not establish statistical independence, causal modularity or semantic disentanglement. Additional probes or ground-truth factor benchmarks are needed to interpret the learned components.
The experiments do not cover pixel-based closed-loop control, stochastic futures, continuous physical fields or tasks with known causal factors. The work is an arXiv preprint, arXiv:2608.20065v1, dated 20 Aug 2026.
Paper data and sources
Original title: Orthogonal JEPA: Factorized Predictive States for Latent World Models
Authors: Taoyong Cui, Pheng Ann Heng, Wanli Ouyang
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text