An arXiv version 1 preprint dated 24 Aug 2026 reports that CAT-GS was strongest in the reported comparisons on CREMA-D and AV-MNIST, but did not surpass the strongest baselines on VGGSound. Its reported fused test accuracy was 86.29% plus or minus 0.15 on CREMA-D, 73.21% plus or minus 0.08 on AV-MNIST and 53.24% plus or minus 0.24 on VGGSound.
The results split across benchmarks
The evaluation used seven benchmarks: CREMA-D, AV-MNIST, VGGSound, UR-FUNNY, CG-MNIST, AVE and CMU-MOSI. Reported sizes included 7,442 CREMA-D clips, 16k UR-FUNNY labeled segments, 4,143 AVE videos and 2,199 CMU-MOSI opinion video clips.
Models were evaluated using held-out test accuracy, with checkpoints selected from validation data. Comparisons generally used three random seeds—42, 123 and 999—and CAT-GS run-to-run variability was reported as a mean plus or minus standard deviation where stated. The default protocol trained for 400 epochs with batch size 16, used stochastic gradient descent and reduced the learning rate at epoch 200.
On UR-FUNNY, CAT-GS scores varied by input combination. It reported 62.12% plus or minus 0.18 with audio and video, 64.49% plus or minus 1.05 with audio and text, 65.25% plus or minus 0.97 with video and text, and 67.55% plus or minus 0.84 with all three. The all-three result was the highest of the four, and the method was strongest in the text-inclusive configurations, but not in audio-video.
On AVE, CAT-GS reported 74.2% plus or minus 0.2. On CMU-MOSI, it reported 76.0% plus or minus 1.2 with video and text, and 78.3% plus or minus 1.0 with video, audio and text. The analysis described the AVE and two-modality MOSI results as the strongest reported results in those comparisons, while the three-modality MOSI score was slightly above DRL.
A CG-MNIST test produced 99.32% for the monochromatic modality and 67.85% for the grayscale modality, while fused accuracy was 97.42% plus or minus 0.23. This combined a high fused score with uneven single-modality scores.
What the controller changes
CAT-GS works at the optimization stage, in the multimodal backward pass—the part of training that sends error signals through the model. It targets gate instability, modality starvation, in which one input contributes too little, and interference when information from different modalities is fused, all within one optimization step.
On CREMA-D, the paper also examined gradients—the model’s training signals—at the fusion layer. Its diagnostics described smoother gating, positive average fusion-gradient cosine similarity and fewer batches with negative cosine similarity. The cumulative visual-to-audio gradient-budget ratio was 1.7 times for CAT-GS, compared with 3.2 times for Joint-Train and approximately 2.0 times for G2 D. The reported pattern was a lower cumulative gradient-budget imbalance under CAT-GS.
The component analysis identified gradient-budget reallocation as the largest removal-block contribution. EMA smoothing was also important, while PCGrad had a smaller effect.
Stress tests mark the boundaries
In a clean-versus-degraded CREMA-D protocol, CAT-GS had smaller fused-accuracy drops than Joint-Train: 14.50 versus 21.24 percentage points under audio degradation, and 8.00 versus 12.07 percentage points under visual degradation.
In teacher-miscalibration stress testing, reported accuracy was 86.3% at R0, 85.2% at R1, 85.1% at R2, 84.4% at R3, 83.6% at R4 and 84.0% at R5. Threshold testing found a maximum accuracy change below 0.5 percentage points, with variance below 0.5%.
The authors state that genuinely weak or domain-shifted teachers remain untested, and that teacher-free CAT-GS control remains future work. The theoretical analysis offers a local one-step descent guarantee, not a global convergence result. The authors also observed an approximately 2% to 5% increase in per-epoch training time.
Paper data and sources
Original title: CAT-GS: Balanced Multimodal Learning via Calibrated Gating and Fusion Surgery
Authors: Mahir Shahriar Tamim, Sharjil Khan, Md. Samiul Alim et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-24
DOI: Not available
Original paper · Full text