An arXiv preprint reports that a unified system for learning from audio and video has, in the authors’ words, consistently surpassed a comparator called Crab across 11 evaluation metrics. But the result is not a clean sweep: in the paper’s displayed audio-visual question-answering comparison, Crab has an overall average of 76.4%, while the proposed method is listed at 76.1%.
A single framework, split by task
The study asks whether one unified framework can handle versatile audio-visual multi-modal learning tasks while limiting the mutual interference the paper associates with naive joint training. Its task-disentangled LoRA combines a task-general low-rank matrix, task-specific modulation matrices and cross-task collaboration experts.
The design also extends to pixel-level tasks. It turns embeddings from generated mask tokens into prompts for a mask decoder, then combines the model’s autoregressive text loss with binary cross-entropy and Dice segmentation losses.
The benchmark table describes dataset instances rather than a participant sample: it reports 66,802 training instances and 17,003 test instances across the listed audio-visual multi-modal datasets.
For the reported implementation, the team used LLaMA-2-7B-Chat, sampled 10 frames per video at 224 by 224 resolution and applied rank-8 LoRA in all linear layers. Training used AdamW with a learning rate of 1 × 10−4; the first fine-tuning stage ran for five epochs, while the segmentation stage ran for 30.
Different tasks, different results
On AVEL, the reported accuracy was 77.8%. On AVVP, the method was reported to have the highest overall Type@AV and Event@AV scores at both segment and event levels. No uncertainty estimates or significance tests were reported for these comparisons.
On ARIG, the method reached 41.1 cIoU and 41.7 AUC, with reported margins of 1.7 and 1.6 points over Crab, respectively. Those figures are benchmark comparisons without reported uncertainty estimates.
On an unseen RAVS subset, mIoU rose from 45.6% to 52.1%, while F1 increased from 63.0% to 69.5%. The authors interpret this as generalization to novel concepts, but it remains a comparison on a benchmark subset without an uncertainty estimate.
The AVQA results need closer reading. In the surrounding narrative, the authors report 76.11% overall accuracy, the best visual-counting performance and modest gains over Crab on comparative questions. But the displayed table lists the overall average as 76.4% for Crab and 76.1% for the proposed method. The mismatch in precision and direction means the paper’s broad claim of consistent superiority should not be read as applying uniformly to this comparison.
A qualified benchmark result
Visual analysis of the learned features points in the same direction as the paper’s main argument: compared with HydraLoRA, the proposed method is reported to produce tighter feature distributions within each task and greater separation between tasks, in both low-rank and full-rank feature spaces. Because this finding comes from visualizations, it does not quantify uncertainty.
Ablation tests generally favored keeping the complete three-expert design. Removing the task-specific modulation matrix Λ lowered reported performance, and removing experts also produced lower performance, particularly when B1 was removed.
Taken together, the paper offers a strong but bounded benchmark record. Its broad superiority claim is an author-reported comparison, while the displayed AVQA figures show that the proposed method does not lead every reported comparison. The document is an arXiv preprint, and its results are best read as technical benchmark findings rather than a universal performance claim.
Paper data and sources
Original title: Task-disentangled Low-Rank Adaptation for Versatile Audio-visual Multi-modal Learning Tasks within a Unified Framework
Authors: Hanyu Xuan, Mengqi Zhang, Junjun Mao et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: Not available
Original paper · Full text