A new arXiv preprint reports that an AI model for identifying bird species reached 87.23% accuracy when it combined audio and video, 2.97 percentage points ahead of the strongest comparison model in the authors' tests. The result comes from BirdPro, a benchmark built around fine-grained bird recognition, and it was higher than the scores reported for either audio or video used alone.
The study asks a specific question: can category-level audio and video information support species-level recognition when the two sources are not strictly synchronized or matched to the same real-world instance? In this asymmetric setting, the modalities carry related label information without necessarily describing the same moment or recording.
A benchmark built around birds
To test that question, the authors built BirdPro from CUB species-level annotations. They used scientific-name queries to retrieve videos and audio from publicly available online resources, then organized the material around 194 bird classes. The reported splits contain 10,054 training videos and 1,911 test videos, alongside 1,531 training audio samples and 388 test samples. There are also 6,022 paired training samples and 470 paired test samples.
In the reported setup, the visual branch processed six sampled frames at input resolution 448, while the audio input had a target length of 204.
Two different jobs for the two signals
ACF-Net, the proposed framework, has two key parts. Optical Flow-Guided Motion, or OFGM, uses optical flow as a spatial guide for fine-grained movement in the foreground. It does so without adding a separate motion encoder. The second part, Asymmetric Cross-Modal Adaptive Fusion, or ACAF, gives more sample-specific weight to whichever modality is more reliable for that example.
That division gives the system an asymmetric job: the visual side is guided toward movement and foreground detail, while the fusion stage decides how much to trust the audio or video signal on a sample-by-sample basis. The paper presents the design as a way to handle cases in which one modality carries a clearer cue than the other.
The combined score was highest
The single-modality comparisons were more modest. In the video-only comparison, the reported accuracy was 71.06%, versus 70.30% for FG-CLIP. In the audio-only comparison, it was 57.45%, versus 47.42% for ImageBind. In both cases, the comparison prose reports the proposed branch above the named comparison model.
The larger result came from fusion, meaning the two modalities were used together. That setting produced 87.23% accuracy, and the paper reports a 2.97 percentage-point advantage over its strongest comparison model. It was the best fusion result reported in the study's comparison.
The researchers also altered the training relationship between the modalities. Training audio from each class was reassigned to a different class, while the visual samples were left unchanged and validation remained clean. Under that mismatch experiment, the proposed model reached 74.26% accuracy, 1.92 percentage points above the strongest comparison model. Because validation stayed clean, this result describes altered training correspondence rather than performance on mismatched validation inputs.
The component results were less uniform
The component tests show that the headline score cannot be reduced to a single branch. In the OFGM condition, video accuracy was 71.92%, compared with 71.06% for the baseline, while fusion was slightly lower. In the ACAF condition, audio accuracy was 57.66% versus 45.96%, and fusion was 84.89% versus 82.98%. With both modules, the full model reported 58.30% audio accuracy, 72.77% video accuracy and 87.23% fusion accuracy.
Visual analysis offered a similar, but qualitative, picture. Feature visualizations showed more compact video representations and improved within-class compactness for audio. Attention was concentrated on the bird's head, neck and body, with the background suppressed.
The authors interpret these experiments as support for ACF-Net's effectiveness and robustness under asymmetric conditions. They conclude that the framework may be useful for fine-grained recognition in realistic, non-ideal cross-modal settings. The work is an arXiv preprint dated 26 Aug 2026.
Paper data and sources
Original title: Asymmetric Cross-Modal Fine-Grained Visual Categorization: ACF-Net and the BirdPro Benchmark
Authors: Bohan Deng, Shuo Ye, Zitong Yu
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text