An AI model designed to detect sarcasm in paired images and text reported 92.62% accuracy, 91.96% precision, 92.82% recall and 92.33% F1 on MMSD. F1 is the paper's combined measure of precision and recall. The authors say that was the best F1 among the listed methods on MMSD. On MMSD2.0, the reported figures were 89.66%, 89.36%, 89.74% and 89.51% for accuracy, precision, recall and F1, with similar gains over the listed baselines.
When the image and text disagree
The study targets cases where the useful sarcasm cue may sit in only one kind of input - words or images - or where the two look superficially aligned while carrying contradictory intent. The system starts with a pretrained CLIP vision-language encoder, fine-tunes both the text and vision encoders, and trains the framework end to end. Its fusion stage uses value-gated, bidirectional cross-modal attention, followed by a dynamic gate that balances text-aware and image-aware representations for each item.
SaCR adds a label-aware contrastive loss. For non-sarcastic samples, it encourages cross-modal similarity; for sarcastic samples, it penalizes excessive similarity. The overall objective combines multimodal classification, auxiliary single-modality supervision and the contrastive term.
Scores from two benchmark versions
The evaluation used the MMSD and MMSD2.0 benchmark datasets. Each version lists 19,816 training items, 2,410 validation items and 2,409 test items. MMSD's training split includes 8,642 sarcastic and 11,174 non-sarcastic items; MMSD2.0's includes 9,576 and 10,240.
The evaluation used the same data splits and metrics as previous work and compared the model with listed baselines, including GPT-5.4 under a zero-shot setting.
What the component tests showed
In Table III, the Full model's accuracy was 92.62% and its F1 was 92.33% on MMSD; the corresponding figures for the version without cross-modal interaction were 87.91% and 87.42%. On MMSD2.0, the Full model was at 89.66% accuracy and 89.36% F1, versus 85.10% and 84.97% without cross-modal interaction.
Every other listed component-removal variant also had a lower F1 than Full, including versions that kept only one direction of cross-modal attention, or lacked the value gate, dynamic fusion gate, SaCR or auxiliary unimodal supervision.
The paper's reporting is not fully consistent. Its main results give MMSD2.0 an F1 of 89.51%, while Table III lists 89.36% for the Full model. The ablation prose reports CMI-related F1 reductions of 5.40% on MMSD and 4.54% on MMSD2.0, but Table III lists Full-versus-without-CMI F1 values of 92.33% versus 87.42% and 89.36% versus 84.97%. That leaves the exact size of the component differences unclear.
A small window into failure
The paper uses Grad-CAM, a visualization of model focus, for a small case study. In two correctly classified sarcastic samples, the highlights fell on litter or meme text. In the failure case, the model focused on a face region and under-attended other ironic cues.
Those observations came from two correct examples and one failure, not an aggregate test of whether the highlighted regions faithfully explain the model's decisions. The benchmark results also came without confidence intervals, significance tests or other formal uncertainty estimates.
The evidence is confined to the two named benchmark versions and the reported qualitative cases. It does not establish how the model would perform on other datasets or real-world content.
Paper data and sources
Original title: Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization for Multimodal Sarcasm Detection
Authors: Hao Guo, Subin Huang, Junjie Chen et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text