Preprint

New AI prompting method reports gains in multimodal humor tests

Preprint: A code-based reasoning graph was associated with higher benchmark scores, but the study did not establish a causal advantage or human-level humor understanding.

A graph before the answer

A preprint under review reports that a prompting method called CaRGo-T produced higher scores on benchmark tests of multimodal humor. Across the reported experiments, the authors state gains of about 1% to 20% for humor understanding and 1% to 3% for humor detection compared with reasoning-based baselines. CaRGo-T has a vision-language model generate a lightweight causal reasoning graph, serialize that graph as code, and then use the graph to condition the final answer.

The graph changes the prompt by placing an intermediate reasoning component before the final answer. The evaluations compare model outputs on benchmark tasks, not human participants. Because the prompt conditions were not randomized, the comparisons do not establish that CaRGo-T itself caused the reported gains. They should be read as results observed in the tested models, datasets and settings, not as a causal effect estimate.

Experiments used GPT-4o, GPT-4o-mini and MiniCPM2; open-source runs used two NVIDIA L40 GPUs with 48GB of VRAM each. In in-context tests, examples contained manually rectified causal graphs and ground-truth answers, with the VLM used only for inference rather than training. The method was tested in zero-shot and in-context forms, meaning with or without supplied examples.

Where the scores moved

For humor understanding, the evaluation used a holdout of five examples, 1,079 YesBut satirical images and 559 MemeCap test samples. Outputs were scored with ROUGE-L, BLEU, BERTScore and Avg. Score. These are the study's reported measures for the understanding task.

In one zero-shot understanding test with MiniCPM, CaRGo-T had the best reported Avg. Score, with a 0.72% gain on satire and a 5.81% gain on meme understanding versus the best baseline. In few-shot understanding, CaRGo-T was reported to outperform baselines across all metrics in the 2-shot setting. With GPT-4o, its improvement over CoT declined from 11.66% at 0-shot to 10.14% at 2-shot and 5.86% at 5-shot.

Detection produced smaller reported gains, but the method was positive against the best baseline in every GPT-4o setting listed. The detection evaluation used six holdout samples, 2,541 YesBut images, including 1,081 satirical and 1,460 non-satirical images, and 2,409 MMSD 2.0 samples, including 1,037 sarcastic and 1,372 non-sarcastic samples. Across 0-shot, 2-shot and 6-shot GPT-4o tests, sarcasm accuracy gains were 2.93%, 3.23% and 2.67%, while sarcasm F1 gains were 0.83%, 1.05% and 0.9%. For satire, accuracy gains were 1.12%, 0.54% and 1.05%, and F1 gains were 0.37%, 0.54% and 0.39%.

Measures of the reasoning

The researchers looked beyond final scores at the reasoning text. In a MiniCPM zero-shot YesBut analysis, KL divergence, a measure of how token distributions differ, was 0.21 with CaRGo-T listed first versus 0.19 with CoT first, 0.21 versus 0.20 against CoD and 0.25 versus 0.25 against CCoT. Low Similarity Fraction, a sentence-similarity measure, was 1 versus 0.85 against CoT, 1 versus 0.84 against CoD and 0.98 versus 0.94 against CCoT. The authors interpret those higher or tied comparisons as more novel lexical and semantic information, but the measures describe generated reasoning text rather than direct humor accuracy.

Another analysis used GPT-4 as a judge to assess whether the ground-truth answer could be inferred from the reasoning. CaRGo-T had the highest Infer Score, 45.11, compared with 40.78 for CoT, 40.68 for CoD and 37.64 for CCoT. The score therefore reflects a model-based judgment, not independent human validation.

A result that still needs checking

An ablation test, which removes or alters one part of a method, also favored the CaRGo-T configuration in GPT-4o. CaRGo-T outperformed unrectified in-context graphs across all reported metrics and beat a definition-augmented variant on three of four zero-shot metrics and four of four in-context metrics. The paper says all reported improvements were significant under an independent two-sample t-test, although the supplied analysis reports no confidence intervals, p-values, power calculation or multiple-comparison procedure.

That leaves a cautious reading. The evidence comes from named benchmark tasks, models and prompting conditions, so it does not establish human-level or human-preferred humor understanding, that the generated graphs are causally correct or faithful, or that the method will generalize beyond the tested models, datasets and shot settings. The manuscript is explicitly labeled a preprint under review, and the paper says its code is available on GitHub.

Paper data and sources

Original title: CaRGo-T: Causal Reasoning Graph-of-Thought improves Multimodal Humor Comprehension
Authors: Abhilash Nandy, Rahul Seetharaman, Aman Bansal et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-24
DOI: Not available
Original paper · Full text

Versions and corrections

  1. Published automatically after legal-source, freshness, evidence, and independent-verification gates passed.