A two-stage method for unlearning identity-related behavior from image-capable AI models reported that it could target answers about selected identities while leaving visual-perception responses largely coherent on the benchmarks tested. It is designed for deletion time, when retain data are unavailable, and uses forget images plus constructed probing questions instead of retain images or the original fine-tuning VQA pairs.
The question is narrower than whether a model can be weakened overall. The study asks whether identity-specific behavior can be reduced without wiping out the ability to answer what is visible in an image. It evaluates LLaVA-1.5-7B and Qwen3-VL-8B-Instruct on MLLMU-Bench and ReMem, using 5%, 10% and 15% forget ratios in the reported experiments.
How the method works
AIM divides the job into two stages. First it learns a visual prompt that defines the target behavior. Then it matches features in the vision encoder, the part of the model that processes images, to that target under a Fisher constraint intended to control the update. The language model remains fixed.
To motivate that split, the researchers inspected hidden states, the internal representations produced as the models process questions. Identity and visual-perception states occupied distinct regions. Identity questions clustered by image identity, while visual-perception questions clustered by question type.
In a pilot, IDK-Rate, the paper's measure of an identity refusal, was 100.0 for held-out identity questions and 10.0 for visual-perception questions on LLaVA. On Qwen, the corresponding figures were 85.0 and 0.0. The reported semantic-preservation scores were 63.3 and 85.2, respectively. No confidence intervals or significance tests were reported for this pilot.
What the benchmark scores show
Those percentages represent concrete benchmark subsets. MLLMU-Bench contains 500 fictitious identity profiles; its 5%, 10% and 15% forget settings target 25, 50 and 75 identities, with 475, 450 and 425 retained identities. A separate real-celebrity set contains 153 profiles. ReMem includes 2,000 fine-tuning VQA pairs, forget targets of 100, 200, 300 and 400 examples for 5%, 10%, 15% and 20%, plus 560 retain and 560 held-out test examples.
On ReMem, AIM's LLaVA results showed a mixed profile. At the 5% forget split, the reported scores were 30.0 for exact match on forget examples, 76.7 for exact match on retain examples, 88.1 for ROUGE, 54.4 for Exposure and 45.2 for the held-out test score. At 10%, the same sequence was 38.3, 78.8, 89.2, 63.9 and 35.1.
That pattern is a measured compromise, not an all-or-nothing deletion result: forget-side scores appeared alongside nonzero retain and test performance. The paper reports no uncertainty intervals or statistical significance estimates for these figures.
Qwen's MLLMU-Bench results followed the same broad pattern. At a 5% forget split, AIM reported forget classification and ROUGE scores of 53.6 and 59.8; retain scores of 66.4 and 65.1; and celebrity scores of 74.5 and 43.6. At 10%, the corresponding pairs were 62.4/60.7 for forget, 65.7/64.3 for retain and 76.4/42.8 for celebrity.
On visual-perception questions drawn from forget-identity images, the AIM row reported ROUGE/GPT score pairs of 57.8/68.9 against the pretrained comparison and 87.5/82.7 against the vanilla comparison. It also reported an appropriateness fraction of 97.3. The results suggest the evaluated answers stayed largely coherent and on topic, but they do not test whether identity information survives through text-only pathways or establish a human privacy outcome.
The tests beyond a single deletion
During the 10% to 15% continual ReMem transition, the AIM row reported 50.0 for forget exact match, 71.8 for retain exact match, 85.2 for ROUGE, 60.1 for Exposure and 56.3 for the held-out test. Under the image perturbations included in the evaluation, forget ROUGE-L changed by no more than 4.7 points from clean images, while retain ROUGE-L fell by no more than 1.8 points.
An ablation compared the two-stage design with a single-stage variant. At the 10% MLLMU-Bench split, the single-stage variant reported forget, retain and celebrity classification/ROUGE pairs of 32.8/38.7, 29.9/36.0 and 19.3/26.1. The two-stage IDK row reported 39.2/49.9, 38.5/51.7 and 51.7/26.1, respectively. The paper interprets the single-stage pattern, with lower retain and celebrity metrics arriving alongside lower forget metrics, as general degradation rather than selective forgetting.
What the study leaves open
Several constraints narrow what can be concluded. AIM updates only the vision encoder, so the study does not show that identity knowledge in a text-only modality was removed. The method also depends on a full Fisher statistic precomputed during fine-tuning; the paper notes that it may become inaccurate under longer schedules or larger updates. The robustness checks cover only the listed image perturbations, not prompt-side, cross-modal or adaptive attacks, and all reported results use one fixed random seed.
The authors describe AIM as a retain-data-free approach seeking a balance between forgetting and retention, with visual perception preserved and continual unlearning supported. The evidence is limited to computational tests of named models and benchmark data. It does not establish complete cross-modal deletion, human privacy protection or legal deletion compliance. No uncertainty intervals or statistical significance estimates are reported for the main numerical results. The work is an arXiv version 1 preprint dated 28 Aug 2026, and it reports support from an IITP grant funded by the Korean government through MSIT for development of cooperative on-device AI among networked devices.
Paper data and sources
Original title: AIM: Anchor Identity Features, Then Match for Multimodal Large Language Model Unlearning
Authors: Wonjun Lee, Jaehyuk Jang, Kangwook Ko et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-28
DOI: Not available
Original paper · Full text