A benchmark designed to test whether retrieved interaction history can derail a model’s current reasoning found the same broad pattern in both systems it examined: every memory strategy scored below a no-memory setup. Even the strongest strategy was more than 10 percentage points lower on each model.
The work, called MemTrapBench, contains 1,050 deliberately constructed multi-turn cases across four scenarios: Cognitive Bias, Task Boundary, Safety and Trauma. The cases were expanded into trap dialogues with GPT-5.4, then passed through automated filtering and expert human review before evaluation. It is an arXiv preprint, version v1, dated 20 Aug. 2026.
The gap appeared in both models
On Gemini-3-Flash-Preview, the no-memory average was 85.16%, compared with 71.17% for EverMemOS, the highest-scoring memory strategy. On Qwen3-30B-A3B-Instruct-2507, no memory averaged 81.83%, while LightMem led the memory strategies at 70.13%.
Overall response quality was judged on correctness, format, relevance and efficiency. The evaluation compared five memory strategies with the no-memory baseline and used three independently generated responses per setting; the reported figures are descriptive averages, with no confidence intervals or p-values.
Some traps mattered more than others
Trap-free controls scored higher than trap-inducing versions in the reported Trauma and Task Boundary comparisons. In Trauma, the average was 84.33% without the trap versus 69.43% with it, and correctness was 91.07% versus 66.40%. In Task Boundary, the corresponding averages were 94.39% and 31.05%, while the no-memory comparison scored 92.29%.
An analysis of memory length pointed in the same direction: average score was 36.03% with 25% of the memory history and 31.05% with the full history, compared with 92.29% without memory. The paper reports a 3.40-point drop between 25% and 50% memory.
A prompt recovered part of the lost ground
To address these failures, the authors tested AdaptiveMem, a prompt-based skill intended to make the model reconsider retrieved memories and avoid memory-induced traps. In the reported Gemini comparisons, adding it was associated with gains of 11.8 percentage points for FullText, 14.9 for LightMem and 11.3 for EverMemOS.
On Qwen, the reported gains were 4.2 points for FullText, 2.5 for LightMem and 2.6 for EverMemOS. In a secondary LongMemEval comparison, four of six settings improved and two were unchanged, with gains of up to 4.0 points for Gemini and 3.0 for Qwen. That comparison randomly sampled 200 instances from each benchmark.
A benchmark signal, not a real-world outcome
Both judging models showed the same direction: across three independent responses per setting, GPT-5.2 gave no-memory outputs an average 92.29%, versus 31.05% with memory; Claude-Sonnet-4.6 gave 95.57% versus 40.07%. GPT-5.2 was the primary judge and Claude-Sonnet-4.6 was used for consistency checks.
That result does not show that memory causes the same failures in general use. The evidence is limited to judge-scored outputs from two named models, five memory strategies and deliberately designed benchmark cases; it includes no human or clinical outcomes. Formal inferential tests were not reported.
The paper says its data and code will be available at the MemTrapBench GitHub repository, but it does not establish whether they are currently accessible. Open questions include whether the pattern replicates across more models and memory architectures, whether expert or human reviewers would see the same failures, and whether AdaptiveMem transfers to tasks outside these benchmarks without adding latency or computational cost.
Paper data and sources
Original title: MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use
Authors: Mengru Wang, Haozhe Luo, Zhenqian Xu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text