The hidden signal in ordinary answers
Some AI models appeared to encode secrets in ordinary answers even when instructed not to reveal them, according to a new preprint. The study asked whether a model protecting a value in its context could still carry information about that value in other observable output properties, allowing an external decoder to reconstruct it.
Researchers placed numeric secrets in the context of eight proprietary frontier language models and told them to keep those values confidential while answering unrelated, benign prompts. They examined the outputs with model-specific adaptive black-box decoders, which use observed response patterns to infer the hidden value.
Primary tests sampled 1,000 secrets for each model and secret length and collected 10 responses for each secret-prompt pair. The adaptive pipeline used 50,000 secrets, with 80% assigned to training, 10% to validation and 10% to testing by secret instance.
Short numbers were easiest to recover
Recovery was strongest for short numbers. In the reported tests, the decoder reconstructed 100% of two-digit secrets on both Claude Opus 4.6 and Gemini 3.1 Pro; on Claude, it got the full four-digit value right 82% of the time. On Gemini, it recovered individual digits in eight-digit secrets with 41% accuracy, compared with 10% chance.
With prompts conditioned on digits recovered so far, Claude's four-digit full-secret recovery was 82%, versus 44% with static templates. Two models in a resistant tier stayed at chance for secrets of four digits or more.
Clues survived beyond the digits
The signal was not limited to numbers. In an auxiliary test, routine outputs supported inference of semantic attributes of memories: the proposed decoder's advantage over chance was 0.319, versus 0.058 for a model-agnostic linguistic judge. At the judge's false-positive rate, the decoder's true-positive rate was 1.7 times higher. The test used a single CIMemories user profile.
A separate, active prompt-injection test used an attacker-engineered output channel against a production-style agent. It recovered digits of a nine-digit SSN: the leading digit was recovered in 97.1% of Claude Opus 4.6 trials and 88.6% of Gemini 3.1 Pro trials, while the remaining eight digits were recovered in 76.5% and 46.8%, respectively. A direct request for the SSN did not elicit it.
Wording and training changed the signal
Instruction wording changed the result for some models. Across five tested instructions, Claude Opus 4.6's per-digit accuracy ranged from 5.4% under terse wording to 64.3% under a verbose policy. Gemini 3.1 Pro's accuracy never fell below 65.6% across those instructions.
A separate comparison of OLMo-3-32B-Think's post-training stages found channel entropy, a measure of the output channel's information capacity, at 2.91 bits in the base model and 1.37 bits after RLVR. The reported stage changes were -0.98 bits for SFT and -0.52 bits for RLVR, while DPO was flat. Because this was a stage comparison rather than a randomized intervention, it does not show that a particular training stage caused the change.
Removing digit characters did not eliminate the signal. In an auxiliary analysis of 178 real WildChat prompts and 80 held-out single-digit episodes, all six analyzed models showed significant non-digit leakage. Claude Opus 4.6 reached 61.3% accuracy using digit-stripped embeddings.
Randomized-label controls remained at chance: none of 42 control cells reached significance, and no multiple-comparison correction was applied.
The evidence has a narrow reach
The authors say it remains open whether the findings generalize beyond the tested predicate families, models and deployments. Open-vocabulary, free-text secrets were not tested.
The primary numeric data were split by secret instance, but the overlap with training varied by length: all 63 distinct two-digit test values also appeared in training, compared with five of 98 four-digit values and none of the eight-digit values. The split therefore did not always test entirely unseen numeric values.
The memory-predicate evaluation used one CIMemories user profile, and the study did not evaluate deployment-side output perturbation or response-length normalization.
Taken together, the tests support a limited warning: refusing to print a secret does not necessarily mean that a model's other observable outputs carry no information about it. Whether the same pattern holds for other models, deployments or free-text secrets remains unanswered.
Paper data and sources
Original title: Inadvertent Context Leakage in Language Models
Authors: Jaiden Fairoze, Neal Mangaokar, Kamalika Chaudhuri et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text