A new preprint reports that a tool-call technique recovered hidden chain-of-thought (CoT)—the step-by-step reasoning text inside a reasoning model—on DeepSeek-V4-Flash. On the OpenThoughts test set, its EchoCoT-LTGO variant met the study’s near-verbatim ASR@90 threshold in 66.4% of cases, compared with 0.3% for REP and zero for Direct Prompting and CoT Synthesis.
Called EchoCoT, the method works through API interactions. It uses a reasoning-replay surface between tool calls, injects instructions iteratively using API-returned fidelity signals, and relies on an LLM optimizer to search for a universal injection trajectory.
The default trajectory allowed up to three injection steps. The last step was fixed to accept the scratchpad and proceed to the final answer, so only the first two injections were optimized.
What counted as a recovery
The paper’s extraction-success measure, ASR, used three thresholds: ASR@99, ASR@95 and ASR@90. At ASR@90, a recovered trace had to be within 10% of the target length and meet a token exact-match threshold of at least 90%. Every sample was included in the denominator, and cases with no tool call counted as failures.
The accessible-CoT evaluation covered three large reasoning models. Their raw traces were withheld from both the attacker and the injection optimizer but retained for evaluation. OpenThoughts supplied nine sources, with 100 questions sampled from each; 540 went to optimization and 360 to a non-overlapping test set. Transfer checks used 100 questions from each of three unseen datasets.
The clearest successes
On the DeepSeek test set, LTGO achieved 46.1% at ASR@99 and 61.9% at ASR@95. At the looser ASR@90 threshold, it reached 66.4%. The analysis reports no confidence intervals for these descriptive results.
One long DeepSeek example was almost exact: 21,109 extracted tokens versus a 21,106-token target, with Token-EM of 0.999. Only three of 1,132 lines differed. That was a single example, not an estimate for all questions.
The trajectory optimized on OpenThoughts was also tested on other datasets. For DeepSeek-V4-Flash, ASR@90 reached 80% on MATH500, 71% on JEEBench and 64% on LiveCodeBench.
Across the three accessible-CoT models, EchoCoT variants were reported to outperform existing methods on textual-fidelity measures—how closely the recovered wording matched the target—with LTGO the highest EchoCoT variant for Token F1, ROUGE-L and Token-EM.
Performance was weaker for the longest traces: ASR@90 ranged from 59% to 86% for CoTs shorter than 16K tokens, but was 32% for CoTs longer than 16K.
The step analysis found no ASR@90 successes after the first tool call. Successful extractions appeared after the second, and the third increased cumulative success, especially for DeepSeek-V4-Flash and GLM-5.2.
Failures were usually about length. Under-extraction, meaning text that was too short, made up 45.6% to 64.0% of failures; over-extraction accounted for 20.2% to 26.2%; and the two length-mismatch categories together represented 71.8% to 84.2%. The study found no refusal responses in the three target models.
The proprietary results are harder to verify
The researchers also evaluated five frontier proprietary reasoning models on 400 questions. Tool invocation rates were 95% to 100%. Summaries were available for at least 87.8% of questions for every model except Opus-4.6, where availability was 31.5%.
But raw CoTs from those systems were not available for direct comparison. The paper therefore relied mainly on provider-reported CoT lengths and summaries as reference proxies, so its proprietary results do not directly establish near-verbatim copying.
In one reported Gemini-2.5 example, the method produced 33,463 tokens from a 32,948-token target. The close lengths alone do not show that the recovered wording matched the hidden trace.
As a check on the proxy, Summary Token Recall had a positive Spearman correlation of 0.532 with Token-F1 across 1,024 triplets from three target models, with p < 0.001. That relationship links the proxy to one fidelity measure; it is not a direct verification of proprietary CoTs.
A warning with a narrow perimeter
The study’s direct ground-truth comparison covered three accessible-CoT models, while its proprietary evaluation covered five models and 400 questions within selected public question sets and API/tool interactions. The findings do not establish that every provider, model version, interface or task is vulnerable, or what attack rates would look like in real-world use. The experiments used public questions, not real users’ private data.
The authors interpret the findings as a practical security risk and recommend stronger protection for hidden CoT assets. The work is an arXiv v1 preprint dated 20 Aug 2026.
Paper data and sources
Original title: EchoCoT: Extracting Hidden Chain-of-Thought from Large Reasoning Models
Authors: Yiting Qu, Ziqing Yang, Chi Cui et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text