Security Vulnerability Exposed: Researchers Extract Hidden Reasoning Traces from Proprietary LLM APIs
A significant security vulnerability has been identified in proprietary Large Language Model (LLM) APIs, allowing for the extraction of hidden reasoning traces. Researchers discovered that model providers return reasoning as encrypted blocks to clients, which are intended to be portable for conversation continuity. However, by replaying these blocks within weaker, jailbroken models from the same provider, the raw reasoning of stronger models—such as Claude Opus—can be extracted verbatim. This technique, demonstrated across OpenAI, Anthropic, and Google models, has led to the leakage of technical identifiers, personally identifiable information (PII), and credentials. The study analyzed 120 Codeforces problems, showing a direct correlation between reported hidden thinking tokens and the decoded reasoning length.
Key Takeaways
- Researchers have discovered a method to steal hidden reasoning traces from proprietary LLM APIs by replaying encrypted blocks in jailbroken models.
- The vulnerability affects major frontier models from providers including OpenAI, Anthropic, and Google.
- Extracted data includes sensitive information such as technical identifiers, PII, and credentials.
- The decoded reasoning length closely matches the hidden thinking-token counts reported by the APIs.
In-Depth Analysis
The Mechanism of Reasoning Extraction
The vulnerability stems from how model providers handle a model's internal reasoning process. When a user interacts with an LLM, the reasoning trace is returned to the client as an encrypted block. This block is designed to be portable, meaning it is sent back to the server to maintain context as the conversation progresses. Researchers found that these blocks can be replayed outside their original context. By injecting an encrypted reasoning block from a high-performance model (like Claude Opus) into a weaker, jailbroken model from the same provider (like Claude Haiku), the raw reasoning can be transcribed verbatim.
Data Leakage and Model Correlation
The study demonstrated the effectiveness of this attack across 120 Codeforces problems. By comparing the number of hidden thinking tokens reported by the API against the token count of the decoded reasoning, researchers confirmed a near-perfect match. Beyond mathematical logic, the extraction process revealed a significant amount of sensitive data. Out of the collected items, the researchers identified 351 distinct leaked items, including 204 technical identifiers, 126 instances of PII, and 23 sets of credentials.
Industry Impact
This discovery poses a major challenge for AI safety and intellectual property protection. The ability to extract "hidden" reasoning undermines the competitive advantage of proprietary models and exposes internal logic that providers intended to keep private. Furthermore, the leakage of PII and credentials through reasoning traces highlights a critical privacy risk in how encrypted state data is managed and validated across different model tiers within the same ecosystem.
Frequently Asked Questions
Question: How are the reasoning traces extracted from the API?
Reasoning traces are extracted by taking the encrypted reasoning block returned by a high-end model and replaying it as an injection into a weaker, jailbroken model from the same provider, which then transcribes the hidden text.
Question: Which AI providers are affected by this vulnerability?
Researchers successfully demonstrated this extraction technique across frontier models from OpenAI, Anthropic, and Google.
Question: What kind of sensitive information was leaked during the research?
The research identified several categories of leaked data, most notably technical identifiers, personally identifiable information (PII), and credentials.
