Stolen Thoughts: How Encrypted AI Reasoning Traces Exposed Secrets and Model IP

By AiCybr Editorial · Published August 12, 2026

A new security paper shows that encrypted reasoning blocks returned by major frontier-model APIs should not be treated as harmless opaque metadata. Researchers found that these blocks could be moved across sessions, users, and models within the same provider ecosystem, then decoded through a weaker sibling model.

The work, published on August 10, 2026 as Stealing Reasoning Traces from Proprietary LLM APIs, demonstrates the issue across Anthropic, OpenAI, and Google systems studied by the authors.

The most important result is not merely that hidden reasoning can sometimes be reconstructed. The paper describes a broader architectural failure mode: encrypted reasoning artifacts were portable enough to become reusable attack inputs.

That creates four distinct risks:

1. extraction of proprietary reasoning traces for model distillation;
2. recovery of secrets and personal data from publicly shared agent logs;
3. exposure of harmful information that appeared only inside hidden reasoning;
4. invisible prompt injection carried inside encrypted reasoning blocks.

The researchers decoded 315,320 reasoning blocks collected from public repositories and report finding 367 personally identifiable information artifacts and 182 credentials.

Those numbers come from the research paper, not an independent internet-wide measurement. They should therefore be read as evidence that the exposure class is real, not as an estimate of how many users or repositories remain vulnerable today.

What the researchers actually found

Modern reasoning APIs may return an opaque or encrypted representation of a model's intermediate reasoning to the client. The client can then send that object back in a later request so the model can continue a conversation without exposing the plaintext reasoning to the application developer.

That sounds like a confidentiality boundary, but the paper identifies a crucial weakness: the encrypted blocks were sufficiently interchangeable across contexts that an attacker could take a block produced by one model and submit it to another model inside the same provider ecosystem.

The attack does not require breaking the cryptography directly.

Instead, the researchers use a weaker or less-restricted sibling model as a decoding oracle. A reasoning block generated by a stronger model is supplied to the weaker model, which is then prompted to emit the corresponding plaintext reasoning.

The paper summarizes the core issue as compatibility across:

• different sessions;
• different users;
• different models within a provider family.

In security terms, the problem is therefore closer to missing contextual binding and unsafe replay than to conventional ciphertext cracking.

Why encrypted does not automatically mean safe

Encryption protects data only within the assumptions of the surrounding protocol.

If an encrypted object can be copied from one security context into another and still be accepted as valid, the system may preserve confidentiality at the byte level while failing at the authorization level.

A useful analogy is a signed access token. A token can be cryptographically authentic and still be dangerous if it is not properly bound to the intended user, audience, session, privilege level, or resource.

The Stolen Thoughts result applies the same lesson to AI reasoning artifacts.

For application developers, the practical rule is simple:

Treat encrypted reasoning blocks as sensitive capability-bearing data, not as disposable telemetry.

They may contain information the visible assistant response never exposes.

The four attack paths

1. Proprietary reasoning extraction

One reason model providers hide chain-of-thought is to reduce leakage of internal reasoning behavior and make large-scale distillation more difficult.

The researchers show that portable encrypted traces can weaken that boundary. If an attacker can recover the plaintext reasoning of a stronger model by presenting its encrypted block to a weaker sibling, the attacker gains training material that would otherwise remain inaccessible.

This does not mean a complete frontier model can be cloned from a handful of traces. Model replication depends on scale, coverage, training methodology, data quality, and many other factors.

But the paper demonstrates that the encrypted trace can become an unexpected side channel for model intellectual property.

2. Credentials and PII in public agent logs

This is the most immediately actionable result for ordinary developers.

Agent frameworks and API wrappers often persist full request/response objects for debugging, reproducibility, observability, or evaluation. Developers may see an encrypted reasoning field and assume it is safe to publish because its contents are unreadable.

The paper shows why that assumption is unsafe.

From 6,708 public agent trajectories, the researchers decoded 315,320 reasoning blocks and report recovering:

The security failure can therefore occur even when the final visible answer is clean.

A model may manipulate an API key, password, email address, private path, customer identifier, database credential, or other secret while reasoning through a task. If the framework persists that reasoning artifact, the secret may survive inside the log even though it never appears in ordinary output_text.

3. Hidden hazardous content

Safety systems frequently focus on what the user ultimately receives.

The paper reports cases where the model's internal reasoning contained hazardous information even though the visible answer refused the harmful request.

If hidden reasoning can later be extracted through another model, the refusal boundary is weaker than it appears from the final output alone.

This does not prove that every refusal contains dangerous hidden instructions. It shows that systems cannot assume hidden intermediate content is harmless simply because the user-facing response is safe.

4. Invisible prompt injection

The fourth attack is especially relevant to agent systems.

The researchers describe embedding malicious instructions inside encrypted reasoning blocks so the payload is not visible to humans reviewing the surrounding trajectory. If downstream systems accept and replay those blocks, they can inherit hidden instructions from data that looks opaque and benign.

That creates a supply-chain-like risk for:

• shared agent trajectories;
• benchmark datasets;
• saved conversations;
• debugging fixtures;
• replay systems;
• synthetic training/evaluation corpora.

The lesson is broader than this specific attack: opaque model-generated state should not automatically cross trust boundaries.

What developers should do now

The paper says the issue was responsibly disclosed and proposes cryptographic and system-level mitigations. Provider-specific deployment status may change quickly, so application teams should not wait for a single vendor-side fix before improving their own handling.

1. Stop publishing raw reasoning objects

If logs, traces, bug reports, fixtures, or GitHub issues contain the complete provider response object, review whether encrypted reasoning fields are present.

Do not assume an unreadable blob is safe to make public.

Prefer an explicit allowlist for observability fields rather than serializing the entire API response.

For example, retain only what the system genuinely needs:

• request identifier;
• model name;
• latency;
• token counts;
• tool-call names and sanitized arguments where appropriate;
• final output after redaction;
• application-specific success/failure metadata.

2. Treat reasoning artifacts like secrets

Encrypted reasoning should receive controls similar to other sensitive execution state:

• avoid committing it to source repositories;
• encrypt it at rest when persistence is required;
• restrict access by service and environment;
• apply retention limits;
• prevent cross-tenant reuse;
• redact it from support bundles and telemetry exports.

The correct classification is closer to a session token or encrypted credential envelope than to harmless debug text.

3. Audit historical repositories and logs

Teams that have publicly shared agent runs should search for provider-specific reasoning object fields and other opaque response payloads.

If a historical log may have contained credentials inside hidden reasoning, merely deleting the log may not be sufficient. The associated secret should be considered for rotation because repository history, forks, caches, mirrors, or downloaded artifacts may persist.

4. Rotate exposed credentials rather than relying on deletion

If an API key, password, access token, database credential, signing secret, or cloud credential appears to have been processed inside a publicly shared trajectory, rotate it.

Deletion lowers future exposure but does not invalidate a copied secret.

5. Do not replay opaque state across users or tenants

Multi-tenant agent platforms should bind persisted model state to the user, tenant, session, and expected model context.

Application code should reject unexpected cross-context reuse even if the upstream provider also performs its own checks.

6. Sanitize imported agent trajectories

Treat downloaded or community-provided agent logs as untrusted input.

Before replaying them:

• strip encrypted reasoning/state fields unless they are explicitly required;
• remove secrets and private identifiers;
• validate tool calls independently;
• prevent imported state from granting authorization;
• run evaluation datasets in isolated environments.

7. Separate model state from authorization

No hidden reasoning block, conversation object, tool suggestion, or model-generated token should decide whether an action is authorized.

Real authorization should come from application-controlled policy: authenticated identity, scopes, resource ACLs, transaction limits, confirmations, and sandbox boundaries.

This matters because invisible prompt injection is far less damaging when the model cannot independently turn hidden instructions into privileged actions.

What providers can change

The paper argues for cryptographic and system-level fixes. At a high level, secure designs should prevent a reasoning artifact produced in one context from functioning as a valid artifact in another unrelated context.

Potential protocol properties include binding an opaque reasoning object to some combination of:

• account or tenant;
• conversation/session;
• originating model;
• intended receiving model or model class;
• expiration time;
• request sequence or nonce;
• permitted operation.

The exact implementation belongs to the provider and must preserve legitimate continuity use cases.

The key requirement is that possession of a valid opaque block should not imply permission to replay it anywhere the provider can technically parse it.

Why this matters for AI-agent architecture

The Stolen Thoughts paper is useful beyond the specific providers tested because it exposes a recurring design mistake in agent systems: model state is often moved between components with weaker trust controls than ordinary application state.

Agent stacks increasingly pass around:

• conversation histories;
• tool results;
• memory objects;
• resumable reasoning state;
• checkpoints;
• browser snapshots;
• embeddings;
• MCP/tool payloads;
• evaluation trajectories.

Every one of these artifacts can become a security boundary if it can influence future execution.

A mature agent architecture should therefore answer four questions for every persisted object:

1. Who created it?
2. Who is allowed to consume it?
3. What authority can it influence?
4. Can it be safely replayed in another context?

If those answers are unclear, the artifact should not automatically cross users, tenants, environments, or trust zones.

What the paper does not prove

There are several important limits to the finding.

First, this is a newly published research result. The paper provides primary evidence, but it should not be interpreted as a statement that every current model endpoint remains exploitable in exactly the demonstrated form.

Second, the counts of recovered credentials and PII come from the researchers' collected public trajectories. They are not a global prevalence estimate.

Third, reasoning extraction is not equivalent to recovering model weights or perfectly cloning a frontier model.

Fourth, the research does not imply that encryption itself is useless. The failure is in how encrypted reasoning state interacts with replay, context, and model compatibility.

Finally, provider mitigations can evolve rapidly after responsible disclosure. Application developers should verify the latest provider documentation before depending on a particular reasoning-state behavior.

Practical risk matrix

Bottom line

The main security lesson from Stolen Thoughts is that opaque AI state is still state.

Encryption can hide what a reasoning object contains while leaving dangerous replay properties intact. If that object can be moved between users, sessions, models, datasets, or agent runs, it deserves the same threat modeling as any other bearer-style security artifact.

For developers, the immediate changes are straightforward: stop publishing raw model-response objects, strip unnecessary reasoning fields from logs, rotate secrets that may have entered public trajectories, isolate imported agent data, and never let model-generated state substitute for authorization.

The broader lesson will remain relevant even if providers completely close the demonstrated attack: as AI systems become more stateful, the security boundary is no longer only the prompt and final answer. It includes every hidden artifact passed between them.

Sources

• Alexander Panfilov et al., Stealing Reasoning Traces from Proprietary LLM APIs, submitted August 10, 2026.
• OpenAI, GPT-5.6 System Card, for broader context on chain-of-thought evaluation and model safety; it is not evidence for the Stolen Thoughts attack itself.