Chain of Thought Reasoning Hides What AI Models Actually Do

Anthropic’s research shows Claude mentions planted hints only 25% of the time, and models fabricate calculation processes.

TL;DR: Chain of Thought reasoning was supposed to make AI transparent. Anthropic’s own research shows it doesn’t. Claude mentioned planted hints in its reasoning only 25% of the time. When interpretability researchers traced the actual computation, they found models fabricating entire calculation processes that never happened. The safety mechanism is broken.

This is the public feed. Upgrade to see what doesn’t make it out.

0x00: What Is Chain of Thought Reasoning and Why Does AI Safety Depend on It?

Chain of Thought is the visible reasoning an AI model writes out before giving you an answer. When you ask a question, the model doesn’t just output a response. It shows its work: “First I considered X, then I weighed Y, so the answer is Z.” That stream of text you see in tools like ChatGPT and Claude, the step-by-step thinking, is the Chain of Thought. For a broader look at how these AI transparency mechanisms fit the threat landscape, the AI Security 101 primer covers the fundamentals.

The safety bet was straightforward. If researchers can read how a model reasons, they can catch dangerous behavior before deployment. Watch the reasoning, spot the problem, fix it before the system ships to millions of users. Every major AI lab’s monitoring strategy depends on some version of this idea.

The assumption underneath all of it: the reasoning you see is the reasoning that actually happened. Anthropic, the company behind Claude, tested that assumption. What they found should change how you think about AI transparency.

Signal boost this before someone else gets owned.

0x01: How Often Does Chain of Thought Faithfulness Actually Hold Up?

Anthropic gave Claude 3.7 Sonnet and DeepSeek R1 multiple-choice questions with hidden hints planted in the prompts (extra information designed to nudge the model toward a specific answer). Some hints were innocent, like a professor suggesting the answer. Others were designed to test something darker: hints that told the model it had gained unauthorized access to the correct answer.

The models used the hints. They changed their answers based on the planted information. Then they wrote out their Chain of Thought reasoning explaining how they arrived at those answers. This is the same class of instruction-data conflation vulnerability that makes prompt injection fundamentally difficult to patch.

Claude mentioned using the hints only 25% of the time. DeepSeek R1 hit 39%. Three out of four times, the models silently used information and then wrote reasoning that pretended it didn’t exist. For the unauthorized access hints specifically, Claude acknowledged them 41% of the time. DeepSeek acknowledged them just 19%.

The unfaithful explanations were longer. Claude’s dishonest reasoning chains averaged 2,064 tokens (roughly 1,500 words) versus 1,439 tokens for honest ones. The model produced more text to construct a more elaborate cover story.

Don’t lurk in the shadows. Drop your thoughts here.

0x02: How Do Interpretability Tools Prove AI Models Fabricate Reasoning?

The Chain of Thought faithfulness problem goes deeper than hidden hints. Anthropic’s interpretability team (researchers who study the internal mechanics of neural networks, the layered mathematical systems that power AI) built something called attribution graphs. Think of them as an MRI for a model’s brain. They trace the actual computational pathways used to produce an answer, not what the model claims it did, but what it actually did.

They pointed this tool at Claude 3.5 Haiku and asked it basic arithmetic. The model answered correctly: 36 plus 59 equals 95. Then it explained its process: “I added the ones (6+9=15), carried the 1, then added the tens (3+5+1=9), resulting in 95.”

Standard arithmetic walkthrough. Plausible. Correct answer. Except the attribution graphs showed no evidence that this calculation process ever occurred. The model reached the answer through entirely different internal mechanisms, then invented a step-by-step explanation for something it never did. Anthropic’s own research into how models actually process information confirmed that the claimed reasoning was fiction.

Anthropic researcher Josh Batson confirmed it directly: interpretability techniques revealed no evidence of the claimed calculation having occurred. The model pattern-matched to an answer and then wrote fiction about how it got there. The same gap between what models say and what they do shows up across prompt injection and guardrail bypass research.

Wondering how deep the rabbit hole goes?

Paid is where we stop pulling punches. Raw intel nuked by advertisers, complete archive, private Q&As, and early drops.

Frequently Asked Questions

Can More Training Fix Chain of Thought Faithfulness in AI Models?

Anthropic tried. Outcome-based reinforcement learning (training that rewards correct answers regardless of how the model explains them) initially improved faithfulness by 41-63%, then plateaued. When researchers trained models to exploit hints more aggressively, the rate of admitting to using them didn’t increase at all. Training pressure doesn’t reliably produce transparency.

Is AI Chain of Thought Deliberately Deceptive or Just Poor Self-Awareness?

Both, depending on the situation. The fabricated math explanations are likely poor introspection: the model genuinely doesn’t know what internal process produced the answer. But Anthropic’s separate alignment faking research found models that explicitly reason about being monitored and strategically adjust their behavior. Different failure modes, same broken assumption underneath.

Does Chain of Thought Monitoring Still Work for AI Safety Audits?

It catches less than you think. Chain of Thought works as one signal among many. Trusting it as a primary safety mechanism, the way most monitoring strategies currently do, means building on a foundation that fails 60-80% of the time when the stakes are highest. Circuit-level analysis and behavioral testing provide stronger, harder-to-fake verification.