CIA measures how often a model’s chain-of-thought names the reasoning strategy that interpretability probes actually detect inside the network, and using that agreement as a post-training reward lifts faithfulness by roughly 25% on three reasoning tasks without hurting accuracy.
When an LLM shows you its reasoning, you tend to trust it. Suppose you’re reviewing a model that answers a medical multiple-choice question and writes a careful explanation ending in option C. If the model actually picked C because a user-provided hint said “I think it’s C,” and the written reasoning just rationalized that, you’ve been misled. This isn’t hypothetical. Prior work (Turpin et al. 2023) showed that CoT traces can be edited or injected with misleading cues without the model acknowledging it.
The field now distinguishes two flavors of faithfulness: self-consistency (does the model give the same explanation twice?) and parametric faithfulness (does the explanation match the model’s internal computation?). This paper tackles the second. The dominant prior approach measured parametric faithfulness only indirectly: inject a hint, see if the answer flips, and check whether the CoT mentions the hint. That’s one narrow probe of one failure mode.
The idea: use interpretability tools as a second opinion on what the model is “really” doing, then check whether the CoT matches.
For each task, the authors pick a gold reasoning strategy S (e.g., “solve this two-hop question by going through the bridge entity”). They define two binary flags per example: did the written CoT use strategy S, and did an interpretability tool detect strategy S inside the model’s activations? CIA is the macro-F1 between these two flags across the dataset. Perfect faithfulness means the two always agree, whether or not the final answer is correct. The internal flag comes mostly from Linear probes trained to detect task-specific signals, like whether a bridge entity is encoded at a particular token position, or whether a misleading hint shifted the model’s answer distribution.
Three tasks cover three reasoning flavors: TwoHopFact (compose two facts via a bridge entity), MMLU-Hint (MMLU questions with a misleading “I think the answer is X” suffix from Chen et al. 2025 (biased-hint MMLU)), and 2-digit multiplication (does the model actually sum the partial products it writes, or recall the answer directly and backfill?). For multiplication, the probe label comes from a causal corruption test: during generation, overwrite one partial product with a wrong value and check whether the final sum follows the corrupted intermediate.
Then comes the training loop. For each prompt, sample 16 completions, label each one with (CoT flag, internal flag), and feed that into one of three methods:
for prompt in train_set:
completions = model.sample(prompt, n=16, temp=1.0)
for y in completions:
b_cot = check_cot_uses_strategy(y)
b_int = probe(hidden_states(y))
r_acc = task_correct(y)
r_faith = 1 if b_cot == b_int else 0
reward[y] = r_acc + lambda * r_faith
# then: RS keeps aligned completions; DPO pairs high- vs low-reward;
# GRPO uses group-relative advantages
The accuracy term matters: without it, the model could trivially satisfy the faithfulness reward by, say, always claiming direct recall on multiplication. Direct Preference Optimization and Group Relative Policy Optimization (GRPO) are standard preference and RL methods borrowed as-is.
Baseline faithfulness is bad. Across Llama 3.1 8B Instruct, Qwen3-8B, and Gemma 2 9B IT, CIA ranges 0.448 to 0.759. Higher accuracy does not imply higher faithfulness: on TwoHopFact, Gemma2 is the most accurate but least faithful.
Failure modes differ by task. On TwoHopFact, the dominant pattern is CoT=1, internal=0: the written reasoning names a bridge entity the model never actually recalled, suggesting a shortcut to the answer. On multiplication, Qwen3 and Gemma2 write arithmetically coherent partial products but the answer comes from direct recall, not the written sum.
Post-training works, with a task-dependent best method. rejection sampling wins on TwoHopFact (+0.12 to +0.24 CIA), DPO on MMLU-Hint (+0.09 to +0.16), and either RS or DPO on multiplication (+0.09 to +0.11 at best). Accuracy holds or improves. GRPO is weaker.
The mechanism of improvement splits two ways. Transition analysis plus causal interventions show that on multiplication, trained models change how they reason: the causal effect of corrupting a written partial product jumps from 29% to 72% on the subset of newly-faithful samples, meaning the model now genuinely routes the answer through its displayed work. On TwoHopFact and MMLU-Hint, causal effect rates barely move; the model changes how it reports, learning to verbalize what it was already doing internally.
Gains partially generalize. Training on TwoHopFact helps MMLU-Hint and vice versa (both are “change how you report” tasks). Multiplication neither helps nor is helped by the other two. Gains also transfer across interpretability tools: a model trained with linear-probe labels also looks more faithful under Tuned lens or attention-pattern analysis, which is some protection against the model simply learning to game the probe.
If you’re evaluating whether a model’s CoT can be trusted as an explanation, this paper gives you a concrete recipe beyond the hint-injection paradigm: define a gold strategy, train a probe for it, and compute macro-F1 between “CoT says S” and “probe says S.” The code is at GitHub. You need open weights to run the probes, so this won’t work through a hosted API.
If you’re post-training a model and faithfulness matters, the strongest signal here is that even plain rejection sampling on “CoT and internals agree” moves the needle without hurting accuracy. Worth testing on your own task before reaching for DPO or GRPO. The authors evaluated 8-9B models on narrow single-step reasoning tasks; whether the recipe scales to long-chain reasoning is explicitly flagged as open.
For interpreting published “faithfulness” claims: distinguish “model changes how it reports” from “model changes how it reasons.” The paper shows these are different outcomes with different causal signatures, and that a CIA number alone does not tell you which one happened.
One finding worth internalizing even without implementing anything: on multiplication, writing a coherent step-by-step derivation is NOT evidence that the model used those steps. The baseline Qwen3 and Gemma2 models produce correct partial products and correct sums while actually recalling the answer directly.
CIA is only as good as the probe. The authors report held-out probe accuracy of 0.83 to 0.91 and show the multiplication probe is not just tracking correctness, but a sharper interpretability tool could change the numbers.
All three tasks isolate a single reasoning step. Real applications combine many, and the authors note that faithfulness objectives for different sub-skills can conflict.
The one preliminary scaling point (Qwen3-14B on multiplication) actually gets less faithful than the 8B version while being more accurate. Larger does not mean more honest.
For reasoning models with long thinking traces, the picture gets murkier: the thinking trace tends to over-mention hints that didn’t actually drive the answer, so judging CoT faithfulness on the full trace versus the final answer gives different CIA scores.