Reverse-engineering neural networks at the level of circuits and features — figuring out what specific weights and activations compute, not just what the model outputs.
Mechanistic Interpretability: From Readable Features to Causal MechanismsMechanistic interpretability asks what internal states and components compute, not merely whether a model produces the right output. The central difficulty is evidential: a feature may be decodable from activations without guiding behavior, while a plausible geometric pattern may reflect lexical, positional, or numerical shortcuts.A useful reading order follows an evidence ladder. First identify what a probe measures and test it under controlled shifts. Then compare internal signals with behavior, intervene against matched controls, and test transfer across vocabulary, tasks, layers, or models. The readings below show why each step changes the strength and scope of the conclusion.The guide begins with language-model representations, moves to context-dependent reasoning and scientific relations, and ends with causal localization in diffusion transformers. Together these cases help distinguish an audit lead from a mechanism that can support prediction or control.
A Decodable Feature Is an Audit Lead, Not a Capability ContractUse this as the main evidence ladder. It connects logical-validity probes, materials steering, and multilingual probe disagreement, showing exactly how accessibility, behavioral use, intervention, and transfer require separate tests. It does not turn the three cases into one shared mechanism.Its strongest detailed results concern synthetic logical verification, one reported grain-size steering setup and transfer cohorts, and multilingual latent-language estimators.Read or listen · 15 min audio
What are you trying to solve?
Separate readable information from behavioral useLinear probes and other readouts establish that information is accessible under a specified measurement. They do not by themselves show that the model uses that information. Start here to learn how held-out splits, lexical baselines, incorrect-answer analysis, random-direction controls, and disagreements between probe families narrow an internal-feature claim.
A Decodable Feature Is an Audit Lead, Not a Capability ContractUse this as the main evidence ladder. It connects logical-validity probes, materials steering, and multilingual probe disagreement, showing exactly how accessibility, behavioral use, intervention, and transfer require separate tests. It does not turn the three cases into one shared mechanism.Its strongest detailed results concern synthetic logical verification, one reported grain-size steering setup and transfer cohorts, and multilingual latent-language estimators.Read or listen · 15 min audio
When Decodability Is Not Enough: Logical Validity Representations, Behavioral Dissociation, and Causal Tests in Language ModelsRead this for a clean dissociation between internal accessibility and output behavior: validity remains decodable under several shifts and on some wrong answers, while probe-direction interventions are weak and nonspecific. It does not show that validity information is causally irrelevant at every site or encoding.Five open-weight transformers on matched synthetic premise–claim pairs; causal tests target the reported probe directions.
Lingua Franca or Probing Artifact? Rethinking Latent Language in Multilingual LLMsRead this before treating a probe label as a literal internal variable. Geometry-based and decoding-based latent-language probes systematically disagree across models and up to 27 languages. The study documents estimator dependence but does not determine which probe reflects operative computation.The abstract reports cross-model, task, domain, checkpoint, and language comparisons without causal intervention.
Why can two probes give different accounts of the same hidden states?Probe families can measure different properties. In multilingual models, a geometry-based GMM probe showed earlier cross-lingual mixing, while decoding-based probes retained sharper language-specific and English-biased signals. The disagreement means “latent language” depends on the estimator; it does not identify which probe tracks the model’s operative computation.A Decodable Feature Is an Audit Lead, Not a Capability ContractLingua Franca or Probing Artifact? Rethinking Latent Language in Multilingual LLMs
Reverse confounds and preserve contextA representation can look organized for the wrong reason. Matched reversals hold most prompt features fixed while changing the relevant relation; held-out representations and identical-token comparisons test whether geometry follows an operation rather than surface form. These readings show how controlled contrasts strengthen a claim without turning every separable pattern into causal control.
Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language ModelRead this for a constructive design that replaces absolute-state patterns with matched state changes and causal interventions. Reversing physical inputs helps distinguish constitutive relations from numerical or lexical shortcuts. The abstract does not establish a universal scientific-reasoning representation.Three open-weight Gemma 4 models; 60 frozen relations, 40 directional laws, and 12 matched intervention cases.
Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMsRead this to see how operation labels can map onto layer-dependent hidden-state geometry after lexical and positional controls. Identical-token and attention-masking results make context dependence concrete. They do not establish that operation separability causes correct answers.Evidence concerns the studied chain-of-thought operation taxonomy and held-out hidden representations.
Formalizing Latent Thoughts: Four Axioms of Thought Representation in LLMsRead this for explicit functional criteria—causality, minimality, separability, and stability—that evaluate representations independently of downstream scores. The audit reveals failures hidden by benchmark accuracy, but the abstract does not show that these four axioms exhaust representation quality.Audit of open-weight dense, reasoning-distilled, and RL-trained model families across 23 reasoning tasks.
How can a study test whether a model tracks a scientific relation rather than an answer-correlated shortcut?Compare otherwise matched prompts in which the physical input direction reverses, then ask whether the hidden-state change follows direct, neutral, or inverse laws. In the Gemma 4 study, controlled transformations oriented 39 of 40 directional laws, while lexical controls were near chance; interventions also shifted constrained answers in all 12 matched cases.Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model
What makes hidden-state geometry evidence about reasoning operations rather than token identity or position?Use held-out representations, lexical and positional controls, and compare identical tokens inside different operation contexts. The chain-of-thought study reports middle-layer separability, distributed span-level alignment, and attention-masking evidence that chunk-onset states depend on preceding reasoning context. This supports contextual organization, not proof that the geometry drives correct reasoning.Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs
How can representation quality be evaluated without relying on downstream accuracy?The axiomatic framework measures causality, minimality, separability, and stability directly on candidate representations. Its audit across 23 reasoning tasks reports that no candidate passed all four, within-task questions were not distinguished, and little information was added beyond input embeddings. It is useful for defining stronger targets than benchmark success.Formalizing Latent Thoughts: Four Axioms of Thought Representation in LLMs
Trace and intervene on generative circuitsMechanistic analysis can localize computation across token spans, attention heads, and depth, then test the resulting hypothesis by intervention. Diffusion transformers provide a useful case because prompt tokens, image latents, and structural template tokens can exchange semantic information during denoising; the tokens that receive semantics initially need not be those that maintain object identity.
Text Template Tokens Are Implicit Semantic Registers in Diffusion TransformersRead this for an end-to-end example of attention decomposition, targeted intervention, mechanistic hypothesis, and efficiency application in diffusion transformers. It traces semantics through image latents into template-token registers and motivates pruning, but does not establish the same routing in other generative architectures.Modern text-to-image diffusion transformers; pruning reports 20% attention-FLOP removal with a 1.4-point GenEval decrease.
How can structural template tokens maintain object identity without carrying much prompt-specific information initially?The reported pathway first injects prompt semantics into image latents, which are then read back into template tokens. Those tokens become dominant image-to-text attention sinks and act as semantic registers during generation. Targeted interventions support their role in maintaining identity across the tested diffusion-transformer computation.Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers
When can a mechanistic finding justify attention-head pruning?When intervention and decomposition identify heads whose removal preserves the target function, the interpretation can motivate a pruning rule. Here, pruning heads that attended most strongly to prompt tokens removed 20% of attention FLOPs with a 1.4-point GenEval drop. That result evaluates the reported DiT rule, not attention-head pruning in language models.Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers