When a reasoning LLM writes a chain-of-thought, functional operations like decomposition, recall, and deduction form linearly separable clusters in mid-layer hidden states, and the same surface token (e.g., “is”) lands in different regions depending on which operation surrounds it.
If you ship an agent that produces long reasoning traces, today you probably grade them with another LLM or a regex over the text. This paper is evidence that the model already knows, in its own activations, which kind of reasoning step it is doing. That opens a cheaper monitoring surface: probe the hidden state instead of re-reading the text. The dominant baseline for this kind of question, Thought Anchors, stays at the text and trajectory level. This work drops down one layer and asks whether the categories we see in the text also exist as geometry inside the model.
The authors take a reasoning trace, cut it into contiguous spans, and label each span with one of eight operations from Polya’s problem-solving framework: Extraction, Direct mapping, Decomposition, Recall, Deduction, Algebraic manipulation, Arithmetic computation, Final answer. The labels come from GPT-5 and were spot-checked against seven human annotators (96.4% majority agreement, GPT-5 matched humans 76.2% of the time).
For each span they grab the hidden state at every layer of three reasoning models: qwen3-8b, Qwen2.5-7B-Instruct, and Gemma4-31B. They normalize, run PCA down to 128 dimensions, then fit LDA on training spans only. Inside that projected space, each operation gets a reasoning-operation vector: the normalized direction from the class mean to the mean of everything else. On held-out spans they score alignment as a dot product with that vector and report AUROC.
The interesting mechanics live in the follow-ups. To rule out “the probe just reads keywords,” they compare against bag-of-words and TF-IDF classifiers, run lexically matched pairs, and restrict to spans dense in digits and math notation. To rule out “it just reads position in the trace,” they train a position-only classifier and also stratify evaluation by relative position. To test causality, they re-run the model with attention masked from the first token of a new operation chunk back to its preceding context.
for span in reasoning_trace:
h = model.hidden_states(span, layer=L) # per-token vectors
x = pool_middle_token(h) # span representation
z = LDA.transform(PCA.transform(normalize(x)))
for op in OPERATIONS:
score[op] = dot(z, direction[op]) # alignment with op vector
predicted_op = argmax(score)
A common assumption is that chain-of-thought is essentially surface behavior: the model emits words that look like planning or recall, and any “structure” is in the text, not the network. This paper argues the opposite. The functional role of a reasoning span, decomposition vs recall vs arithmetic, is a real geometric object in mid-layer activations, and the same literal token is placed in different regions depending on the operation of its surrounding chunk. The load-bearing evidence is not the headline separability number. It is the shared-token analysis plus the attention-masking intervention, which together show the signal is not lexical and not local.
•
The strongest single result is the shared-token analysis. When you take function words like “is”, “of”, “the” that appear in spans of two different operations, in early layers their representations are intermixed, but by middle layers they separate along the corresponding operation directions. Same token, different geometry, driven by surrounding context.
•
Attention-masking the 30 tokens immediately before a new operation chunk reduces the target operation-alignment score across operations. Masking a random earlier block of similar length has a weaker effect. So the operation identity at chunk onset is causally built from preceding reasoning context, not from the chunk’s own tokens.
•
Operations are separable in held-out representations, with peak AUROC in middle layers across all three models. Mean-pooled hidden-state probes reach macro AUROC/AUPRC of 0.937/0.742 on Qwen3-8B, beating the best text-only classifier by 0.041 to 0.097 AUROC and 0.084 to 0.193 AUPRC across the three models.
•
Intra-span variance of the alignment score is high in early layers and drops in the middle: early separability comes from a few cue tokens, middle-layer separability is spread across the whole span.
•
Probes trained only on correct traces still work on incorrect ones. On factual-error spans, macro AUROC is 0.955 (mean-pooled) vs 0.971 on matched non-error spans from the same failed traces. Deduction and Arithmetic Computation degrade the most; Recall barely changes.
•
Generalizes: Qwen3-8B probes transfer without retraining to GPQA-Diamond (0.938/0.764) and MATH-500 (0.948/0.799), and the same pipeline on Llama-3-8B gives 0.958/0.840.
Reach for this when you are building trace-level observability for a reasoning agent. Instead of asking a judge model “is step 14 a recall or a deduction,” you can fit a small linear probe on mid-layer activations and get an operation label per span for effectively the cost of one extra matmul. That could feed a router (“if the model is stuck in Arithmetic computation for too long, switch to a tool call”), a monitor (“unusual jump from Extraction straight to Final answer”), or a labeling pipeline for RL on reasoning traces.
Code and materials are at GitHub. The taxonomy, span-annotation prompts, and probing pipeline are the reusable pieces. The paper itself is diagnostic, not a training method, so there is no released model or dataset of hidden states, you regenerate traces from the open Qwen, Gemma, or Llama checkpoints and refit.
The reasoning categories you see in a chain-of-thought are not just narration; they are geometry the model already computes, and you can read them off mid-layer activations cheaper than you can ask a judge model.
•
Span labels come from GPT-5 with only 84 human-validated examples, and human-GPT agreement is 76%. The probes are learning “what GPT-5 calls decomposition,” which is close to but not identical to a ground-truth cognitive category.
•
Everything is on math and theorem QA with three model families. Whether the same geometry exists for code agents, tool-use traces, or commonsense reasoning is untested.
•
The work is diagnostic only. No experiment shows that steering along an operation vector, or gating on a probe, actually improves an agent’s behavior. That leap is left to the reader.