Training a hybrid Qwen3 reasoning model with RL with verifiable rewards in no-think mode mostly reactivates the model’s existing think-mode behavior rather than teaching new reasoning, with an 8-dimensional activation swap removing 75% of the accuracy gain.
Models like Qwen3 ship with two modes selected by chat-template tokens. In think mode the model writes a long chain-of-thought inside <think>...</think> before answering. In no-think mode the template inserts an empty <think></think> block so the model answers directly. Both modes share the same weights.
A popular recipe takes such a model, runs Group Relative Policy Optimization (GRPO) in no-think mode with a verifier that checks the final answer, and reports improved no-think accuracy. The usual interpretation is that RL taught new reasoning strategies that work without visible chain-of-thought. This paper argues that interpretation is largely wrong. If true, it changes what you conclude from no-think RL benchmarks: a higher score may just mean the model is quietly behaving like its think mode, not that it learned anything new.
The authors train Qwen3-1.7B, 4B, and 8B with GRPO on DAPO-Math-17k in no-think mode, then run three interventions to test whether gains come from “thinking leakage.”
First, they track the model during training. Accuracy rises, but so does response length, the count of reflection words (“wait,” “hmm,” “let me check”), and a judge score where the original model rates responses as more think-like. They also measure the thinking activation shift: project the training-induced change in internal activations onto the direction that separates think from no-think in the original model. For 8B this reaches 0.56, meaning the activations moved 56% of the way toward think mode.
Second, they scale the learned weight update by a factor c > 1 without further training. Larger c pushes the model further toward think mode and even makes it emit </think> tokens that nothing in no-think mode calls for, placed at the same relative position as in real think mode.
Third, they do Cross-model activation patching on an 8-dimensional think subspace estimated from the original model alone. During generation, they replace the target model’s activations inside this subspace with the source model’s values on the same tokens, leaving everything else untouched.
# At every layer l and every token position:
for l in range(1, num_layers):
h_target = forward_layer(target, tokens, l)
h_source = forward_layer(source, tokens, l)
# Q_l: 8-dim think subspace basis for layer l
delta = Q_l @ Q_l.T @ (h_source - h_target)
h_target = h_target + delta # swap only inside subspace
Fourth, they find the handful of attention heads that read the empty think block in the original model, then test whether the RL-induced change is carried by those same heads.
•
Subspace swap removes most of the gain. For Qwen3-8B at step 67, RL lifts AIME accuracy from 0.189 to 0.411 (gain G = 0.222). Patching the original model’s 8-dim think-subspace values into the RL-trained model drops accuracy back to 0.244, erasing 75% of G. The reverse direction, injecting the RL model’s subspace values into the original model, transfers 50% of G. A PCA control subspace of the same size removes only 22%; a random subspace, roughly zero.
•
Scaled updates approach but don’t exceed think mode. Scaling the step-67 update to c=3 reaches 0.72 no-think accuracy, just under the original model’s think-mode accuracy of 0.74. The scaled models also start emitting </think> in up to 51% of responses, at the same relative position think mode uses.
•
A tiny set of heads carries the shift. About 90% of attention heads place under 1% of attention on the empty think block. Knocking out the top 5% of block-attending heads moves the original model most of the way to think mode (R_KL = 72% at 8B). Patching one head, L18H19, from the RL-trained model into the original model reproduces 44% of the full activation shift at 8B.
•
The training signal itself favors think-like responses. Within a GRPO group of 5 responses to the same question, correct responses (which get positive advantage) have higher judge think-likeness than wrong ones by 1.6 to 5.9 percentage points depending on group difficulty. A single gradient step on this signal moves the model toward the think direction; shuffling advantages within a group kills the effect.
The authors are careful: a scaled extrapolation is an exploratory probe, and the activation shift has a large component orthogonal to the think direction too. But across weights, activations, and heads, the same mechanism keeps showing up.
•
If you’re evaluating a hybrid reasoning model after no-think RL, do not read a no-think accuracy bump as evidence of a new non-thinking reasoning policy. Compare against the same base model’s think-mode accuracy on the same budget. If RL stays below think mode, you may just be unlocking existing capability.
•
The paper separates two questions worth tracking independently: does RL add new capability, or does it change how existing capability is accessed? For hybrid models the second is cheap to achieve and easy to confuse with the first.
•
If you work with Qwen3 or similar hybrid models and want to replicate the diagnostic, the GitHub repo is released. The think subspace needs only the original model and some training prompts, no RL run required, so it’s a lightweight check to add before trusting a no-think RL result.
•
Worth testing on other hybrid families (Llama-Nemotron is mentioned but not studied here) and on non-math domains. The paper’s evidence is Qwen3 on AIME math only.
•
All causal evidence is on Qwen3-8B; the activation-shift observation holds across 1.7B/4B/8B, but subspace patching is reported for 8B only. The strength of the effect also grows with model size.
•
Evaluation uses 90 AIME problems with 2 samples per problem; absolute numbers are noisy at that scale and the subspace dimension was pre-specified but the scaling factors in Section 5.1 were explored on the same problems used for evaluation.
•
“Thinking leakage explains the gain” is supported by converging interventions, not by a single clean causal isolation. The RL-induced activation change has a substantial component orthogonal to the think direction (0.58 of the think-direction norm at 8B) that this story does not explain.
•
The claim is specifically about RL done in no-think mode on a model that already has a think mode. It does not say RLVR in general only reactivates pretraining; that broader question is live in the literature the paper cites.