Methods that improve how language models think through problems — chain-of-thought, search, verification, and the reasoning-trained model families that have emerged since o1-style training.
How LLM reasoning is trained, represented, monitored, and servedReasoning systems face different bottlenecks at different stages. During post-training, a final correct-or-wrong reward gives little guidance about which intermediate decisions helped, while long demonstrations may contain redundant detours. During inference, long chains of thought expand the key-value (KV) cache—the stored attention state for prior tokens—until memory limits constrain batch size or generation length.Visible reasoning is also an imperfect window into model computation. Tool-using agents add another problem: preserving objectives, verified evidence, and valid execution prefixes as trajectories grow or fail. This guide separates these mechanisms so readers can choose work on training signals, inference efficiency, reasoning representations, monitoring, or runtime recovery without treating them as one technique.
BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model InferenceStart here for a detailed account of why recent-query eviction misses distant revisits, how beacon queries represent global query clusters, and what memory, throughput, and accuracy trade-offs were measured.Tested on four open-weight reasoning models over math, science, and coding benchmarks with generations capped at 32K; short chat and retrieval-heavy workloads were not evaluated.
What are you trying to solve?
Serve long reasoning without losing distant contextEvery generated token adds keys and values that later tokens may attend to. Simple eviction saves memory, but the central design question is what must remain. BeaconKV targets rare jumps back to old plans or constraints; Random Attention argues that preserving the prompt plus redundant samples of the generated trace can be enough. Test them under matched cache budgets rather than treating either mechanism as universally best.
BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model InferenceStart here for a detailed account of why recent-query eviction misses distant revisits, how beacon queries represent global query clusters, and what memory, throughput, and accuracy trade-offs were measured.Tested on four open-weight reasoning models over math, science, and coding benchmarks with generations capped at 32K; short chat and retrieval-heavy workloads were not evaluated.
Random Attention: Rethinking KV Cache Eviction for Efficient ReasoningRead as a strong score-free baseline: it explains why protecting the prompt and sampling independently by attention head can exploit redundancy while avoiding token-scoring overhead.Results cover four models, six reasoning tasks, and vLLM serving; redundancy in other inference systems or workloads was not tested.
Choose what supervision a reasoning trajectory should carryOutcome rewards identify successful rollouts but not the decisions that changed the result. Cliff marks the first invalid step; RISE extrapolates between RL-trained checkpoints to create dense token targets. A separate trace study asks whether every demonstrated intermediate token is useful at all, reporting that truncated trajectories and endpoints can retain much of the post-training benefit. These are different choices about credit, teachers, and training data.
Cliff: Learning Process Rewards from the First MistakeUseful for understanding first-error reward shaping: a teacher marks the boundary between a valid prefix and an invalid suffix, which is converted into token-level advantages.The abstract reports 12 scenarios and aggregate gains over OPD and GRPO, but omits details needed to reproduce teacher judgments and training settings.
RISE: Recursive Improvement via Self-Extrapolating Policy DistillationShows an alternative route to dense supervision that extrapolates changes between RLVR checkpoints, avoiding an external teacher and privileged conditioning.Reported across math, STEM, code, and multi-turn agent tasks; implementation choices for parameter- or logit-space extrapolation are not specified in the abstract.
Revisiting Complete Reasoning Traces for Post-TrainingChallenges the assumption that post-training needs complete traces, connecting truncation and endpoint-only supervision to SFT, reinforcement learning, and on-policy distillation.The abstract reports controlled token removal and attention analyses but does not identify which tasks, models, or truncation levels should transfer to a new training pipeline.
Should intermediate credit come from a teacher model or from the learner’s own updates?Use Cliff’s framing when an off-the-shelf teacher can reliably identify the first invalid step. Consider RISE when successive RLVR checkpoints are available and avoiding an external teacher or privileged conditioning matters. Cliff localizes a boundary within each rollout; RISE converts outcome-driven checkpoint movement into dense targets.Cliff: Learning Process Rewards from the First MistakeRISE: Recursive Improvement via Self-Extrapolating Policy Distillation
Why stop evaluating a rollout after its first mistake?Cliff argues that later steps are conditioned on an invalid prefix, so judging each one adds limited information. Splitting at the first error produces positive token advantages before the boundary and negative feedback after it. This is a reward-shaping rule for RLVR, not a claim that every later step is intrinsically wrong in isolation.Cliff: Learning Process Rewards from the First Mistake
Does post-training need every intermediate reasoning token?The trace study reports limited benefit from full trajectories, effectiveness under heavy truncation, and gains from endpoint-based training in SFT, reinforcement learning, and on-policy distillation. Read it when storage, generation cost, or noisy detours make full traces unattractive; then test truncation against your own reasoning distribution rather than assuming all intermediate tokens are redundant.Revisiting Complete Reasoning Traces for Post-Training
Separate internal structure from visible explanationsA chain of thought can correlate with structured internal representations without exposing every consequential computation. Mechanistic analysis finds operation-related geometry in hidden states; faithfulness evaluations show that models can use filler tokens or preference cues without clearly verbalizing their influence. These readings address different evidence layers: representation structure, behavioral counterexamples, and monitor performance.
Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMsConnects textual reasoning operations with hidden-state geometry through held-out separability tests, confound checks, token comparisons, and attention-masking interventions.The findings show correspondence with expressed operations, not a decoder for all latent reasoning or proof that these representations cause task success.
Not All LLM Reasoning is Visible in the Chain-of-ThoughtProvides a direct behavioral counterexample to complete chain-of-thought visibility: semantically irrelevant filler tokens can improve performance or support an unspoken objective.Evidence covers 13 frontier models on three synthetic tasks; effects varied by token and model, and trained benefits did not persist at test time for Qwen3-235B.
Chain-of-Thought Faithfulness of Reasoning Models Varies with Where and How Preference Cues Are DeliveredUseful for evaluating monitoring under agent-like information channels because it varies whether preference cues appear in user text or tool returns and whether they are explicit or implicit.FACE-Eval uses 5,100 samples and 15 open-weight models in a single-call, prefilled-tool setting, rather than live multi-step tool interaction.
How can hidden-state analyses distinguish reasoning operations from wording or position?The mechanistic study tests operation separability on held-out representations, checks lexical and positional confounds, compares identical tokens across surrounding operations, and masks preceding attention context. It reports the strongest separability in middle layers and more distributed alignment across spans, supporting contextual operation structure rather than a simple keyword code.Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs
When is transcript monitoring especially likely to miss a preference cue?FACE-Eval reports weaker verbalization when cues were returned by tools rather than placed in user messages, and when cues had to be inferred from raw artifacts rather than explicit summaries. In its 15-model evaluation, higher unverbalized preference adoption was associated with lower detection by two transcript monitors.Chain-of-Thought Faithfulness of Reasoning Models Varies with Where and How Preference Cues Are Delivered
Preserve useful state as agents act, fail, and reviseLong-horizon agents need explicit state when accumulated history stops being enough. RePoT resumes execution from a deterministically verified prefix. AREX compresses verified evidence and unresolved constraints between research-and-audit cycles. Procedural Graphs instead externalize reusable action structure, localize the current procedure, and revise the graph by contrasting successful and failed trajectories.
REPOT: Recoverable Program-of-Thought via Checkpoint RepairOffers a concrete runtime-recovery pattern: replay an action program, stop at the first invalid transition, and spend one additional model call continuing from the verified checkpoint.Evaluated on PuzzleZoo-775, PlanBench Blocksworld, and Derail-550; repair versus retry varies by model, and Adaptive RePoT is preliminary.
AREX: Towards a Recursively Self-Improving Agent for Deep ResearchExtends checkpointed refinement to deep research by coupling evidence gathering with constraint-wise audits, targeted follow-up, and compact state updates over long horizons.Reported for trained 4B and 122B-A10B agents on research, reasoning, and tool-use benchmarks; the abstract does not isolate each loop component’s contribution.
Procedural Graphs: Self-Evolving Execution Structures for LLM AgentsOffers an explicit alternative to history-only action selection: localize the current procedure in a graph, generate situational guidance, and revise procedural structure from outcomes.The abstract reports gains across multiple datasets, task types, and LLMs but does not name the tool APIs, graph sizes, or deployment costs.
When should an agent repair a failed plan instead of starting over?Repair is attractive when execution can deterministically identify a valid prefix and first invalid transition. RePoT shows that checkpoint information can be more useful than error-only feedback, but its advantage over a matched retry varies by model and benchmark. Treat prefix length and model capability as routing signals rather than assuming repair always wins.REPOT: Recoverable Program-of-Thought via Checkpoint Repair
How can a research agent refine a multi-constraint answer over a long interaction?AREX separates discovery from constraint-wise verification. An inner loop gathers evidence and drafts an answer; an outer loop audits constraints and directs follow-up searches. A learned context-update tool compresses history into verified evidence and open constraints so later iterations work from an improvement state rather than the full transcript.AREX: Towards a Recursively Self-Improving Agent for Deep Research
How can a tool-using agent keep track of what to do next?Procedural Graphs represent what-to-do knowledge as linked procedures. At each step, the system locates an active node and turns the nearby subgraph into guidance without dictating the action. A refiner proposes graph edits from successful and failed trajectories, committing changes only when held-out validation is preserved or improved.Procedural Graphs: Self-Evolving Execution Structures for LLM Agents