How can models assign credit to individual steps… | r*cap

Home

Search

Library

Research questionHow can models assign credit to individual steps from trajectory feedback?Feedback on a completed task often leaves unclear which steps helped. Some systems have verified outcomes to learn from; others must work from rubric judgments that estimate quality.

Code Generation & Program Synthesis

Evaluation & Benchmarks

LLM Pretraining & Post-training

Machine Learning

Multi-agent Systems

Natural Language Processing

Reinforcement Learning

Latest papersRecent research connected to this question, newest first.RISE: Recursive Improvement via Self-Extrapolating Policy DistillationApplies to language-model post-training that combines reinforcement learning from verifiable rewards with on-policy distillation, where successive model checkpoints and sparse outcome rewards are available. The reported evidence covers mathematical reasoning, multi-domain STEM, code generation, and multi-turn agentic tasks.research paper · Sep 4, 2026 DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent SystemsThe source concerns failure attribution in LLM-based multi-agent systems from natural-language system traces, defining the decisive error as the earliest action whose correction can reverse a failure. Evidence comes from the Who&When benchmark, evaluated across six LLMs; the reported results are benchmark-based and do not establish performance across other systems or settings.research paper · Sep 4, 2026 ConsensusBench: Benchmark of Consensus Nodes for LLM Reasoning via Outcome Reward DensifyingThe paper builds process signals by clustering semantically equivalent intermediate conclusions across correct rollouts and integrates them into GRPO-style reinforcement learning for mathematical reasoning. It evaluates the approach on several mathematical reasoning datasets and a process-level benchmark; usefulness for other domains, reward constructions, or tasks is not established.research paper · Sep 4, 2026 DRACO: Fine-Grained Credit Assignment with Dynamic Rubrics for Long-Horizon Agent TrainingThe paper introduces dynamically generated rubrics and a closed-form redistribution of trajectory judgments into per-step advantages for policy training, without a trained attribution module. It evaluates this approach on AppWorld and Tau-Bench, including a setting without a frontier judge, so the evidence is limited to those tasks and feedback configurations.research paper · Sep 3, 2026 Coverage, Not Targeting: A Structural Regime in Multi-Turn Agent Credit AssignmentThe evidence covers controlled shared-rollout comparisons on tau^2-bench and BFCL V3, with replication across model families on ToolACE-2-8B. It examines uniform, targeted, shuffled, reward-to-go, and matched-budget reward allocations under terminal verification; conclusions are limited to the reported environments, models, and seeds.research paper · Sep 2, 2026 DMRL: Document-Mediated Reinforcement Learning for Skill Optimization in Advertising RecommendationThe source concerns an LLM-assisted advertising recommendation system deployed on a large-scale short-video ads platform. It describes an upper-level document-editing agent and a frozen lower-level task agent evaluated through A/B testing, with reported results on advertising metrics; broader generalization beyond this setting is not established.research paper · Sep 2, 2026

Related questions

How can reasoning models keep improving on open-ended agentic tasks as human supervision and reliable rewards recede?How can on-policy reasoning models use dense self-guidance without reinforcing incorrect solutions?When should a multi-step LLM agent escalate from a cheaper to a larger model?Should reasoning LLMs combine on-policy distillation and verifiable-reward RL jointly or sequentially during post-training?