Answer-Backtracked Credit Assignment (ABC) turns sparse trajectory-level rewards into dense per-step scores by backtracking from the verified answer to recover evidence clues, then grading each search step against them, letting a 4B agent match ~30B search agents.
You’ve shipped a research agent that does 50+ web searches to answer a hard question. It gets the answer right, but along the way it clicked three irrelevant results and briefly convinced itself of a wrong candidate. Today, if you train on outcomes, those bad steps get reinforced anyway because the trajectory succeeded. And the failed trajectories that found a critical clue on step 20 before drifting? Those useful steps get punished. The dominant baseline for training search agents (Search-R1 QA setting and its descendants) treats every step in a trajectory as uniformly good or bad based on the final answer. That’s the credit assignment problem this paper attacks.
The key trick is that once you know the correct answer, you can work backwards to figure out what a good search should have discovered. The authors call this Answer-Backtracked Clue Recovery: given a query and its verified answer, an LLM runs its own ReAct loop with real web search to reconstruct the intermediate entities, facts, and relations that a valid evidence chain must contain. For the example in the paper, a four-constraint query with answer “CeraVe” yields six clues (Ceramides as the ingredient, L’Oréal as acquirer, Eugène Schueller as founder, etc.).
Then every step in every rollout trajectory gets scored against this fixed clue set. Each step starts at a base of 1.0. Discovering or verifying a correct clue adds +0.8. Ruling out a wrong candidate adds +0.4. Wrongly dismissing a correct clue costs -0.8. Submitting a wrong final answer costs -1.0. Scores are clipped to [0, 2.0]. Crucially, a step that finds a real clue in a failed trajectory still earns positive credit, and a step that discards a correct clue in a successful trajectory still gets penalized.
These step scores feed two training stages. ABC-SFT reweights the per-turn loss by a sigmoid of the step reward, so high-scoring turns dominate the gradient. ABC-GRPO plugs the step scores in as per-step rewards inside Group Relative Policy Optimization (GRPO), replacing the usual trajectory-level advantage with a discounted step-level advantage.
for (query, gold_answer) in training_set:
clues = backtrack_clues(query, gold_answer) # LLM + web tools
for trajectory in rollout(agent, query, n=8):
for step in trajectory:
step.reward = score_step(step, query, clues) # rubric
# ABC-SFT: weight loss by sigmoid(step.reward)
# ABC-GRPO: advantage = discounted sum of step rewards
update_policy(trajectories)
The prevailing approach trains search agents on whether the final answer was right, treating the trajectory as an atomic success or failure. This paper shows the opposite. Because the ground-truth answer implicitly defines the evidence chain that should have been discovered, you can score each step against that chain independently of the final outcome, which turns the sparse binary signal into dense per-step supervision. The evidence isn’t the headline benchmark score. It’s the reward-distribution analysis showing ~10% of steps in failed trajectories are high-quality, and ~4% of steps in successful trajectories are low-quality. Trajectory-level supervision gets the sign wrong on both.
The load-bearing finding is the reward-distribution audit: in the 8.5K training trajectories, ~10% of steps in failed trajectories score above 1.0 (useful clue discovery) and ~4% of steps in successful trajectories score below 1.0 (erroneous or wasteful). Outcome-only training reinforces exactly the wrong subset of both groups. That’s what the mechanism fixes.
Secondary evidence that the mechanism translates to capability:
•
On BrowseComp, the 4B ABSeeker hits 37.3% without context management and 55.3% with it. On BrowseComp-ZH, 39.1% / 52.9%.
•
With context management, it beats reported ~30B search agents like Tongyi DeepResearch and OpenSeeker on BrowseComp and BrowseComp-ZH, despite the 7-8x parameter gap.
•
It generalizes: trained only on BrowseComp-style data, it reaches 77.0% / 46.0% on xbench (2505 / 2510) and 81.6% on GAIA-text.
•
Ablations show ABC-SFT beats standard SFT on 4 of 5 benchmarks, and ABC-GRPO beats trajectory-level GRPO on all 5.
•
During RL, ABC-GRPO produces both higher validation accuracy and longer search trajectories than trajectory-level GRPO, suggesting the step signal encourages richer exploration rather than shortcutting.
Reach for this when you’re training an agent that runs long tool-use trajectories toward a verifiable endpoint, and you have ground-truth answers but not ground-truth paths. Instead of throwing away failed rollouts and treating successes as monolithically good, run a backward LLM pass from each gold answer to reconstruct the clues, then score each step of each rollout against that clue set with a simple rubric. Use those scores as SFT loss weights, GRPO rewards, or both. The recipe applies naturally beyond web search to any domain where outcomes can be decomposed into intermediate subgoals that a verifier LLM can check.
The paper doesn’t link a code or model release in the provided text. Training uses Qwen3-4B as the backbone with 8.5K SFT trajectories drawn from OpenSeeker plus 1K RL questions, and uses DeepSeek-V4-Flash as the clue-recovery and step-scoring model. Both the clue-recovery and step-scoring prompts are given verbatim in the appendix, which is enough to reimplement the pipeline.
When your agent has a verifiable endpoint, grade the path, not just the arrival. The gold answer already encodes what a good trajectory should have found. A cheap backward LLM pass turns that into per-step supervision, and per-step supervision fixes the parts of a trajectory that outcome rewards can’t see.
•
Clue recovery and step scoring both depend on a strong external LLM (DeepSeek-V4-Flash here) running its own web-tool loop. If that scorer is wrong or biased, its errors are baked into the training signal, and the paper doesn’t audit scorer reliability against human labels.
•
The method assumes questions have a unique, verifiable answer with a well-defined evidence chain. That fits BrowseComp-style factoid retrieval; it’s less obvious how to backtrack clues for open-ended research, subjective queries, or tasks where multiple valid evidence paths exist.
•
Scale is only demonstrated at 4B parameters on ~8.5K SFT + 1K RL examples. The authors flag scaling to larger backbones as future work, so whether the gap over trajectory-level training holds at frontier scale is untested.