DART-SD trains a tool-calling agent by finding the exact step where its own failed rollout first leaves the region of teacher-verified recoverable states, then applying Supervised Fine-Tuning loss only to the recovery suffix after that breakpoint, leaving already-correct prefix tokens ungradient-updated.
You’ve shipped an agent that calls five APIs to answer a customer question. On a failed run, steps 1 and 2 were fine, step 3 called a redundant tool, steps 4-5 went off the rails. Today you either full-trajectory Supervised Fine-Tuning on a teacher’s clean 5-step trace (which punishes the model’s valid step 1) or run Group Relative Policy Optimization (GRPO) with a single terminal reward (which smears blame uniformly across all five steps). Both destroy the parts the model already got right. This is the global forcing problem the paper attacks, and it names Group Relative Policy Optimization (GRPO)-style RL and standard Supervised Fine-Tuning as the dominant baselines it wants to replace.
The key structural observation: many tool tasks have sub-goals you can solve in any order, so the space of valid solutions forms a lattice with many paths converging on the same information state. Flattening this into one linear trajectory (what Supervised Fine-Tuning and turn-level RL both do) is what the paper calls topological collapse.
The contribution is a way to align a student’s failed rollout against a graph of teacher rollouts at the level of what information has been acquired, not which specific calls were made.
1.
Build the state graph. For each task, collect successful and failed teacher rollouts. Every tool response is normalized to an information atom: two different tools returning the same fact map to the same atom; error/empty responses map to nothing. A node is the cumulative set of atoms plus a count of “useless” operations since the last informative call. This is the Interaction-State Transition Graph. Order-independent paths naturally reconverge at the same node, producing the diamond shape.
2.
Define a recoverable region. Nodes on successful teacher rollouts that sit within a budget of steps from a success terminal form R+, the success-reachable region.
3.
Find the breakpoint. Replay the failed student rollout in the same state space. Project each student state onto R+ (a main-node student state matches any teacher main node whose information set is a subset of the student’s). The first step where the student state stops projecting is the Critical Topological Breakpoint.
4.
Generate a recovery, supervise only that. Take the student’s prefix up to the Critical Topological Breakpoint, hand it plus sampled teacher traces to a generator to synthesize a continuation, then run Supervised Fine-Tuning with a token mask that zeroes out everything before the Critical Topological Breakpoint and the final answer.
5.
Iterate. Repeat over rounds; as the student improves, its Critical Topological Breakpoint moves later, so supervision naturally shifts to harder recovery behaviors.
istg = build_graph(teacher_success + teacher_fail)
R_plus = reachable_within_budget(istg, teacher_success)
for round in range(5):
for task, rollout in student_failures():
states = replay_in_state_space(rollout)
ctb = first_t_where(states, lambda s: not projects_to(s, R_plus))
recovery = aug_gen(task, rollout[:ctb], sample(istg))
train_seq = rollout[:ctb] + recovery
loss_mask = mask_only(recovery.assistant_tokens_pre_answer)
sft_step(train_seq, loss_mask)
The prevailing fix for a failed agent trajectory is to either imitate a teacher’s full clean trace (Supervised Fine-Tuning) or apply an outcome reward across all steps (Group Relative Policy Optimization (GRPO)). Both overwrite the parts of the student’s attempt that were already valid. This paper argues the opposite. The unit of correction should be the single transition where the student first leaves the region of states from which the teacher demonstrates recovery. Everything before that transition is protected from gradient updates entirely. The evidence that carries this claim is not the headline benchmark lift, but the tracked Critical Topological Breakpoint position across training rounds and the shrinking of successful trajectory length below the golden reference.
The load-bearing evidence is the Critical Topological Breakpoint-position trajectory across iterations. On failed training rollouts, the average Critical Topological Breakpoint position advances from 0.348 at iteration 1 to 1.452 at iteration 5, meaning the model executes a progressively longer valid prefix before departing from recoverable behavior. This is the mechanism the paper claims: localized supervision extends the student’s capability boundary rather than resetting it.
•
On the in-domain FTRL benchmark benchmark plus four out-of-domain suites (Berkeley Function Calling Leaderboard, ToolHop, \u03c4-bench, RoTBench), DART-SD wins the average on both Qwen3-4B (39.17 vs 37.62 for standard SFT) and Qwen3-8B (45.58 vs 41.64).
•
Distilled 8B student beats its own teachers (a mix of Qwen3.6-27B and GLM-5.2) on FTRL benchmark, ToolHop, and \u03c4-bench.
•
Successful trajectories shorten across iterations from 4.23 to 3.55 tool calls, ending up shorter than the golden reference (4.02), so the gains are not from calling more tools.
•
Ablation: adding Critical Topological Breakpoint localization on top of plain self-distillation lifts Solve-F1 from 38.10 to 39.51; adding progressive iteration takes it to 43.93; adding the full Interaction-State Transition Graph projection (vs an LLM-judge picking breakpoints) reaches 45.66.
•
General capability holds: on IFEval, AIME24/25, MMLU, DART-SD averages 49.89 vs 44.18 for plain SFT, so tool-specific training didn’t crater reasoning.
Reach for this when you’re training a smaller open-source model to be a tool-calling agent and you already have a stronger model producing rollouts you can grade for success. The pattern is: instead of full-trajectory SFT on the teacher’s clean traces, replay the student’s failed attempts, find the earliest state that no successful teacher path can reach from, and only fine-tune on a synthesized continuation after that point. The engineering cost is the state-abstraction layer, deciding when two tool responses carry the same fact, which the paper implements as a two-stage (rule-based canonicalization then LLM-judged) atom assignment.
The paper doesn’t mention a code release. Training data is the FTRL benchmark set (~2,215 tasks with verifiable feedback); evaluation uses public benchmarks (Berkeley Function Calling Leaderboard, ToolHop, \u03c4-bench, RoTBench). Models are Qwen3-4B and Qwen3-8B students with Qwen3.6-27B and GLM-5.2 teachers. Five iterations, one epoch each, learning rate 5e-7, eight rollouts per task at temperature 0.7. If you want to replicate, you’ll rebuild the Interaction-State Transition Graph pipeline yourself.
Don’t grade the whole trajectory. Find where the student first left the region a teacher could recover from, and grade only what comes after. This assumes you can define “same information state” across two different tool sequences, which is the real engineering cost. If you can’t build that abstraction cleanly, the whole approach collapses back to trajectory matching.
•
The information-atom abstraction is task-specific and partly built by an LLM judging tool responses jointly. On tool ecosystems where responses are unstructured or don’t decompose into discrete facts, that abstraction gets fuzzy and the Critical Topological Breakpoint projection loses its meaning.
•
All experiments run on Qwen3-4B and Qwen3-8B with strong teachers of the same family. Whether the self-distillation loop compounds usefully on much weaker or much stronger students, or across model families, isn’t tested.
•
The success-reachable region is defined from the teacher’s rollouts within a step budget. If the teacher only shows a narrow slice of the valid solution lattice, the student’s genuinely novel-but-correct exploration will look non-projectable and get overwritten as if it had failed.