AutoSFT splits SFT data research across a meta agent, an executor, four analysis subagents, and a synthesizer, so an automated loop can repair, replace, and write training data rather than only reweighting categories.
If you hand an LLM agent the full post-training stack and tell it to improve a model, recent audits of such runs (PostTrainBench) show the agent almost always picks supervised fine-tuning and then spends its time editing the training data. That matches a growing body of work saying a small, well-chosen dataset can beat a much bigger one.
The problem: existing automated systems give the agent exactly one lever, either picking rows from a fixed pool or calling a generator. A human data engineer does more: adjusts category proportions, prunes bad rows, repairs broken ones, replaces overlong responses, writes new examples for missing skills, and reorders data. The paper calls these six families the data action space. When the authors run a single Codex agent harness agent driving GPT-5.5 on an instruction-following task, over 33 rounds it only ever reweights category proportions. Its context window fills with training and evaluation logs and gets compacted (summarized and truncated) ten times, discarding research history mid-search.
The core idea is structural, not algorithmic: separate the jobs that compete for one agent’s attention, and get proposals from several angles instead of one.
•
A meta agent owns each round and decides the final recipe.
•
An executor subagent runs training, checkpoint conversion, and benchmark evaluation in its own session. It returns only file paths and a short status report, so verbose logs never touch the meta agent’s context.
•
Four analysis subagents each read the just-finished round from a different starting point: the training data, the per-example failures, the training-loss dynamics, and the history of earlier rounds. Each returns one concrete proposed data change.
•
A synthesis subagent writes brand-new examples for capabilities the pool barely covers, like “repeat the prompt verbatim with no comma.” Each generated row must pass a programmatic verifier for the target constraint, a semantic review, and an n-gram overlap check check against the evaluation sets.
Every round retrains from the original base checkpoint. Only the candidate pool and the recipe carry across rounds, so a round’s effect is attributable to its data change rather than to accumulated training. The loop looks roughly like:
for t in rounds:
C_t, F_t = executor.train_and_eval(M0, R_t) # separate session
proposals = [sub.analyze(D_t, R_t, F_t, H) for sub in analysis_subs]
new_rows = synthesis.generate(F_t, pool=D_t) # verifier-gated
D_next = D_t + new_rows
a_t = meta.decide(proposals, F_t, H)
R_next = a_t(R_t, D_next)
H.append((R_t, F_t, a_t))
The meta agent and subagents share a pool and a skill library (named procedures like “recipe authoring,” “training submission,” “failure-case analysis”). The four analysis skills share an identical “proposal” section word for word. Only the evidence source differs, which is where their disagreement comes from.
The authors add each component on top of the single-agent baseline and hold the base model, pool, token budget, trainer, and evaluation fixed inside each comparison.
•
Executor cuts context loss roughly in half. Over matched elapsed time the single-agent run compacts its context 10 times; the executor run compacts 5 times. Shell commands drop from 58% to 29% of the meta agent’s recorded events, and status-check calls (which were 23% of the base agent’s commands) move out of its session.
•
Analysis broadens the action mix and raises scores. On IFEval, the analysis configuration hits 63.33 at round 6, already above the base run’s best over 50 rounds, and reaches 67.10 at round 30, a +6.5 lead at equal rounds. With analysis, repair and resupervision actions together make up 55% of moves; the base stays entirely in the mixture (reweighting) family. Specific unlocked actions pay off: selecting data by validation results sets a new per-run best in 41.7% of its uses.
•
Synthesis adds another jump when the pool is exhausted. Under a 100M-token budget, adding synthesis lifts best IFEval from 78.15 to 81.58 (+3.4) and best IFBench from 49.02 to 51.95 (+2.9). The authors trace one capability, repeat_prompt, that sat near 40% pass rate for 8 rounds and jumped to 75.6% the round its synthesized batch entered. A different target, odd_even_syllables, got batches in 3 rounds and stayed at zero, which they read as evidence that supplying data cannot teach a constraint the model cannot yet follow.
•
The structure transfers to code. On MBPP+ with Qwen3-1.7B-Base and a Python instruction pool, the single agent shows no sustained improvement; adding analysis gets to 64.8% by round 13 (+3.7 over base); adding synthesis reaches 65.8% by round 6. Here the system has no ready-made capability taxonomy and builds one from the benchmark and the pool.
•
If you are automating data curation with an agent and only seeing it reweight categories or re-rank a pool, the diagnosis to borrow is that one agent is sharing its context between execution and analysis. Moving training-job management, log polling, and evaluation into a separate session with its own context, returning only artifact paths and a short status, is worth testing. The paper measures this change in isolation.
•
If you already have multiple proposer agents but they converge on similar recipes, the paper’s recipe is to give each one a different evidence source (data, failures, loss curves, history) rather than a different action menu. All four share the same proposal format.
•
Treat synthesis as something to add only after selection stops helping. The authors deliberately use a larger token budget for the synthesis comparison because, with a small budget, pool selection alone saturates the recipe. Expect gains concentrated on narrow, verifier-checkable constraints the pool undersupplies, not on fundamentally hard behaviors.
•
Code is available: GitHub. The paper estimates roughly 130 GPU-hours and about USD 1,900 in API calls for one 20-round synthesis-scale run, so this is not a cheap loop to reproduce.
Everything is one base model at one scale (1.7B parameters). The authors flag that capabilities this model cannot learn from more data may be learnable at larger scale, and that each configuration is run once per comparison, so variation between independent searches is not bounded. Synthesis helps aggregate scores by fixing a handful of undersupplied, programmatically verifiable constraints; it does not teach capabilities the model cannot yet follow at all. Generalization is tested on one additional domain (code) with short runs. And because the system optimizes benchmark scores, the authors specifically report off-target benchmarks and run an n-gram overlap check between synthesized data and evaluation sets, which is worth preserving if you adapt the setup.