TerminalHorizon synthesizes long-horizon terminal-agent training tasks by running an independent agent against each candidate task and reconstructing the environment wherever the agent could skip the intended exploration, implementation, or verification work. Training on ~3K such trajectories lifts a 27B base across 12 benchmarks.
If you’re training an agent to live in a shell (write code, debug a service, run a data pipeline), you need demonstrations of what sustained work looks like: explore the repo, form a hypothesis, run something, read the error, revise, verify. Most synthetic terminal datasets instead give you lots of short, self-contained tasks. The paper reports averages well under 50 turns for competing synthesis methods, versus 52.8 mean turns (P90 of 98) for its own.
The deeper problem the authors flag: when you auto-generate a task environment from a real-world problem, the generator often solves half the problem for the agent. The scaffolding files already contain the key function, so the task reduces to flipping a constant. The agent completes it without ever exercising the behaviors you wanted to teach. Prior terminal-synthesis work (for example RST-27B with ~327K trajectories, or Terminal-Universe-27B with 32K) scaled coverage but didn’t specifically guard against this collapse.
TerminalHorizon is a two-stage pipeline that builds Terminus-2-style task packages (instruction, environment, reference solution, private verifier, execution config).
Stage 1, Challenge-Guided Synthesis. A Planner Agent picks under-covered professional domains. A Deep Research Agent reads real documentation, issues, and code to produce a task seed. A Construction Agent builds an executable task from that seed. Then the key step: an Independent Challenger, a separate agent given only the public instruction and environment, actually tries to solve it. Its trajectory is compared against the intended work via Horizon Gap Analysis: did the agent genuinely implement the function, or did the environment already contain it? Gaps trigger a Refine step that rewrites the initial state, tool outputs, or scaffolding to force the missing work back into the agent’s path. Loop until the trajectory shows the intended behaviors.
Stage 2, Progressive Horizon Expansion. Accepted tasks are extended under the same goal with new requirements that depend on earlier results (broader roles, deeper mechanisms, richer evidence, or method upgrades). The expansion is designed so later steps consume artifacts and observations produced by earlier steps, which keeps the agent revisiting and revising.
for batch in batches:
briefs = planner(accepted_tasks, feedback_log)
for brief in briefs:
seed = deep_research(brief)
task = construct(seed)
while True:
traj = independent_challenger.rollout(task.public_view)
diagnosis = review(task, traj) # horizon-gap analysis
if diagnosis.ok: break
task = refine(task, seed, traj, diagnosis)
for _ in range(expansion_depth):
task = expand(task) # new reqs depending on prior results
accepted_tasks.append(task)
The resulting corpus is TerminalHorizon-3K: 1,535 task environments and 2,852 teacher trajectories collected with DeepSeek-V4-Pro-0813 under Terminus-2, then filtered with an LLM reviewer. Standard SFT on Qwen3.5-27B produces TerminalHorizon-27B.
•
On Terminal-Bench 2.0, TerminalHorizon-27B scores 55.8%, a +14.2 point gain over the Qwen3.5-27B base, and reaches 60.67% on 2.1. It beats Terminal-Universe-27B on 2.0 by 3.0 points while using under one-tenth the trajectories.
•
Gains transfer to real-world productivity benchmarks: AutomationBench improves by +11.7 pp Pass Rate and +23.3 pp Partial score; VitaBench by +11.3 pp; tau2-bench Airline/Retail by +5.5 and +2.6 pp.
•
General reasoning is preserved and slightly improved (MATH-500 at 98.9%, AIME 2024 +3.4 pp, AIME 2025 +6.6 pp). So agent SFT didn’t degrade math.
•
Scaling from ~1K to ~3K trajectories keeps improving on TB 2.1 with no visible saturation, and the method helps three different Qwen sizes (27B, 35B-A3B, 122B-A10B), each gaining 13-15 pp.
•
Case studies on RSI-Exam (12-hour autonomous research budgets) show qualitative differences: the base model often declares victory early based on a self-chosen proxy metric, while TerminalHorizon-27B tends to probe its own metric, run leave-one-out style diagnostics, and keep the stronger solution in the final submission. On a multi-view point-cloud task, the final hidden F-score rises from 48.04 to 77.68.
The authors’ interpretation is that the gain comes from behavioral density per trajectory (an average of 8.6 “core patterns” per trace across exploration, diagnosis, construction, verification, coordination), not from more tokens. The paper doesn’t isolate challenge-guided reconstruction from progressive expansion in a clean ablation, so the two stages’ individual contributions aren’t separated.
•
If you’re building synthetic agent training data, the central lesson is operational: run an independent agent on your generated tasks and diff its trajectory against the work you intended. If the agent skipped a step, your scaffolding probably solved it in advance. This is cheaper than it sounds and catches a failure mode that pure difficulty-scaling misses.
•
For teams fine-tuning open coding agents, the data-efficiency result is worth testing: ~3K dense long-horizon trajectories beat ~30-500K short ones on the same base model and harness in this paper. Before scaling trajectory count 10x, worth checking whether your trajectories actually contain multi-stage dependencies, since the paper measured a median “feedback chain” of 25 turns.
•
If you evaluate agents on self-chosen proxy metrics during autonomous research, the point-cloud case is a cautionary data point: a model can hit a proxy of 1.0 by overfitting to the reference view while missing most of the scene. A leave-one-view-out or held-out slice as a sanity check is cheap and the paper shows it changes behavior.
•
Data and models are released on HuggingFace and a project page, so you can inspect the trajectories directly rather than rebuilding the pipeline.
•
No ablation separates challenge-guided reconstruction from progressive expansion, so you can’t tell which stage carries the gain.
•
The main teacher is DeepSeek-V4-Pro-0813 and the base is Qwen3.5-27B; the paper reports gains at other Qwen sizes but doesn’t test non-Qwen bases.
•
Benchmark numbers are averaged over 3-4 runs with substantial compute (144 H20 GPUs for SFT). The RSI case studies are one run per model per case, so treat qualitative behavior differences as suggestive rather than statistically established.
•
The 12-hour autonomous research results depend on a specific harness (Terminus-2 in an E2B sandbox) and don’t carry over automatically to other agent scaffolds.
•
Several cited prior works and some models here have 2026-dated arXiv IDs, so cross-check availability before building on specific baselines named in Table 1.