LongHorizon-Harness runs long agent tasks as a Manage-Execute-Audit loop: a manager holds task state outside the executor’s context, a fresh-context executor does one subtask, and a read-only auditor verifies the environment before state updates. On WeaveBench with Qwen 3.7-Plus, this lifts pass rate from 51.8% to 80.7% without changing the model.
You’ve shipped a coding or computer-use agent that runs a 30-step workflow. Somewhere around step 18 the agent misclicks a dialog, then spends the next 400 steps convinced it’s making progress because its own log says so. When you inspect the trajectory later, the mistake is buried under thousands of tokens of self-narration, and the agent’s own “I completed step 18” note has already been treated as fact by every downstream step.
The dominant pattern in current harnesses (Claude Code, Codex CLI, OpenClaw) is one growing context that plans, executes, and judges itself. That conflates three jobs. This paper separates them.
The core move: task state lives outside the model’s context, as an explicit structured record of requirements, artifacts, and facts. Each round does one subtask, then throws the executor’s trajectory away. Only audited facts survive to the next round.
Three roles, each a separate model call with a fresh context:
•
Manager: reads the current task state plus prior audit reports, picks one unresolved requirement, writes a subtask contract (goal, acceptance criteria, boundary constraints). It never touches the environment.
•
Executor: gets only that contract and runs a real agent backend (Claude Code, Codex CLI, etc.) in a bounded episode. It’s the only role allowed to change the environment. Its raw reasoning is discarded when the episode ends.
•
Auditor: gets the contract and the executor’s summary, then inspects the environment through read-only tools (screenshots, ls, file parsers). It decides complete | incomplete | blocked based on what it can see, not what the executor claimed.
Only the auditor’s report crosses round boundaries. Executor claims are never trusted as state updates.
state = init_state_from_task(T)
audit_history = []
for i in range(N_MAX):
decision, contract = manager(T, state, audit_history)
if decision in ("done", "blocked", "ask"): break
exec_report, new_env = executor(contract, env) # fresh context
audit = auditor(T, state, contract, exec_report, new_env) # read-only
audit_history.append(audit)
state = manager_update(state, audit) # only audited facts persist
A thin AgentAdapter lets you plug any existing agent framework in as executor without rewriting its internal loop. The harness only controls the context supplied, the tools exposed, and the budget.
The prevailing pattern is one long agent session that plans, acts, and judges its own progress inside a growing context. This paper shows the opposite. Take the task state out of the model’s context entirely, admit only environment-verified facts into it, and run each subtask in a fresh executor that gets discarded afterward. The load-bearing evidence isn’t the headline pass rate. It’s that the same model, same executor backend, and same benchmark improves by ~29 points when the only thing that changes is who owns the state.
The cleanest isolation is the matched WeaveBench comparison: Qwen 3.7-Plus with Claude Code as executor, everything held constant except the surrounding harness. Pass rate goes from 51.8% to 80.7% and mean score from 0.702 to 0.835. Every one of the eight benchmark domains improves. Because the executor backend is unchanged, the delta is attributable to the state-management layer alone.
•
OSWorld 2.0: Qwen 3.7-Plus binary completion 2.8% → 8.3% (roughly 3×), partial score 21.5% → 35.2%. On a 34-task Opus 4.7 subset, 20.6% → 35.3%.
•
Terminal-Bench 2.1: Qwen 3.7-Plus 69.7% → 77.2%, and 83.1% with GPT-5.6 Luna via Codex. This matters because Terminal-Bench is pure CLI. No GUI routing, no visual perception. So the gain isn’t a computer-use trick.
•
Cost isn’t uniform: on Qwen, average output tokens on OSWorld go from 28.9K to 104K per task, roughly 3.6× overhead. On Claude Opus for the same Games tasks, tokens drop from 16.5M to 11.1M. Stronger models close contracts in fewer audit-replan rounds.
•
Failure floor lifts more than the ceiling: on WeaveBench Games, every Qwen task scoring ≤0.04 with the baseline recovers to 0.30–0.92. Qwen with the harness (0.733 mean) beats Opus with the baseline (0.680).
•
Where it doesn’t help: analytical Terminal-Bench categories (mathematics, video-processing, some data-science) show regressions. Independent audit catches wrong answers but can’t supply a capability the model lacks.
Reach for this when you’re shipping a multi-hour agent workflow where individual steps are cheap but the whole trajectory is fragile. A code-migration agent, an ops runbook executor, a browser-based data-collection pipeline. Today your agent probably keeps its plan and its execution log in the same conversation and asks itself “did that work?” between steps. The pattern here: hold the checklist in a database row, spawn one bounded agent invocation per checklist item, then run a separate read-only verifier against the environment (git diff, pytest, screenshot compare, file hash) before you tick the item off. Never let the executor mark its own row done.
The paper doesn’t release code or the AgentAdapter in the text provided. The framework is described in enough detail to reimplement, but if a public repo exists, this version of the paper doesn’t link it. Benchmarks used (WeaveBench, OSWorld 2.0, Terminal-Bench 2.1) are all pre-existing releases with their own official infrastructure.
Executor claims are hypotheses; only environment inspection produces state. The reason long agent runs go off the rails isn’t context length. It’s that the same model writes the plan, does the work, and grades the work, and the grading gets recorded as ground truth. Split those three jobs across three fresh calls and the trajectory stops eating its own tail.
•
The gain requires a real verifier. If your task’s success condition can only be judged by the same model that did the work (subjective quality, hidden benchmarks with fuzzy semantics), the auditor can confidently certify a wrong answer. The Terminal-Bench regressions on video-processing and MTEB tasks show this directly.
•
Token cost is model-dependent, not bounded. Qwen spent 3.6× more output tokens on OSWorld; Opus spent less. On a weaker backbone the harness pays for reliability with repeated audit-replan cycles, which can dominate your inference bill.
•
Independent auditing catches errors; it doesn’t create capability. Tasks bottlenecked by a single hard step (visual precision, algorithm design, mathematical reasoning) barely move. This is a reliability layer for models that could solve each step in isolation, not a substitute for model quality.