Stateless Language Agents (SLA) sustain long research runs by moving all persistent state out of agent conversations and into a harness that rebuilds each agent’s context per invocation, reaching a strong baseline’s final kernel performance with >84% fewer tokens.
You want to point a coding agent at an open-ended optimization problem (make this kernel faster, port this library to another language) and let it run for a day. Current autoresearch frameworks like CORAL and SwarmResearch do this by spawning several Claude Code or Codex agent harness sessions that each keep their own long-running conversation, share notes, and pick their own next experiments.
The authors show this breaks over long horizons in two concrete ways. In one CORAL run on a GPU kernel task, the last real improvement happens at 231M tokens, but the system keeps burning tokens up to 1B. 98% of the final 1,500 agent sessions make no tool calls at all because each agent, resuming its own conversation, keeps re-reaching the conclusion that it is done. In another trace, two agents independently implement the exact same libexpat feature from the same parent within 13 minutes, even after one posted a note claiming the direction. Growing conversation histories also cause Context rot, where agents degrade or stop early.
The standard benchmarks hide this. On the AlphaEvolve-popularized 26-circle packing task, four methods converge to within 2.3e-10 of each other by 5M tokens, so they are really measuring who hits the ceiling first.
The principle is “stateful search with stateless agents.” Nothing an agent accumulates (its conversation, scratch files, planning notes) survives its invocation. The harness owns the research state: the pool of candidate solutions, their measured scores, and recent outcomes grouped by search direction. For every call, the harness builds a fresh, role-specific context from that state.
Two roles. An Advisor runs once per epoch. It sees a harness-written summary of evidence across all search directions plus the current global best, and it emits concrete experiment assignments. A pool of parallel Workers each sees only its assignment, its own retained candidate, and local feedback (last trial’s score and correctness diagnostics). Workers cannot see each other’s workspaces or the full history. Because the Advisor is stateless too, it reconsiders assignments each epoch from current evidence rather than from its prior reasoning.
The harness keeps a per-Worker local candidate even when it trails the global best, so a promising-but-behind direction has time to mature. When the Advisor redirects a Worker, that Worker restarts from the global best. Statelessness is enforced, not requested: Workers run as unprivileged processes under Linux Landlock, saved sessions are cleared between trials, and evaluators run outside Worker environments.
state = load_checkpoint() # candidates, scores, history
while tokens_used < budget:
ctx = harness.build_advisor_context(state)
assignments = Advisor(ctx) # fresh session
results = parallel_map(workers, assignments,
lambda w, a: Worker(
harness.build_worker_context(state, w, a)))
for r in results:
score = evaluator.run(r.candidate)
state.record(r, score)
Evaluated against EvoX, CORAL, and SwarmResearch on four FrontierSWE porting tasks, Anthropic’s VLIW SIMD kernel task, three GPU kernel problems from SOL-ExecBench, and a cryptography task from FrontierCS, at budgets up to 1B cumulative tokens.
•
SLA wins every task at full budget. On Anthropic kernel optimization with Codex agent harness, all three SLA runs beat all nine baseline runs; the worst SLA run (1122 cycles) still beats the best baseline run (1191 cycles). SLA reaches the strongest baseline’s final cycle count using 93.1% fewer tokens with Codex, 84.4% fewer with Claude Code.
•
Horizon flips rankings. On all four FrontierSWE tasks, SLA trails the strongest baseline at 25% of the token budget (175M), then leads on all four at 100% (700M). A short-budget evaluation would have called SLA worse.
•
Each design choice matters, and effects compound. From shared checkpoints, removing Advisor context reconstruction, Worker isolation, or Advisor assignments each cut mean progress. Removing Worker isolation costs 5-11 kernel cycles over a 100M-token continuation but 224 cycles over a full 1B-token run, more than an order of magnitude larger.
•
Coordination is cheap. The Advisor uses 0.24-0.51% of tokens and 1.2-2.3% of model cost. SwarmResearch’s long-running Shepherd uses 8.4-10.7% of tokens and 6.1-7.4% of cost; its average call reads about 99K input tokens because its conversation keeps growing.
•
Worker count is task-dependent. Going from 1 to 15 Workers cuts wall-clock time by 6.6-7.3x. A pool of 15 Workers trails a pool of 1 by 337 cycles at 25% of the budget on Anthropic kernel but finishes within 4 cycles. On FrontierSWE the best width varies per task.
•
If you are building a long-running agent loop (coding, kernel tuning, algorithm search) and seeing agents declare victory early or duplicate each other’s work, the fix the paper’s evidence supports is architectural: move the durable record (candidates, scores, direction summaries) out of agent conversations and into a harness you control. Reconstruct contexts per call. This gets you cleaner ablations as a side effect, since you can checkpoint and resume from identical state.
•
Evaluate at multiple token budgets, not just one. The paper’s horizon-flip result is a direct warning: a method that looks worse at 25% of a budget may lead at 100%, and vice versa. If your current eval stops at a few million tokens, you are mostly measuring time-to-ceiling.
•
Worth testing: use a cheaper model for Workers and a stronger one for the Advisor. At a matched $100 budget, GPT-5.4-mini Workers beat GPT-5.5 Workers on Git-to-Zig but not on Anthropic kernel. The separation of coordination from implementation is what makes this choice available.
•
Reference implementation is at GitHub.
Most configurations are a single run; only Anthropic kernel with Codex and the checkpoint ablations are replicated three times, because a single long run costs hundreds to thousands of dollars. Budgets are matched on model tokens only. The cost of running and scoring experiments (compilation, GPU time) is not counted, which matters if your evaluator is the expensive part. All tasks have fast, executable evaluators, so the approach is untested on goals that are vague or slow to score (open-ended research writing, human-judged outputs). The ablations vary what each agent sees but never let an agent keep its conversation, so the paper does not directly isolate statelessness from context reconstruction. The FrontierSWE subset is four tasks chosen to not need GPUs, so coverage is narrow.