RRSI regularizes the loop where an LLM rewrites its own agent scaffolding, capping edits per round, pruning dead components, and rejecting cost-heavy or noise-driven wins, so the evolved harness generalizes instead of memorizing the Evolve set it was scored on.
Modern LLM agents are mostly Agent harness code: prompts, control flow, tool wrappers, memory, context management. A frozen backbone plus a good harness often beats a bigger model with a naive one. So people started automating harness design: run the agent on some tasks, let another LLM read the failures, propose edits to the prompts or control flow, keep whatever raises the score, repeat. This is Recursive self-improvement (RSI) at the system level (weights stay frozen, the scaffold evolves).
The problem: you only have a finite set of tasks to score against, and you reuse it every round. The proposer starts fitting that set. Recent baselines like Meta-Harness show big gains on the tasks they evolve against, and much smaller (sometimes negative) gains on held-out benchmarks. This is the Adaptive overfitting problem, but at the level of agent code instead of model weights. The authors want a harness that transfers across benchmarks, tool interfaces, and even backbone models, not one that memorizes the evolve set.
The core move: keep the edit space fully open (any prompt, tool, memory, subagent can change), but regularize how the search walks through that space. RRSI borrows analogies from classical ML regularization and applies them to the proposal and selection steps of harness evolution.
On the proposal side, three constraints. First, an annealed edit budget: early rounds may bundle up to b_max independent edits per candidate, later rounds are forced down to 1. Fewer bundled edits means you can actually tell which change caused which score movement. This is their L0-style cardinality constraint. Second, evidence-aware credit assignment: every past candidate’s diff, hypothesis, score change, and accept/reject verdict is logged, and the proposer conditions on that history so it stops re-proposing edits that already failed. Third, structured exploration: when progress stalls (no gain above the noise band \u03b4 for w rounds), reserve slots for components the search hasn’t touched yet.
On the selection side, a candidate must clear several non-negotiable gates before it can replace the current harness. A leakage critic reads the diff and rejects edits that hardcode benchmark task names, answers, or entity-specific logic. A stability floor: the candidate’s score must be at least S* - \u03b4, where \u03b4 is calibrated by re-evaluating the base harness repeatedly to measure evaluation noise. A cost-aware rule: if the score gain \u0394S exceeds noise, extra token cost \u0394C is only allowed up to \u03b2\u2080 + \u03b2\u2081\u0394S (a small base allowance plus a slope, in plain terms: bigger wins buy you more compute headroom). Finally, structural pruning: components that have been exercised but produced no positive gain within the pruning window get flagged for deletion in the next round.
for t in range(T):
b_t = anneal(b_min, b_max, t, T) # shrink edit budget
feedback = analyze(H_t, evolve_set)
prune_targets = [c for c in touched if best_gain(c) <= 0]
candidates = propose(H_t, history, budget=b_t,
explore=unused_components if stalled else None,
delete=prune_targets)
candidates = [c for c in candidates if not leakage_critic(c)]
scored = [(c, evaluate(c, evolve_set)) for c in candidates]
admissible = [c for c, (s, cost) in scored
if s >= S_star - delta
and cost_rule(s - S_t, (cost - C_t)/C_t)]
H_t = argmax_score(admissible) or H_t
Eight benchmarks across three domains: coding (Terminal-Bench, SWE-bench Verified), agentic workspace (Harvey LAB as the evolve set plus JobBench, GDPval, APEX-Agents held out), and engineering design (EngDesign as evolve, Frontier-Eng held out). Backbone is Claude Opus 4.8, frozen. Baselines are four recent harness-evolution methods, all starting from the same base harness H\u2080 with matching candidate budgets.
On the evolve splits, RRSI’s gains are modest: +6.0 on Terminal-Bench, +4.9 on EngDesign, +1.1 on Harvey LAB. Baselines like Meta-Harness score higher on the evolve split. The story flips out of distribution. On the three OOD agentic benchmarks, prior methods add roughly 0 to 1 point over H\u2080 on average; two of them (AHE, TTHE) finish below the untouched base harness. RRSI averages 43.6 vs 39.7 for H\u2080, gaining between 3.5 and 4.7 points across JobBench, GDPval, and APEX-Agents. Frontier-Eng gains 4.3 Medal points, a 24.3% relative lift. No held-out split regresses.
Ablations show both regularizer groups matter and act differently. Removing proposal-side constraints costs 0.2 on the evolve split but 1.7 OOD. Removing acceptance-side constraints raises the evolve score by 1 point, drops OOD by 2.6, and inflates token cost by half. Fully unregularized evolution posts the highest evolve score (92.8) and the lowest OOD gain, at 3.80M tokens per trial vs RRSI’s 2.42M. Cross-backbone: a harness evolved with Gemini 3.5 Flash on Terminal-Bench still helps a much weaker Gemini 3.1 Flash Lite that never saw the search (11.2 \u2192 14.6), suggesting the retained mechanisms aren’t a fit to one specific policy.
The deterministic-grader results (EngDesign, Frontier-Eng use frozen simulators, no LLM judge) rule out the concern that gains come from writing in a way judges reward.
If you’re building an agent-evolution loop that scores candidates on a fixed task set, the RRSI checklist is worth borrowing even without their full framework. Concretely:
•
Calibrate a noise band first. Run the unchanged base harness repeatedly, measure score variance, and refuse to accept any candidate whose score falls below best_so_far - \u03b4. This alone prevents the slow downhill drift the paper attributes to noise-chasing.
•
Tie cost to measured gain. A candidate that adds 30% inference cost for a within-noise score bump is almost certainly overfitting or padding. The \u0394C \u2264 \u03b2\u2080 + \u03b2\u2081\u0394S rule is one concrete way to encode this; the specific coefficients in Table 5 are tuned per domain.
•
Run a leakage critic before scoring, not after. Once a leaking candidate posts an inflated score, it biases every subsequent round of proposals. Rejecting on the diff (looking for task names, hardcoded answers, benchmark-specific branches) is cheap.
•
Log per-edit outcomes and feed them back to the proposer. The paper’s evidence-aware history is basically a rejection-sampling memory: don’t re-test what already failed. Worth testing even if you skip the rest of RRSI.
Caveats on scope: the evaluation uses one strong frozen backbone (Claude Opus 4.8) plus one Gemini cross-check. If you evolve harnesses for a much weaker or much narrower model, the gains may be smaller (the Gemini 3.1 Flash Lite result is +3.4 vs +14.1 for the search policy). The regularization hyperparameters (\u03b4, \u03b2\u2080, \u03b2\u2081, budget schedule) are set per domain from the evolve set; you’d need to recalibrate for a new task type. Code and project site are released: github.com/google-research/rrsi.
The backbone is frozen throughout; RRSI says nothing about co-training weights and harness together. The evolve sets are still finite and reused across rounds, so the method reduces adaptive overfitting rather than eliminating it. The classical-regularization analogies (L0, L1, L2) are qualitative; the algorithm does not optimize any norm-penalized objective, and the authors are explicit about this. Finally, the OOD wins are on benchmarks that share task type with the evolve set (coding harness tested on other coding suites, workspace harness on other workspace suites); the paper does not claim transfer across domains (e.g., a coding-evolved harness helping engineering-design tasks).