Harness Handbook auto-generates a behavior-indexed map of an agent codebase so a coding agent locates every scattered edit site before planning a change, lifting plan-quality win rate by 10–19 points while using fewer planner tokens.
You maintain an agent framework: prompt builders, tool routers, state managers, retry logic. A product manager asks for a small behavior change (“require three confirmations before submitting, not one”). The relevant code lives in four places: an init, a per-run reset, and two branches of the main loop. A coding agent with repo search and read_file will find the obvious spot and miss the reset, quietly breaking the second run.
The paper calls this behavior localization: mapping a natural-language behavior request onto every implementation site it touches. Prior work on repo maps, code search, and Retrieval-Augmented Generation over source makes files easier to inspect, but still leaves the agent to reconstruct which files jointly implement a given runtime behavior.
The core move is to build a second artifact next to the repo, organized by what the system does at runtime rather than by files. It has three levels: a system overview, per-stage component overviews, and leaf entries that each point to a real source range. Alongside it sits a state register view, listing every read and write site for each piece of cross-stage state. That register view is what surfaces the non-obvious mirror sites keyword search misses.
Construction runs in three phases. Phase I is deterministic Static program analysis: parse the repo, build a call graph, record signatures and line ranges, log unresolved calls instead of guessing. Phase II uses an LLM to assign functions (or whole files, for larger repos) to execution stages, with a critic loop that reviews each assignment. Phase III synthesizes the L1–L3 document tree and validates that every leaf locator still resolves in the current source. Anything that can’t be validated is frozen, not guessed.
At modification time, a workflow called Behavior-Guided Progressive Disclosure walks the handbook coarse-to-fine, then opens the real repo to verify candidate sites before the planner writes an edit plan. After execution, any non-empty diff triggers resynchronization that refreshes only the affected entries.
def handle_request(q, handbook, repo):
stages = pick_stages(q, handbook.L1, handbook.L2)
stages += follow_state_registers(stages, handbook.Z)
candidates = select_L3_entries(stages) + expand_call_graph(...)
evidence = [c for c in candidates if verify_in_repo(c, repo)]
plan = planner(q, evidence)
new_repo = executor(repo, plan)
if diff(repo, new_repo):
handbook = resync(handbook, diff)
return plan, new_repo, handbook
The prevailing approach to helping coding agents navigate large repos is to make the code more inspectable: better search, repo maps, long context, memory over files. This paper argues the opposite. The bottleneck isn’t inspecting code, it’s knowing which code jointly implements one runtime behavior; fix that with a separate behavior-indexed artifact and keep the repo as authoritative ground truth. The load-bearing evidence is that a weaker planner with the handbook matches stronger reference models on localization metrics, not just the raw win-rate lift.
The finding that carries the thesis is the localization comparison against reference plans from stronger models. Against Claude Opus 4.8 and GPT-5.5 references, all 24 recall/precision/F1 comparisons improve for the handbook arm, with F1 gains of +5.0 to +18.8 points at file and symbol granularity. Crucially, recall and precision move together, so the planner isn’t just returning more guesses. “Wrong” (zero overlap with the reference) falls by up to 25.9 points. A weaker planner with the map matches stronger models without one.
•
Plan-quality win rates: 38.3% vs 28.3% on Codex harness and 45.6% vs 26.7% on Terminus-2, consistent across three independent judges.
•
Token cost drops while quality rises: −12.7% on Codex, −8.6% on Terminus-2. The gain isn’t bought with a bigger context budget.
•
Gains hold across all three request types (Query, Cross-file, Search-Hostile) and all three difficulty levels, ranging +16.3 to +33.3 points by type.
Reach for this pattern when you’re shipping a mid-sized agent framework and every behavior change (“add a per-command cwd”, “mask secrets in every capture path”) requires touching four files nobody remembers are coupled. Instead of asking your coding agent to re-explore the repo each time, precompute a behavior-indexed map keyed by execution stage and shared state, and make your agent consult it before it writes a plan. The state-register view is the piece that pays off most: it turns “find every read/write of this flag” from a fragile grep into a lookup.
The authors publish a project page with prompts, algorithms, and the SKILL.md manifest used to expose the handbook to a planner. The evaluation targets two real open-source harnesses (Terminus-2 in Python, Codex harness in Rust) at very different scales, and the appendix includes full construction and BGPD prompt templates you can adapt. The paper doesn’t specify a released code repo for the handbook builder itself beyond the project link.
Index your codebase by runtime behavior, not by files, if you want an agent to change it reliably. The map is cheap to build from static analysis plus a critic loop, and it earns its keep by surfacing the scattered state-register writes and mirrored fallback paths that keyword search reliably misses. The repo stays authoritative; the handbook just tells the agent where to look first.
•
The evaluation uses LLM judges (including the planner’s own model, DeepSeek-V4-Pro, as one of three) scoring 30 requests per harness. That’s a small, judge-mediated benchmark, and while the direction is consistent across judges, absolute win-rate numbers should be read as directional.
•
Resynchronization after every diff is described but its cost and failure modes on churning repos aren’t measured. If the stage skeleton keeps invalidating, you fall back to a full rebuild, which the paper doesn’t cost out.
•
The two evaluated harnesses are 6 files and ~2,300 files respectively. Behavior on a truly sprawling monorepo, or on a harness with heavy dynamic dispatch that static analysis can’t resolve, is an open question. The paper explicitly logs unresolved calls rather than guessing, so coverage on such repos would degrade gracefully but visibly.