EvoSafeHarness automatically searches a per-deployment safety wrapper (natural-language policy plus executable code) around a frozen agent model, using its own failure traces plus a fresh-context adversarial reviewer, cutting average attack success from 45.6% to 10.0% at ~3 points of utility.
You’ve shipped an LLM agent that reads emails, browses tickets, or calls tools with real side effects (moving money, editing files, sending SMS). Two things can go wrong. A malicious user can just ask the agent to do something harmful. Or attacker-controlled text inside a document, page, or tool result can smuggle in instructions the agent then executes. This is Indirect prompt injection, and it crosses a trust boundary the model itself often can’t see.
The standard response is a harness: a wrapper around the agent loop that inspects prompts, tool calls, and results, and blocks the bad ones. Prior harnesses like CaMeL, DRIFT, Progent, and SafeHarness are hand-designed once and reused across every model and every application. The paper’s core observation: this transfer breaks down. A wrapper strict enough to stop a weak model overblocks a well-aligned one (Claude Sonnet 4.6 loses ~24 utility points under CaMeL for little safety gain). A filesystem-shaped defense doesn’t understand wash trading. So safety-utility is deployment-dependent, and a fixed guardrail can’t sit at the frontier everywhere.
The idea: treat harness design itself as an optimization problem, one per (model, domain) pair. A coding LLM proposes a candidate harness, runs it against the target agent on benign and attack tasks, reads what failed, and edits. The harness is a pair (P, C): a natural-language policy P injected into the system prompt, and executable C that hooks pre-tool-call, post-tool-call, and prompt-transform. The Designer can write any Python it wants inside those hooks: block calls, rewrite arguments, maintain per-trajectory state (e.g. a ledger of trades), call quarantined classifiers.
The key departure from generic pipeline optimizers like DSPy or Meta-Harness is that naive reward maximization on “minimize attack success” has three failure modes: refuse-everything harnesses score well, rules overfit to benchmark-specific filenames or paths, and buggy code silently fails open. EvoSafeHarness counters these with four coupled pieces:
•
A Designer that proposes edits, reading past candidates’ code, scores, and failure traces from an archive.
•
A Criticizer in a fresh context that asks “could an attacker rename, relocate, or rephrase to bypass this?” and rejects rules keyed to literal benchmark tokens before evaluation is spent.
•
A Cascade Test Environment with cheap-to-expensive stages (static check, smoke prefix, confidence gate using a bootstrap on partial evidence, full evaluation). Weak candidates die early.
•
An Analyzer that decomposes each failure into benign / direct-attack / indirect-attack channels so the Designer knows what kind of thing broke.
archive = distill(prior_defenses) # warm start from CaMeL, DRIFT, ...
while budget_left:
parent, edit = designer(spec, archive)
H = edit(parent) # revise P, C, or both
while (v := criticizer(H)).flags_evasion:
H = repair(H, v)
if not static_ok(H) or not smoke_ok(H): continue
s_prefix = eval_prefix(H, model, tasks, q)
if ucb_lower(s_prefix - s_no_defense) < -delta: continue
score = extend_eval(H, model, tasks)
archive.add(H, score, v)
return best(archive)
The cascade reuses trajectories across stages (nested), so admitted candidates don’t re-run earlier tasks.
The main evaluation is on DecodingTrust-Agent (DTAP), with a 5-victim × 3-domain grid (filesystem, finance, telecom). Each cell gets its own independent search; baselines are ported once and frozen. On the held-out split, EvoSafeHarness wins the score (utility minus ASR) in 14 of 15 cells and cuts average ASR from 45.6% to 10.0% at a 3.3-point utility cost. Progent reaches similar ASR (10.5%) but only by dropping utility to 56.4% versus 79.8% for the searched harness. A paired McNemar test on the pooled 1,050 attacks: 385 neutralized against 11 newly opened, p ≈ 2×10⁻⁷⁸.
On AgentDojo the searched harness hits 82.8% utility at 0.0% ASR, roughly twice CaMeL’s utility at the same zero-ASR point. The same harness, applied unchanged to the unseen AgentDyn suites (different tools, different workflows), holds 75.0% utility at 0.0% ASR, while CaMeL collapses to 0% completion and Progent to 9.7%. That transfer is the paper’s evidence that the searched harness enforces provenance and task scope, not memorized tool names.
On Agent-SafetyBench, where harms include unsafe requests and misinformation in final answers (not just tool calls), it has the lowest unsafe-behavior rate on every victim. Against a PAIR attack attacker adaptively rewriting prompts with a refinement budget of 16, mean ASR stays below 20%, still under the undefended baseline on static prompts.
Ablations (§7.5, Kimi-K2.5 fixed): removing the fresh-context Criticizer leaves the search score unchanged but drops held-out score by up to 31 points, because search-set-only optimization drifts into token-keyed rules (e.g. matching literal tool names like query_customer). Removing the security warm start costs ~10 points per domain. The nested cascade saves 43.6% of evaluation tokens and selects the same defense in all 22 replayed banks.
The qualitative finding: fixing the domain and varying the model changes how a safety relation is enforced (semantic judge vs deterministic gate, cached vs recomputed, pre- vs post-tool-call). Fixing the model and varying the domain changes which relations exist (filesystem needs command/path/dataflow; finance needs a trade ledger to catch wash trading, where a sequence of individually legal trades collectively harms).
•
If you’re building an agent product and currently rely on one hand-written guardrail across models, this paper is direct evidence that’s leaving safety-utility on the table. The actionable move: treat the wrapper as something you fit per deployment, using your own eval set of benign tasks and attempted attacks. You don’t need this specific framework to do that.
•
The fresh-context adversarial review is the transferable trick. Any optimization loop that maximizes a safety metric on a fixed test set will find token-keyed shortcuts. An independent reviewer that gets only the candidate code and asks “what’s the cheapest rename/rephrase bypass?” is cheap insurance and empirically the difference between +5 and -31 points on held-out.
•
Worth testing: separating benign, direct-attack, and indirect-attack outcomes as distinct feedback channels rather than a single scalar. The paper shows the Designer needs this signal to know whether to strengthen provenance handling or effect authorization.
•
The paper’s residual audit (§7.3) is a useful map of what harnesses can and can’t do. Harms compilable to a runtime relation (wash trading, credential exfiltration, volume floods) go to zero. Harms that require judging truthfulness of otherwise in-scope actions (targeted scams, misleading financial advice) are largely untouched, because no destination or ownership invariant certifies content veracity.
•
The code and site are released. Baseline ports (CaMeL, DRIFT, Progent, SafeHarness) come with the artifact, which is genuinely useful if you want to reproduce the frontier. •
Search is expensive at two levels. Outer optimization used ~40M fresh tokens on a strong coding model (default Claude Opus 4.8) plus 2.5B cached tokens; inner evaluation ran ~59K victim calls for ~$329 across the reported grid. That’s per (model, domain). You re-pay when either changes non-trivially.
•
The outer Designer model matters. Swapping among four frontier coders, mean ASR ranges from ~5% to ~18% at similar utility. A weaker searcher finds a looser relation.
•
Adaptive attacks were tested with a bounded refinement budget of 16 iterations on AgentCanary; ASR was climbing (up to 25% for one attacker). This isn’t a proof of robustness against a determined adaptive adversary with more compute.
•
Some harm classes have no structural signature. Client-targeted financial scams stayed at 60% residual ASR because the action itself is in scope and only the content is deceptive. If your threat model centers on content veracity, this approach alone won’t cover it.
•
Held-out utility intervals overlap with no-defense in every cell (only 30 benign tasks per cell), so the small utility cost is within sampling noise at the cell level, not proven negligible.
•
The framework assumes you can enumerate a domain specification (tools, threats, trusted context, judges). For open-ended agents where the tool surface itself is dynamic, adapting the recipe is not straightforward and the paper doesn’t address it.