AAArena tests whether an AI agent can iteratively rewrite the code of a game-playing bot from match replays and ladder feedback, with the base model frozen. The strongest configuration tops 6 of 12 frozen human leaderboards from a long-running student competition.
Imagine you want an LLM to improve a program by playing it against opponents, reading the logs, and editing the source, rather than by fine-tuning weights. That’s the setup here. The authors call this Adversarial Heuristic Learning, a specialization of Heuristic Learning where the AI agent is the “learner” but what gets updated is code and supporting files, not model parameters.
The concrete testbed is the Tsinghua University Agent Competition, an annual contest where students write bots for a new adversarial game each year. The authors packaged 12 past games plus 1,920 archived human-written bots with frozen Elo ratings into a benchmark called AAArena. The question: given the rules, an SDK, replay docs, and a budget of matches, can a coding agent climb a real human ladder?
This matters because it’s a cleaner test of “self-improving agent” claims than most. The opponents are real, the ratings are fixed, and the agent has to translate rules into working strategy code, not just pick good moves in-context.
The loop is straightforward: the agent reads game rules and existing code, edits the bot, then spends from two separate budgets to get feedback. Small matches (budget: 128 opponent-slots) let the agent pick specific opponents and get back detailed replays. Full-pool evaluations (budget: 16) run the current bot against every archived human program and return Elo rating, rank, and per-opponent outcomes. The agent decides when to edit, when to probe, and when to submit for a full-pool score. The best full-pool score it ever posted is retained as the final result, so a bad edit doesn’t erase an earlier good snapshot.
Feedback flows through a persistent workspace, so the agent keeps notes across revisions (the context window gets compacted, but files persist). The base model’s weights never change; all learning lives in the code and notes.
policy, notes = starter_code, {}
while budget_remaining() and rank > 1:
opponents = agent.choose_opponents(notes, replays)
replays = run_small_matches(policy, opponents) # dense feedback
policy = agent.revise_code(policy, replays, notes)
if agent.decides_to_submit():
elo, rank, outcomes = full_pool_eval(policy)
notes = agent.update_notes(replays, outcomes)
retained = max(retained, (elo, policy))
return retained
Games are ordered by rule-description size, measured as the node count of an abstract syntax tree for the rules written in a shared pseudocode. The suite spans 1,127 to 6,368 AST nodes per game. For calibration, the authors note Go rewrites to 501 nodes and chess to 1,551, so these competition games have larger rule specifications than classical board games.
Seven model-and-harness configurations were tested. The headline: Opus5.5 running inside Claude Code earned 6 gold medals (rank-1 finishes) across the 12 games. Every other configuration used Codex agent harness as the harness, so this is a joint model-plus-harness comparison, not a pure model comparison. No configuration topped the other 6 ladders, and those 6 skew toward larger rule descriptions: 5 of 6 small-rules games had a rank-1 finish, versus 1 of 6 large-rules games.
Using GLM-5.3 for the deeper analysis (chosen because its logs are complete and comparable), runs fell into three patterns: success (reach rank 1 and stop), progress (net improvement with bad revisions mixed in), and stagnation (plateau after initial gains). The retention rule matters here. On Dorado, the agent climbed from rank 41 to rank 2 through specific code changes: fixing mine-allocation logic, concentrating hero upgrades (a single revision worth +147.7 Elo, lifting rank 14 to 3), then conditional counterattacks. Bad experiments like unconditional use of a Sacrifice ability dropped it to rank 43, but the retained snapshot survived.
Ablations (on Pacman, AntWar, Miracle) isolated what matters:
•
Opponent selection: letting the agent pick, or climbing a ladder from rank 30 upward, beats random sampling or always playing the top 4. Top-4 was worst, consistent with repeated mismatches being uninformative.
•
Dense replays beat binary win/loss across all three games. State-action-consequence traces give the agent material to diagnose failures.
•
Batch size: 2 or 4 opponents per small-match request beat 1 or 8, trading matchup breadth against the number of distinct revision cycles.
•
Tripling the budget (to 384 small, 48 full) on four plateaued games moved two ranks slightly, left two unchanged, and reached rank 1 on none. More interaction alone did not break the plateau.
•
Off-policy replays (watching matches between other players’ bots instead of its own) helped. On Miracle, where on-policy stuck at rank 26, off-policy reached rank 15.
A code audit of all 84 retained champions found decision logic was 79% rule-based control plus heuristic scoring, 13% graph search, and only 8% lookahead search, combinatorial optimization, or inference combined. Lookahead search exceeded 10% of code in only 7 of 84 champions. Within-game Spearman correlation between non-rule share and Elo averaged 0.35, which the authors flag as an association, not evidence that replacing rules causes improvement.
•
If you’re building a coding agent that iterates against an evaluator, the retention rule here is cheap and important: keep the best-scored artifact separately from the working copy so exploratory edits can’t erase progress. The Dorado trajectory shows this preserved a rank-2 policy through later rank-43 experiments.
•
The dense-feedback result argues against collapsing evaluator output to a scalar. If you can afford to pipe structured traces (state, action, outcome) back to the agent, the paper’s three test games all improved under a fixed interaction budget. Worth testing in your own loop before assuming binary pass/fail is enough.
•
The off-policy replay finding is the most portable idea: showing the agent recordings of stronger third-party solutions helped it escape a plateau that self-generated replays could not. If you have a corpus of expert trajectories unrelated to your current agent’s behavior, it may be worth feeding them in as reference material.
•
Treat the “6 gold medals” result as a ceiling under this specific protocol (frozen opponents, 128/16 budget, max reasoning effort). It’s not evidence that agents beat humans at open-ended game development; the opponents don’t adapt back, and the agent gets to probe the evaluator many times.
•
Rule-description complexity is a plausible difficulty signal. If you’re picking a task to stress-test a coding agent, large rule specifications may expose weaknesses that short ones hide.
The paper doesn’t link a public code or data release in the supplied text.
Full-pool evaluations serve as both training signal and reported score, so the headline Elo numbers measure optimization against the evaluator, not generalization to unseen opponents. Opponent ratings are frozen; the archived bots cannot adapt to counter the agent’s strategies, which inflates the realism gap versus a live competition. The Opus5.5 result confounds model and harness (Claude Code vs Codex agent harness), so you cannot cleanly attribute the 6 gold medals to the model alone. Ablations ran on only 3 of 12 games with a single model, so the opponent-selection and batch-size recommendations are indicative rather than general. And the algorithm-family analysis is an automatic classification of final code; it describes what shipped, not what caused improvement during development.