This survey argues that agents which self-improve alone hit a ceiling, and organizes the emerging literature on co-evolution, where agents, their peers, and their environment adapt to each other, into a three-stage progression that ends with the evolution mechanism itself becoming evolvable.
You’ve shipped an agent that browses your product’s admin UI, calls internal tools, and gets graded by a fixed rubric. After a month of RL fine-tuning on logged traces, gains flatten. The reason isn’t the model. The tasks, the grader, and the tool surface never changed, so there’s nothing new for the agent to learn. Most current work on [Self-evolution](glossary://self-evolution) sits here: one agent updates itself against a static counterpart. This survey argues that sustained improvement requires the counterpart (peers, tasks, rewards, or the world model) to move too, and it maps who has tried what.
The authors formalize an agentic system as agents plus environment, where each agent has a model backbone and a [Agent harness](glossary://harness) (memory, tools, prompts, workflows). Co-evolution is defined strictly: at least two components must both persistently change AND each change must reshape the other’s future updates. Mere information exchange doesn’t count. From there they build a three-stage taxonomy of expanding freedom.
•
Stage 1, Agent–Agent. Agents adapt against other agents in a fixed environment. Adversarial (attacker vs defender, [GAN (Generative Adversarial Network)](glossary://gan)-style), collaborative ([MARL (Multi-Agent Reinforcement Learning)](glossary://marl)-style shared reward, or role-split like coder vs unit-test writer), or with the org chart itself changing (who plays which role).
•
Stage 2, Agent–Environment. The environment moves too. Three sub-axes: which tasks the agent sees (curriculum, generated tasks), how it’s graded (evolving reward models, [Reinforcement Learning from Human Feedback](glossary://rlhf)-style preference models, LLM-written reward functions), and where it acts (procedurally generated levels, or learned world models like [WebEvolver](glossary://webevolver) that simulate a website).
•
Stage 3, Meta co-evolution. The rules of evolution themselves evolve: what to update, when to trigger it, how to generate variants, how to evaluate them. The authors flag [Red Queen Gödel Machine (RQGM)](glossary://rqgm) as the clearest current example and treat systems like [PromptBreeder](glossary://promptbreeder) as precursors that lack a lower-level co-evolving substrate.
A sketch of what a Stage 2 loop actually looks like in code:
while not converged:
tasks = task_generator.propose(agent.skill_estimate)
traj = agent.rollout(tasks, env)
reward = reward_model.score(traj) # reward model also learns
agent.update(traj, reward)
task_generator.update(traj) # tasks track agent frontier
reward_model.update(traj, outcomes) # grader tracks agent too
The prevailing default when an agent plateaus is to scale the model, add more logged data, or hand-tune the harness. This survey argues the opposite. The ceiling comes from adapting only one side of the loop, so the fix is to let the counterpart, the tasks, the grader, or the world, adapt against the agent as well. The clearest evidence is the cross-paper meta-analysis in the survey’s Figure 4: within matched settings, letting the counterpart evolve consistently beats keeping it frozen, though gains taper as the loop matures.
This is a survey, so the load-bearing evidence is a meta-analysis across papers rather than one benchmark number. The authors only compare within a paper (same backbone, same eval), then look across papers.
•
Enabling counterpart evolution helps in most matched settings across Stages 1 and 2. The paired improvement is positive in a majority of comparisons the authors could match, spanning adversarial safety training, coder-tester loops, and task/reward generation.
•
Gains flatten over rounds. Normalized trajectories across papers show most of the improvement arrives early, then curves bend toward a plateau. The authors read this as motivation for Stage 3.
•
Stage 3 is mostly empty. By the authors’ strict definition (mechanism revision driven by, and then reshaping, a lower-level co-evolving system), essentially only [Red Queen Gödel Machine (RQGM)](glossary://rqgm) qualifies today. Everything else they surveyed is a Stage 1 or 2 loop, or a single-agent self-referential system that lacks the coupled substrate.
•
The map itself is the contribution. Prior surveys on multi-agent systems, agent harnesses, or self-evolution treat co-evolution as a subtheme; this one uses it as the organizing axis and draws explicit lines against [Self-play](glossary://self-play), evolutionary search, and continual learning.
Reach for this survey when you’re deciding what to make adaptive next in a shipping agent. If your coding agent has plateaued against a fixed test suite, Section 4.2 points to work that co-trains the tests with the coder ([CURE](glossary://cure)). If your safety filter is overfitting to yesterday’s jailbreaks, Section 3.1 is a menu of attacker-defender setups. If your web agent runs out of useful trajectories, Section 4.3.2 covers learned world models that can simulate cheap rollouts. The taxonomy is most useful as a checklist: for each component (agents, peers, tasks, rewards, environment, mechanism), ask whether it’s currently frozen and whether unfreezing it is what your system actually needs.
The paper itself releases no code or dataset. Its artifact is the taxonomy and the reference list, which is dense and current (many 2025 and 2026 entries), so it’s a reasonable starting point for a literature scan into any specific sub-area.
If your agent has stopped learning, the fix is usually not a bigger model. It’s finding the counterpart, the tasks, the grader, or the world, that also needs to move.
•
The cross-paper Figure 4 aggregates results across different backbones, benchmarks, and metrics; the authors normalize within each paper before averaging, so absolute magnitudes should not be read as effect sizes you’d reproduce.
•
“Persistent, mutual” is a strict definition, and the survey excludes a lot of interaction-heavy multi-agent work that practitioners would call co-evolution informally. If you disagree with the boundary, the taxonomy will feel narrow.
•
Stage 3 is largely aspirational. The safety, monitoring, and governance requirements the authors flag for meta co-evolution are stated as desiderata, not solved problems, so anyone actually running a self-modifying loop in production is ahead of the literature, not behind it.