This survey proposes a five-level hierarchy (L0-L4) for classifying agentic visual generation systems by the latest point at which a controller can causally change a generation decision, and pairs it with a matched-evaluation protocol that holds generators, tools, and budgets constant while varying only the controller’s decision-making scope.
Building a system that generates images, video, slides, or 3D scenes increasingly means wrapping a diffusion model or renderer in a controller (usually an LLM or VLM) that plans, picks tools, inspects intermediate outputs, and retries. The field calls all of this “agentic,” but the label is applied inconsistently: a paper might claim agenticity because it uses multiple roles, or reinforcement learning, or a long tool chain, none of which tell you whether the system can actually recover from a bad generation.
If you’re picking a baseline or writing an evaluation, this ambiguity bites. A “multi-agent” storyboard planner that hands one prompt to a frozen video generator is doing something categorically different from a system that inspects a rendered frame and rewrites the plan, even though both get called agents. The authors argue existing surveys enumerate mechanisms (planning, tool use, memory, RL) without answering the load-bearing question: how late in the trajectory can evidence still change what happens?
The organizing idea is maximum causal reach: classify a system by the deepest point in a generation trajectory where its controller can still change a future decision. This produces five labels:
•
L0 Fixed Support: generators, retrievers, evaluators, benchmarks. No deployed controller making generation-level decisions. This is the inclusion boundary, not a peer level.
•
L1 Conditioning Control: controller builds the input (prompt, layout, retrieved reference, storyboard) for one predetermined executor. Commits before generation happens. Example: LayoutGPT.
•
L2 Execution Control: controller picks which generation, editing, or rendering operation runs. Example: Visual ChatGPT routing among tools, or ComfyUI-Copilot assembling a node graph. Still open-loop if the result doesn’t redirect the next call.
•
L3 Outcome-Adaptive Control: controller inspects an intermediate output and uses that observation to change the next action within the current task. Example: SLD localizing a compositional failure and re-sampling that region.
•
L4 Experience-Adaptive Control: controller retains something from a completed task (a capability profile, a distilled procedure, a verified workflow, or a parameter update) and uses it on a later independent task. Example: GenEvolve distilling successful trajectories into reusable procedures.
Assignment goes top-down. Test for cross-task persistence first; if absent, test for outcome-to-action links; then operation selection; then specification construction. Ambiguous cases get the lower level. The authors are explicit that titles containing “self-reflective” or “multi-agent” don’t count as evidence.
A capability vector records which of the four abilities a system demonstrates; the primary level is the highest one:
def assign_level(system):
if system.changes_control_on_later_task(): # L4
return 4
if system.observation_changes_next_action(): # L3
return 3
if system.selects_and_invokes_operation(): # L2
return 2
if system.constructs_specification(): # L1
return 1
return 0 # L0
One subtle point: within-task memory (remembering what you tried on step 3 while doing step 7) is ordinary trajectory state and does not establish L4. RL also doesn’t determine level; the level follows the learned policy’s inference-time action space, not how it was trained.
The second contribution is a level-conditioned evaluation protocol. To claim a benefit from adding controller scope, hold the generator, tool set, sampling budget, and evaluator fixed, then compare against a counterfactual that removes exactly the new capability. For L2, that means comparing your router against a fixed, random, or oracle route over the same tools. For L3, run a paired open-loop version that gets the same budget but can’t see intermediate renders. For L4, remove, shuffle, or stale the memory store and check whether later-task performance drops.
This is a survey, so “findings” are corpus statistics rather than model scores. The authors annotated systems across seven task collections (image, video, editing, 3D, world, slide, UI) with level, mechanism, feedback, memory, and provenance fields.
•
Growth is concentrated at L3. The half-year release timeline shows sparse L1 early on, then a steep post-2025 rise dominated by outcome-adaptive systems. Within-trajectory feedback is now the modal form of “agentic” behavior.
•
L4 stays rare. Cross-task experience reuse is a small fraction of the corpus, which the authors read as evidence that persistent memory is substantially less mature than within-task correction.
•
Single-controller architectures dominate every level. Multi-role setups cluster at L1-L3; unified multimodal policies appear at every level but remain uncommon.
•
L3 is prevalent across all modalities, but its operational meaning varies: editing and UI tasks expose rendered state that’s cheap to inspect, while video, 3D, and world tasks need consistency across time or viewpoints.
The authors flag a specific evaluation risk for L3 systems: apparent improvement from “outcome-driven repair” can actually come from best-of-many sampling where the selector, not any diagnosis, is doing the work. They recommend separating the controller critic, the stopping critic, and a held-out evaluator, and reporting diagnosis precision and regression on previously-satisfied constraints, not just final quality.
If you’re building an agentic visual generation system, use the top-down assignment test on your own design before claiming a capability. If your “self-correcting” system generates 8 candidates and picks the best, and the picker doesn’t depend on inspecting a specific rendered defect, you’re at L2 with a selector, not L3. Being honest about this changes which baselines you owe a comparison against.
If you’re evaluating someone else’s system, the matched-counterfactual design is the actionable part. For a router claim, the fair baseline is a fixed or random route over the same tool set, not a weaker tool set. For a repair-loop claim, the fair baseline is an open-loop run with the same total compute budget. Without that matching, extra tool calls or extra samples can carry the gain.
If you’re deciding whether to add memory across tasks (L4), the survey’s warning is worth internalizing: evaluate on chronologically held-out tasks, and specifically test forward transfer, negative transfer, stale-experience recovery, and rollback cost. A profile that helped on early tasks can silently hurt when tool prices, endpoints, or user preferences shift. The paper doesn’t provide a benchmark for this, but it names the axes you should report.
If you’re reading the literature, the taxonomy gives you a filter for the “agentic” label. When a paper claims agenticity, ask: what’s the deepest decision the controller can change? If the answer is “which prompt goes into a frozen model,” that’s L1 regardless of how many roles the pipeline has.
The project repository (structured corpus with level, mechanism, and provenance fields per system) is mentioned as released; the paper points to it but the URL isn’t given in the excerpt.
This is a taxonomy paper, not an empirical comparison. The hierarchy is a classification claim, not evidence that higher levels produce better outputs. The authors are careful about this: L4 systems aren’t shown to outperform L3 systems, only to have broader causal reach. Output quality is a separate axis.
The corpus is what the authors chose to include, weighted toward controller-bearing systems (only four L0 records retained). The half-year growth curves reflect this selection, not the full generative-modeling literature. Counts through 2026-H2 are as of a fixed August 2026 cutoff.
The assignment rule assigns the lower level when a causal link isn’t demonstrated. This is conservative by design but means a system with a genuine L3 mechanism that the paper failed to evaluate cleanly gets marked L2. Reproducibility of the taxonomy depends on how strictly you apply the tests.
Finally, the proposed “generator-as-controller” future direction, where a unified visual policy handles observation, tool use, and revision on shared state, is a research position, not a demonstrated regime. The authors are explicit that a single model with a fixed inference path stays at L0 no matter how large it is.