Text-to-image synthesis and editing — diffusion, autoregressive, and hybrid approaches, plus controllability techniques like ControlNet, LoRA, and editing pipelines.
Unified image generation: routing, efficient diffusion, and editable outputsThis guide focuses on three engineering problems within image generation: combining image understanding with synthesis, reducing diffusion-transformer cost, and choosing between raster editing and structurally editable design. It is not a survey of autoregressive generators or spatial adapters; use it when the hard decision is how capabilities share a model, how a diffusion stack reaches interactive latency, or whether pixels are the right final representation.Start with the featured overview if one model must inspect and produce images. Continue to capability composition and agent orchestration when generation also requires editing, reasoning, or tools. For deployment constraints, compare attention pruning, few-step distillation, and tokenizer–backbone–kernel co-design without treating results from different systems as a common ranking.For posters and infographics, generation quality alone may be insufficient: exact text, movable elements, and later manual edits favor a workflow that keeps layout in HTML/CSS and uses image models for isolated assets. If the deliverable is a finished raster image, native instruction-based editors may be the better fit.
Separate conflicting computation without severing shared knowledgeUnderstanding, generation, and editing can share useful knowledge while still competing for computation. These readings separate three mechanisms: route clean and noised visual tokens differently inside a unified model; distill several flow-based capabilities into one student; or let an agent policy coordinate reasoning, tools, and drawing. They solve different forms of interference and should not be treated as interchangeable designs.
DanceOPD: On-Policy Generative Field DistillationRead for a distinct way to compose text-to-image, local editing, global editing, realism, and guidance by distilling capability fields on student rollout states. It does not evaluate multimodal token routing.The abstract covers capability composition in flow-matching models, without implementation-level results.
ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image GenerationUseful when generation must coordinate reasoning, search tools, and drawing under one adaptive policy. It supports agentic orchestration, not claims about faster sampling or editable layers.The abstract compares ToolArtist with fixed and partially agent-controlled open-world generation pipelines.
When does joint image understanding and generation help rather than interfere?It helps when tasks share latent knowledge and the architecture preserves semantic interaction while specializing conflicting computation. In the featured study, clean image and text tokens stay on the language-model branch, while noised generation tokens use a separate branch connected through attention. Dense sharing and modality-based splitting produced opposite imbalances.Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System
How can one flow model combine text-to-image and editing capabilities?DanceOPD represents each source capability as a velocity field over a shared flow state space. A student queries a selected field on states from its own rollout and learns through velocity matching, aiming to add editing, realism, or guidance while preserving anchor generation quality. This addresses capability distillation, not multimodal token routing.DanceOPD: On-Policy Generative Field Distillation
When is an agent policy more appropriate than a fixed generation pipeline?Consider agent control when requests require search, external knowledge, multi-step reasoning, and image synthesis to be coordinated adaptively. ToolArtist post-trains one policy to reason, invoke tools, and generate images instead of prescribing a fixed workflow. It addresses open-world orchestration rather than denoising speed or layered output.ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation
Reduce diffusion-transformer cost at the right layerDiffusion and flow systems can spend time in attention, repeated sampling, tokenization, or systems overhead. One reading uses causal analysis to choose attention heads for pruning; another releases a 2–4-step open generator; a third co-designs the tokenizer, native-resolution backbone, kernels, and four-step variants. Match the intervention to the bottleneck, and evaluate each result in its reported hardware and model setting.
Text Template Tokens Are Implicit Semantic Registers in Diffusion TransformersRead to understand how semantic identity moves through a DiT and how causal interventions motivate selective attention-head pruning. It does not provide a pruning guarantee for every backbone.In the studied DiTs, pruning removed 20% of attention FLOPs with a 1.4-point GenEval decrease.
LLaDA-Image: Building Strong Image Generators with Fully Open Training RecipesA practical starting point for an open 6B DiT recipe, image-only pretraining, instruction-oriented editing, and 2–4-step distillation. The abstract does not isolate which component causes each quality gain.Results concern LLaDA-Image and English and Chinese Qwen-Image-Bench tracks.
Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and EditingRead for tokenizer–backbone–system co-design spanning native-resolution packing, kernel fusion, alignment, and four-step distillation. Its latency figures should not be compared directly with differently configured systems.Turbo latency is reported at 1024² on one NVIDIA A100.
Where does a DiT maintain prompt-specific object identity during denoising?Structural template tokens begin with little prompt-specific information but become attention sinks that causally maintain identity. Prompt semantics first enter image latents and are then read back into those tokens. The tokens that encode semantics at input therefore need not remain the main semantic registers during generation.Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers
How can attention be pruned without directly removing the mechanism that maintains identity?The interpretability study derives a training-free rule from causal interventions: prune heads that attend most strongly to prompt tokens rather than assuming those heads preserve identity. In the studied DiTs, this removed 20% of attention FLOPs with a 1.4-point GenEval decrease. Test the rule against the target architecture and quality criteria.Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers
Choose between raster editing and editable document structureInstruction-following image editors return flattened pixels. Posters, infographics, and related assets often need exact text, movable elements, and continued manual editing, making the output representation part of the system design. A coding-agent workflow can generate isolated visual assets while preserving text and layout as native HTML/CSS; native image editors instead optimize the raster workflow.
Editable Visual DesignUseful when a flattened bitmap is the wrong deliverable: it combines VLM planning, generated standalone assets, HTML/CSS, and rendering feedback to retain real text and movable layers.The abstract covers posters, infographics, related scenarios, and GUI-based editing.
LLaDA-Image: Building Strong Image Generators with Fully Open Training RecipesA practical starting point for an open 6B DiT recipe, image-only pretraining, instruction-oriented editing, and 2–4-step distillation. The abstract does not isolate which component causes each quality gain.Results concern LLaDA-Image and English and Chinese Qwen-Image-Bench tracks.
Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and EditingRead for tokenizer–backbone–system co-design spanning native-resolution packing, kernel fusion, alignment, and four-step distillation. Its latency figures should not be compared directly with differently configured systems.Turbo latency is reported at 1024² on one NVIDIA A100.
How can a system produce attractive graphics with accurate text and editable layers?Editable Visual Design assigns requirement analysis, planning, and aesthetic judgment to a vision-language model; calls an image generator for standalone assets; writes HTML/CSS; and iterates using rendered visual feedback. Real text and decoupled layers remain available for dragging and layout changes instead of being baked into a bitmap.Editable Visual Design