CommerceGen-Studio turns messy e-commerce product metadata into publishable main images by chaining a prompt-optimizing agent, a layout-rewriting VLM, and a multi-condition diffusion generator trained with alternating reinforcement and distillation, running at a few seconds per image in production.
On e-commerce search pages, a shopper decides whether to click based on the product image and title. Modern search engines resolve a query to a specific variant (a Stock Keeping Unit, or SKU (Stock Keeping Unit)), but merchants rarely upload a nice marketing image per variant. They upload plain white-background product shots. Dropping those directly into search results measurably hurts click-through rate.
So you need to automatically generate a polished main image per SKU: product in a scene, trademark intact, selling-point text rendered correctly in Chinese, no hallucinated specs. Prior approaches split this across cascaded “plan then render” systems that accumulate errors, or use end-to-end poster models that trade off text accuracy against subject fidelity. Open editors like FLUX.2 [klein] or Qwen-Image-Edit lack the upstream step that reconciles conflicting SKU metadata and the downstream layout control the authors need.
The pipeline has three stages that mirror the actual problem: figure out what to say, decide where to put it, then render it.
Stage 1, selling-point extraction. SKU info is scattered across the item image’s OCR text, the SKU image’s OCR, structured attributes, title, and category. These sources overlap and contradict (the item image says “iPhone 17” but the SKU is actually an iPhone 18). PERL-Agent treats prompt construction as an optimization loop with four roles: a Generator drafts a selling-point list, a Judge scores it on a six-dimension rubric (source compliance, expression quality, category fit, red-line check, structure, business value), an Analyzer attributes the lowest-scoring dimension to a failure pattern, and a Refiner edits the prompt with a minimal diff. One category-adaptive prompt covers everything. The loop optimizes the worst dimension, not the mean, because a fluent but non-compliant description is unusable.
Stage 2, layout rewriting. Layout Thinker Optimization (Layout Thinker Optimization) inserts a VLM between the selling-point list and the generator. It reads the list plus the white-background product and trademark crops, then emits a layout caption that specifies subject position, background style, text hierarchy, and trademark placement. Training uses supervised fine-tuning followed by Group Relative Policy Optimization (GRPO) with two reward domains: one scores captions directly, the other renders them with the frozen generator and scores the resulting images. At each step, a coin flip picks one domain to backprop through, which prevents the two reward signals from fighting each other in the same gradient update. Unlike the generator’s reinforcement stages, LTO fully fine-tunes the upper transformer blocks rather than using LoRA, because layout planning requires re-attending to the product image’s structure, which the authors found a low-rank update too restrictive to capture.
Stage 3, multi-condition generation. SKU-DiT is a 6B diffusion transformer that jointly conditions on the white-background image, trademark crop, and layout caption. It is trained with Reinforce-Distill-Reinforce (Reinforce-Distill-Reinforce):
# Phase I: train two specialist teachers with DiffusionNFT
teacher_layout = train_rl(backbone, data=complex_layout, reward=R)
teacher_subject = train_rl(backbone, data=fidelity, reward=R)
# integrate into one student via on-policy distillation
student = distill_opd(teachers=[teacher_layout, teacher_subject])
# Phase II: compress multi-step sampling to few-step
student_fast = dynamic_backward_sim_distill(student, steps=8)
# Phase III: re-run RL on the few-step student
final = train_rl(student_fast, data=cleaner_subset, reward=R)
The reward is a geometric mean of instruction following, content fidelity, and aesthetics, minus an OCR penalty from PP-OCRv6 when text is wrong in non-product regions. The geometric mean punishes imbalanced quality, so a single bad dimension tanks the score. Phase III matters because distillation blunts font strokes and softens trademark edges, and a second round of RL on the already-fast student recovers that detail under the sampling budget it will actually face.
The authors release CommerceGen-Bench, 400 hard cases from live search traffic (thin or transparent subjects, tiny trademarks, long selling-point text), scored 0 to 10 on five dimensions by Gemini-3-Flash.
•
Among open-source baselines on both single- and multi-image editing tracks, with and without prompt enhancement, CommerceGen-Studio has the highest Overall score. The next-best open-source system, Qwen-Image-2.1, scores 7.29 / 6.76 without prompt enhancement vs. CommerceGen-Studio’s 9.40 / 9.44.
•
Against much larger closed-source systems (GPT-Image-2.5, Seedream 5.0 Pro, Nano Banana Pro), the 6B model is competitive overall and beats several of them on Instruction Following and Product Integration, the two dimensions the authors call out as determining whether an image is publishable.
•
Ablation on 400 bench samples: starting from an open-source VLM + base SKU-DiT scores 6.509; adding supervised fine-tuning of the rewriter lifts it to 7.832; adding reinforcement + distillation reaches 8.253 at 8 sampling steps; the third RL phase recovers to 8.271; adding full LTO yields 9.440. The LTO step is the single largest late-stage gain.
•
Qualitatively, baselines without prompt enhancement fail with product-identity drift, dropped or garbled Chinese text, and unstable composition when multiple reference images must be fused. The authors note these as observations, not claims from the respective papers.
•
The system runs in production at Alibaba generating millions of main images per day at a few seconds latency per image.
If you are building a product-image pipeline where metadata is heterogeneous and partially contradictory, the PERL-Agent pattern (closed-loop prompt refinement with a multi-dimension rubric, optimizing the worst dimension and rolling back on the specific failing axis) is a transferable idea that does not require any model training. Worth trying before you reach for fine-tuning.
If you have a text-plus-layout generation problem and your single-step prompt rewriter keeps producing captions that look fine but render badly, the dual-reward GRPO setup in LTO is informative: score the caption directly and score the rendered output, normalize within each domain’s rollout group, and pick one domain per optimization step. The paper’s framing is that mixing both rewards in the same gradient causes interference.
For latency-constrained diffusion deployment, the RDR recipe (train the full model with RL, distill to few steps, then run a second round of RL on the few-step student) is a concrete answer to the usual complaint that distillation destroys exactly the fine details reinforcement learning installed. The second RL round is cheap (~2.5K grouped trajectories) relative to the gain.
CommerceGen-Bench itself is positioned as a reproducible hard-case benchmark across e-commerce categories; the paper describes the collection and judging protocol but the supplied text does not include a release link, so check the arXiv page before planning to use it.
Caveats on scope: the baselines are run through CommerceGen-Bench with Gemini 3 Flash as judge, so comparisons inherit that judge’s biases. Beating closed-source systems on PI and IF is specific to this benchmark and these two dimensions; on Product Fidelity and Artifacts, GPT-Image-2.5 and Seedream score higher.
The generator backbone Swift-Image 6B and several training components (DiffusionNFT, DiffusionOPD, the Qwen3.8-27B LLM used for all four PERL-Agent roles) are in-house or recent and may not be independently reproducible outside Alibaba. The paper does not specify training compute.
The reward is defined by rubrics scored by a VLM judge, so the system learns what the authors could articulate in a rubric, not what shoppers actually click on. The authors flag incorporating real user behavior as future work.
Evaluation uses Gemini 3 Flash as judge on their own benchmark. The authors note that prompt enhancement adds explicit placement and style constraints, so cross-setting Instruction Following deltas conflate prompt difficulty with model capability and should not be read as pure capability changes.
The comparison is almost entirely on CommerceGen-Bench; there are no numbers on an external editing benchmark in the supplied text, so the generality of the ranking beyond this hard-case set is not established here.