SCOPE trains a prompt-rewriting LLM for frozen image generators by pooling policy-update signals within semantic phrases (objects, counts, colors, relations) rather than across arbitrary subword tokens, so one image reward credits coherent compositional edits instead of fragments.
Suppose you have a frozen image generator like FLUX or SD3 in production. Users type “four red handbags on white pedestals, cup above the couch,” and the model gives you five handbags or the wrong layout. Retraining the generator is expensive and can wreck image quality, so a common fix is to put a small LLM in front that rewrites the prompt into something the generator handles better.
The standard way to train that rewriter is reinforcement learning: sample several rewrites, render each, score the image with an automatic evaluator, and nudge the LLM toward rewrites that score higher. Group Relative Policy Optimization (GRPO) is the current default for this loop because it skips learning a value network. The catch: the image gives you one scalar reward for the whole rewrite, and vanilla Group Relative Policy Optimization (GRPO) spreads that reward uniformly across every subword token. But the semantic unit that actually mattered (“four red handbags”) straddles multiple tokens that the tokenizer split arbitrarily. The reward signal gets fragmented across subword boundaries that have nothing to do with the compositional edit being rewarded.
The core idea: let the rewriter emit its own semantic segmentation alongside the rewrite, then weight the RL update so tokens inside the same semantic phrase move together.
Concretely, the rewriter (a Qwen3-8B model) is prompted to output something like <answer>a white orange | on a wooden table | lit from above</answer>. The | characters mark span boundaries. Before rendering, the trainer strips the separators and sends the plain prompt to the frozen generator. A format reward encourages the model to actually produce valid <answer>...|...</answer> structure; the paper reports over 99.2% of rollouts parse cleanly, with uniform Group Relative Policy Optimization (GRPO) used as fallback on the rest.
For each span, SCOPE looks at how much the current policy’s log-probability on those span tokens has shifted versus the pre-update snapshot. It averages those shifts within the span (so long spans don’t dominate), exponentiates with a sensitivity parameter, and clips the result to the range [0.8, 1.2]. In plain terms: spans whose token probabilities are moving in a consistent direction get their loss contributions scaled up or down together, while the completion-level advantage from the image reward stays shared across the whole rewrite. The per-span coefficients are then normalized so their token-weighted mean is 1, which keeps the overall update magnitude the same as vanilla Group Relative Policy Optimization (GRPO). The coefficients are detached from the gradient graph, so they modulate loss weight without introducing a new learning signal.
for rewrite in group: # N-1 sampled + 1 original
spans = parse_pipe_separated(rewrite.answer_field)
for s in spans:
delta = mean(log_pi_new[t] - log_pi_old[t] for t in s.tokens)
v[s] = clip(exp(beta * delta), 0.8, 1.2)
w = v / token_weighted_mean(v) # normalize, detach
A_tilde[t] = completion_advantage * w[span_of(t)]
loss += clipped_grpo_surrogate(ratios, A_tilde)
No extra image renders, no second reward model, no external parser. When the sensitivity parameter is zero the method collapses back to vanilla Group Relative Policy Optimization (GRPO).
The rewriter is trained against three frozen generators (Qwen-Image, SD3, FLUX.1-dev) and evaluated on GenEval and T2I-CompBench++, with vanilla Group Relative Policy Optimization (GRPO) sharing SCOPE’s reward pipeline as the main comparison.
•
Headline numbers from the paper: average relative gains over vanilla Group Relative Policy Optimization (GRPO) of 9.48% on GenEval and 7.62% on T2I-CompBench++.
•
On GenEval, SCOPE wins 17 of 18 per-category cells across the three backbones. The biggest lifts show up in object counting and spatial positioning, which the paper flags as persistent weak spots for frozen generators. On SD3, Position goes from 0.57 to 0.83 versus Group Relative Policy Optimization (GRPO); on FLUX.1-dev, from 0.62 to 0.84.
•
On T2I-CompBench++, vanilla Group Relative Policy Optimization (GRPO) and the RePrompt baseline actually underperform the unadapted Qwen-Image baseline on the overall score (0.6247 and 0.6246 vs 0.6280). The paper attributes this to uniform token updates distorting fine-grained descriptive phrases during exploration. SCOPE reaches 0.6665 and beats the unadapted baseline in every category.
•
Ablations isolate what matters. Applying the same per-token likelihood-shift reweighting without pooling into spans does worse than vanilla Group Relative Policy Optimization (GRPO) on all three backbones (the authors read this as subword noise disrupting coherent phrases). Replacing semantic boundaries with random partitions or fixed 3-token windows drops the Qwen-Image GenEval score from 0.95 to about 0.86 to 0.87. Both the pooling and the linguistic boundaries contribute independently; it is not just “group neighboring tokens.”
One qualitative observation the paper highlights: on a counterfactual “white orange” prompt, SCOPE’s rewriter is the only system (including the commercial gpt-image 2) that produces an actually white orange, which the authors credit to spans keeping “white” and “orange” bound as one semantic unit during training.
•
If you are already running Group Relative Policy Optimization (GRPO)-style RL on a prompt rewriter for a frozen image generator, SCOPE is a drop-in change to the loss: ask the model to emit |-separated spans, pool policy-ratio shifts inside each span, reweight the clipped surrogate. No new rewards, no extra renders, no new models. The paper reports it recovers vanilla Group Relative Policy Optimization (GRPO) exactly when the sensitivity parameter is zero, so you can anneal in.
•
If your pain point is specifically counting and spatial relations on compositional prompts, that is where the gains concentrate. If your evaluator rewards something else entirely (aesthetic preference, style fidelity), the paper does not evaluate that, so it is worth testing before assuming the mechanism transfers.
•
The approach depends on the rewriter being capable enough to reliably emit well-formed spans. The paper uses an 8B model and sees a 99%+ parse rate; smaller or less instruction-tuned rewriters may hit the fallback path more often and lose the benefit. Worth checking the parse rate early in training.
•
As a conceptual takeaway, the ablation that semantic boundaries beat random windows is the useful piece of knowledge even if you do not deploy SCOPE: it suggests that when a scalar reward judges a composition holistically, splitting credit along linguistically meaningful chunks of the output does more than smoothing credit across neighboring tokens.
•
Everything is measured on compositional-alignment benchmarks (GenEval, T2I-CompBench++) whose evaluators also provide the training reward. The method is optimizing against the same scoring model used to judge it, which inflates gains relative to deployment with a different quality notion.
•
The Qwen-Image row of the RePrompt comparison is the authors’ own reproduction, not numbers from the original RePrompt paper, which did not report Qwen-Image.
•
The hybrid advantage rule (switch between baseline subtraction and group z-normalization based on a threshold) is a tuned choice; the ablation shows all three advantage variants beat vanilla Group Relative Policy Optimization (GRPO), but the exact ranking depends on it.
•
Span coefficients are hard-clipped to [0.8, 1.2]. That bounds how aggressively any one span can be up- or down-weighted, which keeps training stable but also limits how much a truly dominant phrase can be emphasized.
•
Gains are reported as averages over three backbones on two benchmarks. Generalization to other generators, other reward models, or non-English prompts is not evaluated here.