AgenticGen turns advertising video generation into a two-stage reasoning agent (pick a creative strategy, then write an executable draft) and trains both stages with rewards learned from real ad-delivery clicks, lifting online business metrics over a plain Supervised Fine-Tuning agent.
Suppose you run TikTok’s ad pipeline. An advertiser uploads product footage and you need to turn it into a short video that people actually click and buy from. Today, a video foundation model like Seedance 2.0 can render a slick clip from a prompt, but nothing in that model knows whether the resulting ad will perform once served. The optimization target the business cares about (click-through rate, conversion rate, advertiser value) never reaches the generator.
The paper’s baseline is that pre-existing setup: a non-agentic pipeline where humans hand-configure which editing template, music track, or reference video to use, and a generation model just executes. There is no closed loop from post-delivery metrics back to those decisions. The authors want a system where accumulated online feedback actually reshapes future generation choices.
Instead of asking one model to output a finished ad, AgenticGen splits the job into two reasoning steps performed by a VLM (specifically a fine-tuned Qwen3-VL-8B-Thinking):
1.
Strategy selection. Given the product info and available assets (video clips, images, their past performance), the agent picks a strategy group from a fixed catalog. Strategies fall into three families: lightweight asset editing (swap music, add hook, strengthen call-to-action), reference-guided generation (imitate a top-performing ad’s storyline), and cross-asset remixing (reorder or replace clips).
2.
Draft generation. The agent turns the chosen strategies into a structured draft: concrete music picks, storyboard, clip order, overlays, tool calls. An execution layer then runs Seedance 2.0 and CapCut to render the actual video.
Splitting the pipeline this way creates two places where a reward signal can bite. The training loop has four pieces:
•
Impression-balanced delivery. For each product, 12 candidate videos are shown to users with equal impression opportunity, bypassing normal recall/ranking. This strips out most of the advertising-system bias so CTR differences reflect the video itself.
•
Performance reward model. A Bradley-Terry loss pairwise scorer trained on winner-vs-loser pairs from that balanced traffic. It reads not just video frames but also audio/ASR, dense captions, storyline, and ad-specific cues (hooks, CTAs).
•
Rubric reward model. A separately trained judge (a fine-tuned Qwen2.5-Omni-7B) that scores pairs against human quality standards covering script, content, decoration, and audio. This catches Clickbait and visual defects that CTR alone rewards.
•
Two-phase policy training. First Direct Preference Optimization warms up both the selection and draft policies on the impression-balanced preference pairs. Then Group Relative Policy Optimization (GRPO) does on-policy RL: for strategy selection, the reward mixes a global prior (how well each strategy performs on average) with a local prior (overlap with the winning group for this specific product); for draft generation, it mixes the performance reward and the rubric reward, following Pref-GRPO’s pairwise win-rate trick. Invalid outputs get zeroed out by a format/constraint indicator.
# per training step, alternating between the two stages
for product in impression_balanced_batch:
strategies = policy_sel.sample(product, catalog, constraints)
drafts = policy_draft.sample(product, strategies)
videos = render(drafts, seedance, capcut) # executes tools
R_sel = lam*global_prior(strategies) + (1-lam)*jaccard(strategies, winner)
R_draft = w*perf_rm(videos) + (1-w)*rubric_rm.winrate(videos)
R_sel *= valid_format(strategies); R_draft *= valid_format(drafts)
grpo_update(policy_sel, advantages_from(R_sel))
grpo_update(policy_draft, advantages_from(R_draft))
Online A/B in the TikTok ad system is the headline evidence. The paper reports two successive comparisons:
•
Replacing the pre-agent, hand-configured pipeline with the SFT-only AgenticGen lifted CTR by +3.48%, CVR by +2.30%, Advv by +9.61%.
•
Adding DPO+GRPO on top of that SFT agent lifted CTR by +2.72%, CVR by +2.63%, Advv by +9.61% further. The second comparison is what isolates the RL contribution from the agentification.
Advv (advertiser value) grows faster than CTR×CVR would predict; the authors attribute this to higher-bid traffic having stronger source materials, which the agent can exploit better.
Offline results support the design choices rather than prove deployment gains:
•
Pairwise Bradley-Terry reward modeling beats pointwise regression on held-out impression-balanced pairs (60.85% vs 52.92%). Absolute accuracy is moderate because the reward model does not see user-side features that production CTR models use.
•
Adding audio, storyline, and ad-specific features to the reward model each contributes measurable gains over video-only input.
•
The rubric judge, after SFT, aligns much better with human labels (69.50% vs 55.40%) than with online preferences (53.30%), which is precisely why the authors treat it as a complement, not a substitute, for the performance reward.
•
In a GRPO reward ablation on draft generation, performance-reward-only maximizes the performance win rate but leaves rubric near chance; rubric-only inverts that trade-off; the weighted fusion wins on average, retaining most of the performance gain while pulling rubric up to 55.86%.
These are all within-system comparisons using the authors’ own reward models as judges. That is fine as an ablation of the training recipe, but note that on the offline draft-generation ablation, “winning” is measured by the same family of reward models being optimized.
•
If you are building a generative system whose real objective is a delayed, noisy online metric, the concrete pattern here is worth copying: reserve a slice of traffic for impression-balanced delivery, so within-group comparisons cancel out ranking/recall bias, then train a pairwise reward on those clean comparisons rather than regressing on raw CTR. This is the piece that makes the rest of the pipeline tractable.
•
If your generation task has any natural decomposition (pick a template, then fill it in), AgenticGen’s split gives you two places to attach rewards instead of one. Worth testing when a single end-to-end reward is too sparse to move intermediate decisions.
•
Pair a metric-driven reward with a rubric-driven one when the metric can be gamed. The ablation shows single-reward GRPO trades off in predictable, unwanted ways; the fusion weight (they use 0.6 toward performance) is a knob you’d retune per domain.
•
What this paper does not show: that the same recipe works without control of the delivery pipeline. The impression-balanced pool, the ability to log full trajectories, and the volume (1M pairs over a month) are prerequisites. If you only have hosted-API access to a generator and no ability to run balanced traffic experiments, the reward-modeling step here does not directly transfer.
•
All headline lifts are from A/B tests inside TikTok’s own ad system with the authors’ impression-balanced protocol. The paper does not report results on any external benchmark, and the strategy catalog is described only abstractly (“commercially sensitive”) in the appendix, so exact replication is not possible from the paper alone.
•
The reward models judge the RL policies in the offline ablations, so “win rate” gains there partly measure fit to the reward, not independent quality. The online A/B is the load-bearing evidence.
•
Feedback is dominated by CTR; CVR improved too, but the paper’s own analysis notes the reward model can’t see user-side cross features, so absolute reward accuracy is moderate (~61%). The system relies on RL to extract signal despite a noisy teacher.
•
The base VLM is Qwen3-VL-8B-Thinking distilled from a larger teacher, and the rubric judge is Qwen2.5-Omni-7B. Whether the recipe holds with different backbones or smaller models is not tested.