WorldReward is a single VLM-based judge that scores both “did the video follow the camera command?” and “does it look good?” by chopping paired videos into short action-aligned chunks, judging each chunk from a structured image grid, and voting. It becomes the reward signal for RL fine-tuning of an interactive video world model.
Imagine you’re shipping an interactive video generator: the user drags a mouse to pan the camera left, tilt up, walk forward, and your model has to produce a coherent video that actually executes those controls without the scene melting. To train or evaluate that model with RL, you need a reward that catches both mistakes at once: the camera went the wrong way, AND the wall texture drifted three seconds in.
Today people stitch together two unrelated signals. A geometry model like Depth Anything 3 recovers the camera path from the generated frames and compares it to the commanded trajectory. A separate image-quality scorer like HPSv3 rates sampled frames for aesthetics. The dominant recipe combining these, WorldCompass, uses both in parallel but each is blind to the other’s job. A trajectory checker can’t see flickering; a frame scorer can’t tell if the camera turned the wrong way.
The core idea: give one VLM a compact, well-organized view of the failure region and let it judge both dimensions from the same evidence. Feeding a whole long video and a 20-step action list to a VLM buries short-lived evidence (a single “turn left” affects maybe 4 frames out of hundreds).
So WorldReward compares two candidate videos A and B generated from the same source image, caption, and action trajectory, and does it chunk by chunk. Each chunk covers four consecutive actions. For each chunk the VLM receives a six-image bundle: the original source image (anchors scene identity), a frame-grid overview showing start/middle/end frames of the chunk for both videos, and four action-level panels that put A’s and B’s first-and-last frame for each action side by side. The VLM then emits two verdicts per chunk: an action-consistency winner (A, B, or Tie) and a visual-quality winner covering temporal stability, motion plausibility, and artifacts.
Chunk verdicts are aggregated by majority vote into two video-level preferences. Those two preferences plug into an RL loop for the world model using Pref-GRPO-style pairwise win-rate rewards inside the DiffusionNFT optimizer.
chunks = split_actions(actions, size=4) # prepend idle slot for source frame
for k, chunk in enumerate(chunks):
evidence = build_six_image_input(source_img, video_A, video_B, chunk)
r_act[k], r_vis[k] = vlm.judge(evidence, caption, chunk) # each in {A,B,Tie}
R_act = majority_vote(r_act)
R_vis = majority_vote(r_vis)
Training data is built by having Gemini 3.1 Pro produce chunk-level structured reasoning over ~100k chunks, then a GPT-5.5-driven tool-using agent audits each annotation (adaptively pulling in panels rather than seeing everything at once), then humans confirm the agent’s proposed edits. The base VLM is Qwen3.5-9B fine-tuned to imitate the audited reasoning and verdicts.
The usual reflex for evaluating a controllable video generator is to bolt together specialized scorers, one for geometry and one for pixels, on the theory that no single model can do both jobs well. This paper argues the opposite: give one VLM the right compact evidence, the same six-image chunk view, and it can judge action-following and visual quality more reliably than either specialist, because the two judgments are grounded in the same interpretation of what the camera just did. The load-bearing evidence isn’t the headline win over specialists; it’s that annotation refinement (agent audit + human) is what pushes the student model past its own teachers.
The most telling result is the annotation-refinement ablation. Training on raw Gemini 3.1 Pro distillations gives 68.69% average agreement with human preferences, roughly matching the teacher used directly as judge. Adding the GPT-5.5 agent audit jumps it to 75.94%. Adding human confirmation of agent edits reaches 77.33%. The student ends up beating both proprietary VLMs it was distilled from, and the gain comes from cleaning the labels, not from model scale.
•
On WorldReward-Bench (760 human-labeled paired generations), WorldReward tops every baseline on all three dimensions: action, appearance, motion. It beats GPT-5.5 by 3.42, 1.45, and 3.56 points respectively.
•
Used as the RL reward for post-training HY-WorldPlay 1.5, it beats WorldCompass on combined-action accuracy by 1.58 to 2.78 points and on basic-action accuracy by 2.28 to 5.81 points across short, medium, and long horizons, while also raising HPSv3 visual quality in all settings.
•
Ablating any single input component (source image, frame grid, or action panels) hurts. Removing the frame-grid overview causes the biggest average drop.
•
Randomizing A/B order costs only ~1 point, so the model isn’t leaning on position.
Reach for this pattern when you’re training or evaluating a controllable generator (interactive video, agent trajectories, anything with a commanded action sequence and a long output). Instead of running one verifier for “did the action happen” and another for “does the output look right,” package short windows of paired outputs into a structured multi-image prompt for a VLM judge, and let it produce both scores from the same look. Then vote across windows so one flashy or broken segment doesn’t dominate.
The authors released a project page. WorldReward-Bench (760 human-annotated paired camera-conditioned video generations with separate labels for action, appearance, and motion) is a directly reusable evaluation asset for anyone building camera-controlled video models. The paper doesn’t specify a code or model-weight release beyond the site.
One judge, one look, two scores beats two specialists judging in isolation, but only if you feed it evidence organized around the moment of failure. The chunking and the six-image layout are doing as much work as the VLM itself; the annotation cleanup is doing more work than the distillation.
•
The approach assumes you can cleanly slice outputs into short action-aligned windows with matched paired candidates. Domains without discrete commanded actions or without a natural pairwise setup won’t fit.
•
The reported quality gains lean on proprietary frontier VLMs (Gemini 3.1 Pro to generate reasoning, GPT-5.5 to audit) plus human review. Reproducing the pipeline without that labeling budget is an open question the paper doesn’t address.
•
Post-training results use DepthAnything3 and HPSv3 as evaluation metrics, but HPSv3 is also the visual reward inside the baseline WorldCompass. Some of the head-to-head lift may reflect WorldReward optimizing for a broader notion of visual quality than what HPSv3 alone measures at eval time.