GE-Act 2.0 is a robot manipulation model that predicts a full future video in one denoising pass and reads actions from it, and it trains its visual generator, its action decoder, and its own Control-Oriented Autoencoder (CoAE) from scratch on manipulation data instead of borrowing a pretrained video generator.
Suppose you want a single robot policy that can pick, pour, wipe, stack, and fold across kitchens it has never seen. Two dominant recipes exist. The first, Vision-Language-Action model policies, maps camera plus instruction straight to motor commands. It scales, but the model never has to model physics explicitly. The second, world-action models, first predicts what the scene will look like next, then infers the action that would produce that change. This gives you an inspectable video of the robot’s “plan” and, in principle, lets you pretrain on cheap data: action-free video for the generator, and instruction-free robot logs (including failures) for the action decoder.
The catch: almost every prior world-action system starts from an off-the-shelf video generator like WAN and spends its effort bolting an action head onto it. Nobody has really answered what happens if you pretrain the whole world-action stack from scratch on manipulation data, and how capability scales with that data. This paper is that experiment.
Three components, each solving one concrete problem.
A compact visual code (CoAE). Standard video autoencoders keep hundreds of tokens per frame because they optimize for pretty reconstructions. That’s wasteful when the downstream job is control. Control-Oriented Autoencoder (CoAE) compresses each frame 64x on each spatial axis down to a 4x6 grid, so 24 tokens per frame, and trains not just for pixel reconstruction but also to match features from three frozen teachers: SigLIP-2, V-JEPA 2.1, and DINOv3. Semantics, spatiotemporal structure, and dense visual detail. On an action-recovery probe, this 24-token code trails the 256-token teachers by only 13 to 31 percent.
One-step future prediction (SVP). A conventional video diffusion model produces a future by running K sequential denoising steps. If you want to backprop an action loss through that future during joint training, you have to differentiate through all K steps. Expensive. So the single-step visual planner uses MeanFlow: instead of learning the instantaneous velocity at each noise level, it learns the average velocity across the whole noise interval, so one forward pass takes pure noise to a clean future. Instructions are grounded in the current scene via a frozen Qwen3.5 VLM whose per-layer text states are gated and fed into the generator through cross-attention. The output is multi-scale: several dense near-term frames plus a couple of sparse far-horizon frames.
Inverse dynamics model (IDM). A separate network that takes the current observation, the predicted future latents, and proprioception, and outputs a dense chunk of joint-space actions via Flow matching. Because SVP is one-step, IDM can be pretrained alone on raw robot trajectories (no language, no success labels) and then wired to SVP later.
KASO, the alignment trick. Here’s the subtle bug. During joint training, IDM sees a generated future and is asked to predict the recorded action. But manipulation has multiple valid ways to complete a task. Reach left or reach right, grasp now or later. The recorded action reflects one mode; the freshly sampled future may depict another. Training on that mismatched pair teaches IDM to average across modes and collapse behavioral diversity. The authors call this the validity gap. Knowledge-Aligned Selective Optimization (KASO) fixes it by sampling N candidate futures per step, scoring each by whether the current IDM would produce roughly the same action from it as from the recorded future, and only backpropagating through the top-k (they use k=1 out of N=4).
for step in training:
z_rec, a = batch # recorded future, recorded action
noises = [sample_noise() for _ in range(N)]
with no_grad():
candidates = [svp.one_step(obs, instr, n) for n in noises]
v_ref = idm.velocity(a_noised, obs, s, z_rec)
energies = [||idm.velocity(a_noised, obs, s, z_hat) - v_ref||^2
for z_hat in candidates]
best = argmin(energies)
z_selected = svp.one_step(obs, instr, noises[best]) # replay WITH grad
loss = fm(a, idm(obs, s, z_selected)) + L_svp_recorded + L_idm_recorded
loss.backward()
A controlled toy problem with four visual modes mapping to two action modes shows the failure vividly: naive end-to-end co-training collapses both distributions, retaining the pretraining losses saves the video modes but still collapses actions to the mean, and only KASO recovers the bimodal action structure.
Scaling the co-training data works, and it works broadly. They train four checkpoints on nested pools of 300, 1,200, 5,000, and 30,000 hours of manipulation data, then evaluate zero-shot (no per-task fine-tuning) on 100 real-robot tasks across 20 skill groups, on two robot embodiments: G1-OP and G2-90D. On G1-OP the suite-level success rate rises from 17.1% to 44.1%. On G2-90D it goes from 13.4% to 31.1%. Neither curve looks saturated. 19 of 20 G1-OP skill groups and 18 of 20 G2-90D groups improve from smallest to largest scale.
Cross-embodiment transfer is real. G2-90D is under 2% of the co-training mixture, yet it gains 17.7 points. Several skills with under 2 hours of G2-90D-specific data still go from 0% to nonzero success. The authors read this as evidence that the shared corpus transfers usefully to a data-scarce embodiment.
Data coverage predicts skill success. For each skill group, plotting log training hours against logit success rate gives Pearson r=0.80 and Spearman rho=0.85, with roughly 1.94 logit units of gain per decade of data. This is what the paper leans on to argue that uneven per-skill capability is largely a coverage phenomenon, not raw motor difficulty.
Instruction grounding. On a controlled scene suite, the model reliably follows object identity, color, direct left/right position, and shape references (Follow Score at least 90%), stumbles on size superlatives, and mostly fails on ordinal references like “the second cup from the left” (13% to 27%). This tracks the frequency of those descriptors in both natural referring-expression corpora and their own training data. Qualitative stress tests further show the policy can redirect mid-reach when the target instruction changes, and will place a cup into a shoebox when told to, overriding the more conventional “cup goes on table” association.
Isolated KASO ablation. In a matched 300-hour alignment stage on top of the full pretrained components, KASO lifts single-object pick success from 12% to 40% and four-object pick success from 27.5% to 37.5% over plain end-to-end co-training. Note this is an alignment-stage ablation, not the full 30,000-hour model.
•
If you’re building a world-action robot policy and defaulting to a pretrained natural-video generator as the visual backbone, the CoAE result is worth taking seriously. A domain-specific autoencoder aligned to multiple frozen teachers gives you a much smaller token budget per frame with almost no loss in downstream action recovery. Worth testing in your own stack if inference-time token count is a bottleneck.
•
The KASO idea generalizes beyond robotics: any setup where you jointly train a generator and a consumer under multimodal ground truth risks the same validity gap. Best-of-N filtered by consumer compatibility, with the discarded candidates thrown away rather than downweighted, is the concrete recipe. The ablation shows this matters even at modest scale.
•
If you’re planning a data-collection campaign for a new robot embodiment, the cross-embodiment transfer result is encouraging but not a license to skip the new data entirely. Coverage of skill categories in the shared corpus predicts downstream success, and rare descriptors (ordinal references, size superlatives) remain weak. Broaden the tail rather than deepen the head.
•
Simulation benchmark results (RoboTwin, LIBERO-Plus, GenieSim-Instruction) are reported with in-distribution SFT then OOD evaluation, not zero-shot. Treat them as “can this model adapt reasonably” rather than “is this model zero-shot deployable in your simulator.”
•
Released artifacts: the paper links a project page at ge-act-v2.github.io; no code or weights release is explicitly promised in the supplied text.
•
The main scaling comparison is not compute-matched. Each rung trains for one epoch or a fixed compute budget, whichever comes first, so improvements bundle data volume with total compute. The paper is upfront about this being a practical rather than isolated scaling study.
•
“Zero-shot” here means no per-task fine-tuning and held-out scenes, backgrounds, lighting, and object instances, on robots that dominate the training mixture. It does not mean deployment to a new robot morphology or a new lab from scratch.
•
The KASO ablation runs on a 300-hour alignment stage starting from full-scale pretrained components. Its absolute success rates should not be read as the full model’s performance, and the ablation does not sweep k or N.
•
Absolute success on many tasks remains modest. 44% suite-level success at 30,000 hours means most trials still fail, and 13 of 20 skill groups score below 50% on the harder embodiment. This is a low-level manipulation policy, not a general-purpose planner; long-horizon tasks and open-ended reasoning are explicitly out of scope.
•
Descriptor-frequency to grounding-success is described as an association, not a causal claim. Ordinal references may be hard both because they’re rare and because they require set-relative reasoning.