ABot-World-0 is an action-conditioned video world model that streams a playable, keyboard-controlled world at 720p and up to 16 FPS on a single NVIDIA RTX 5090 by pairing a causal few-step Diffusion Transformer (DiT) with a distillation stage (LongForcing) that supervises the model’s own long rollouts against an extended-horizon teacher.
Suppose you’re building an interactive demo where a user presses W/A/S/D and expects the scene to keep evolving coherently for minutes. Today, most controllable video generators either need a datacenter GPU or fall apart after a few seconds of autoregressive rollout: characters morph, motion freezes, textures repeat. Prior interactive world models like Genie 3 from DeepMind proved the concept but aren’t something you run on a desktop. This paper’s contribution is a full-stack recipe, model, training, and inference, that makes a persistent playable world fit in ~19 GiB of VRAM on one consumer card while staying stable over hour- and day-scale sessions.
The model is an action-conditioned video Diffusion Transformer (DiT) built on top of the pretrained Wan2.2 backbone. Actions are 8-key keyboard presses (WASD for movement, IJKL for camera), packed to align with the Variational Autoencoder’s temporal compression and added directly to the patch embeddings. A separate reference-character memory stores identity tokens for third-person characters so appearance doesn’t drift.
Training goes in three stages. First, a bidirectional teacher learns action-to-video dynamics with full future context. Then it’s converted to a causal student via teacher forcing (causal attention mask, ground-truth history) and ODE distillation that compresses many denoising steps into a few. The key stage is LongForcing: the student generates long self-rollouts, and an extended-horizon teacher supervises those trajectories at the distribution level using Distribution Matching Distillation (DMD). The intuition is that short-horizon training never sees the drifted states the model actually enters after 30 seconds of rollout, so you train explicitly on those states.
# Rollout loop (conceptual)
kv_cache = BoundedKVCache(window=W)
memory_tokens = vae.encode(reference_images)
while True:
action_chunk = read_keyboard(next_4_frames)
latent_chunk = dit.denoise_few_step(
noise, history=kv_cache, actions=action_chunk,
memory=memory_tokens, steps=few)
frames = light_vae.decode(latent_chunk)
stream(frames)
kv_cache.append_and_evict(latent_chunk)
On the systems side: a pruned LightVAE decoder, FP8 (and more aggressive) quantization of DiT linear layers, SageAttention2 kernels, a Triton Fast-RoPE, FramePack-style module swapping to keep peak VRAM down, and a bounded local KV cache so context memory doesn’t grow with session length.
The usual fix for autoregressive drift is architectural: sliding windows, attention sinks, structured memory, keeping the model tethered to an initial frame. This paper argues the opposite. Drift is a training-distribution problem, so train on the drifted states directly, use the student’s own long rollouts as inputs, and let an extended-horizon teacher regularize the resulting distribution. The load-bearing evidence is the 60-second ablation against a Causal Forcing-style baseline, not the headline FPS number.
Over 60-second rollouts, LongForcing keeps HPSv3 scores higher and saturation, blur, and patch-repetition metrics lower than the Causal-Forcing-style baseline. The gap widens in the second half of the rollout, which is exactly the regime short-horizon training doesn’t cover. That’s the finding that makes the thesis land.
•
On WorldRoamBench, the model posts competitive scores across action controllability, visual quality, physical mechanics, and memory retention against Genie 3, HappyOyster, LingBot-World, and HY-World 1.5. The paper doesn’t claim a single-number SOTA.
•
Systems ablation on one RTX 5090 at 1280\u00d7704: the base config and a SageAttention2-only variant OOM. Adding LightVAE gets to 9.1 FPS at 20.5 GiB. Adding FP8 DiT reaches 12.4 FPS at 15.9 GiB. Adding Fast-RoPE reaches 13.3 FPS. More aggressive low-bit configs push up to 16 FPS with peak VRAM below 19.3 GiB. Action-to-first-frame latency is reported as 1.2 s.
•
Qualitative: hour- and day-scale rollouts retain scene structure; the model produces plausible physical effects (footprints in snow, water disturbances, wall blocking, railing collision) it was never explicitly supervised on.
Reach for this design if you’re building an interactive video experience, a game-like demo, a driving sim, an embodied-agent training environment, where a user’s discrete actions need to drive a coherent visual stream on a single consumer GPU. The practical recipe transfers even if you swap the backbone: train a bidirectional action-conditioned teacher, distill causally with teacher forcing then ODE distillation, then add a self-rollout distribution-matching stage against a longer-horizon teacher. On the inference side, the paper is a useful checklist: lightweight VAE, FP8 linears, efficient attention kernel, bounded KV cache, memory-aware module swapping.
The paper doesn’t link a code repo, released weights, or a public dataset. The training data mixes AAA game recordings, 3D Gaussian Splatting-reconstructed proprietary scenes from AMAP, and internet video, most of which is non-public. WorldRoamBench is cited as an external benchmark. Treat this paper as an architectural and systems blueprint rather than an artifact release.
If your autoregressive model drifts, don’t just widen its memory, train it on the drifted states its own rollouts produce. Architectural anchors (sinks, reference frames, sliding windows) buy time but eventually restrict what the model can imagine. Distribution-matching against a longer-horizon teacher on the student’s own trajectories attacks the actual mismatch: the states at second 45 look nothing like the states at second 5, and short-horizon training never taught the model how to behave there.
•
Nothing is released. No code, no weights, no data. Every claim rests on the paper’s own eval; there’s no way to reproduce or probe failure modes independently.
•
The comparison to Causal Forcing is described as an “adapted baseline” under the authors’ own protocol, not the original method as its authors would run it. That’s the load-bearing ablation, so the framing matters.
•
Latency and FPS numbers are for a specific resolution (1280\u00d7704, 12 decoded frames per chunk) on one specific GPU. The 1.2 s action-to-first-frame delay is still noticeable for anything requiring tight control; this is closer to “navigable video” than to a game engine’s input responsiveness.