BadWAM attacks robot policies that couple action prediction with future-imagination by finding small image perturbations that shift the executed action while leaving the imagined future visually intact, cutting closed-loop task success from 96.5% to 43.1% on one model.
You’re shipping a robot manipulation stack that uses a World-Action Model and you added a safety monitor: before the arm moves, the model renders what it thinks the next few seconds will look like, and a checker inspects that rollout for anything unsafe. The intuition is that if the imagined future looks fine, the action must be fine. This paper shows that intuition is wrong. An attacker with only black-box query access to the model can nudge a camera frame within an imperceptible bound and desynchronize the two outputs: the future prediction stays close to the clean one, but the action chunk drifts enough to fail the task. Prior adversarial work on embodied policies (for example BadWorld) went after either the perception label or the imagined rollout itself. BadWAM instead targets the alignment between imagined future and executed action.
A World-Action Model takes an image plus a language instruction and emits two things: a short action chunk to execute, and a predicted future (either latent states or decoded video frames). BadWAM defines two interface-level distances: how far the attacked action chunk drifts from the clean one, and how far the imagined future drifts. The stealthy attack wants the first distance large and the second small, under a bounded pixel perturbation. Since the attacker has no gradients, BadWAM uses Zeroth-order optimization: at each replanning step, sample a handful of random perturbation directions, query the model on positively and negatively perturbed inputs, estimate a finite-difference gradient of the objective, and take a projected step inside the L-infinity ball. Two attack modes fall out of the same optimizer by swapping the scalar objective. The action-only attack maximizes action drift alone. The imagination-preserving attack adds a penalty term weighted by lambda that discourages changes to the predicted future, trading raw attack strength for stealth.
for step in episode:
obs = camera()
delta = zeros_like(obs)
for _ in range(query_budget):
u = random_rademacher_direction()
j_plus = objective(model(obs + delta + c*u))
j_minus = objective(model(obs + delta - c*u))
delta = clip_linf(delta + eta * (j_plus - j_minus)/(2*c) * u, eps)
action, future = model(obs + delta) # robot executes action
The scalar objective is either action-drift alone, or action-drift minus lambda times future-drift.
The prevailing safety intuition for world-action models is: if the model can imagine what will happen, we can inspect that imagination and catch bad plans before execution. This paper shows the opposite. A plausible imagined future is not evidence of a safe action, because the two outputs can be pulled apart by a small input perturbation. Safety has to be defined as synchronization between imagination and action, not plausibility of imagination alone. The clearest evidence is the matched-strength comparison: under the same perturbation budget, the imagination-preserving attack reduces predicted-future drift on 39 of 40 LIBERO tasks while still tanking closed-loop success.
•
The stealth mode barely costs attack strength. On the joint WAM, the action-only attack drops success to 61.5% and the imagination-preserving attack drops it to 63.0%, a ~1.5 pp gap despite the second one explicitly holding the imagined future close to clean. That gap is the load-bearing finding: it means a future-inspecting monitor gains almost nothing.
•
Headline degradation is large across three WAM variants. Clean success on LIBERO is 96.5% / 96.7% / 100.0% for action-only, joint, and IDM WAM models; under the action-only attack it falls to 43.1% / 61.5% / 66.1%, all within an L-infinity bound of epsilon=0.06.
•
Failures compound, they don’t strike once. Per-replan action shifts grow over the episode in failing rollouts, and pass@k curves for attacked models stay flat below clean across 20 trials, so it isn’t unlucky-seed noise.
•
Failures cluster where geometry matters. On the action-only WAM, spatial tasks fall from 96.5% to 16.0% and long-horizon from 92.5% to 23.5%, while object-centric tasks stay at 93.0%.
•
Perturbations transfer between WAM variants (source-model attacks still drop target success by 30-40 pp), so this isn’t overfitting to one action head.
•
Simple defenses are diagnostic, not solutions. JPEG-noise ensembling recovers attacked success to ~89-90%, but the paper flags these as non-adaptive, and a consistency-based detector catches only 13-21% of attacked replans at a 5% false-positive rate.
Reach for this if you’re building or auditing a robot policy that exposes imagined futures for a safety gate, Model Predictive Control (MPC) planner, or imagine-then-check verifier. The concrete lesson: your monitor needs to score the joint consistency of predicted future and action chunk, not just the visual plausibility of the future. The paper’s diagnostic protocol, log action distance, predicted-future distance, per-horizon shifts, and per-suite failure distribution, is directly borrowable as a red-team checklist for a WAM-based controller.
Code, project page, and model artifacts are released at GitHub, with a homepage and a Hugging Face collection linked from the paper. Experiments run on LIBERO and RoboTwin, both public manipulation benchmarks, so a red team can reproduce the closed-loop protocol without new data collection. License is not specified in the paper text.
A world model that can imagine a safe future is not the same as a policy that will execute one; safety has to be measured as agreement between the two, not the appearance of either.
•
The threat model assumes the attacker can inject bounded perturbations into the raw camera stream before the model sees it. If your deployment protects the sensor pipeline (signed frames, trusted camera firmware), this attack surface shrinks considerably.
•
The imagination-preserving attack’s stealth advantage is measured against a hypothetical monitor that looks only at visual future plausibility. A monitor that already checks action-future consistency, which is what the paper recommends building, would presumably catch it; the paper doesn’t evaluate such a monitor.
•
Query cost is nontrivial (17 forward passes per replan at the default budget, tens of seconds of prototype-implementation wall-clock), so a rate-limited or latency-monitored inference API partially defends by making the online search expensive.