UI-Venus-2 is a foundation GUI Agent from Ant Group that scales three coupled axes (170+ multilingual apps, deep-research task generation, and dual trace/step VLM-as-a-Judge verification) to train a single model covering mobile, web, desktop, grounding, and CAPTCHAs.
You ship a browser or desktop automation agent. It works on demo flows, then breaks on a Chinese-language app you didn’t train on, on a login page that throws a CAPTCHA, or on a task where the agent “claims” success but the file was never saved. The team’s diagnosis is that these failures are not a single-model problem. They are a joint failure of narrow environment coverage, sloppy synthetic tasks, and reward signals that can’t tell partial progress from actual completion. Prior open agents in this line (UI-Venus-1.5) had the training recipe. UI-Venus-2 keeps the recipe and instead scales what the agent sees, does, and gets rewarded for.
The backbone recipe is inherited: three stages on top of Qwen3.5-9B and Qwen3.6-27B initializations. Stage 1 is multimodal mid-training on simulated mobile, web, and OS trajectories. Stage 2 is per-domain offline Reinforcement Learning for GUI Agents (grounding, CAPTCHA, mobile, web, computer separately). Stage 3 merges those specialists into one policy via MOPD.
The interesting bit in Stage 3 is that vanilla on-policy distillation treats reasoning tokens and action tokens the same, but only the action actually moves the environment. So they weight the distillation signal by whether the student’s action is (a) fully correct — suppress it, (b) right type wrong parameters — supervise the parameter span, or © wrong type — supervise the type token and mask the now-meaningless parameters. The teacher also gets a hidden hint containing the correct action type when scoring the student, so its per-token score reflects “given the right action, how off is this token,” while the student never sees the hint.
The other contribution is a closed-loop data factory. A per-app capability catalog is seeded from docs and forums, tasks are sampled from it under a validity gate, trajectories are executed, and every rollout (success or failure) writes state observations back into the catalog to steer future sampling. Verification runs at two levels: Self-Grounded Verification (SGV) extracts keypoints from the task, segments the trajectory into windows, and classifies each run as completed, partial, infeasible, or failed. Sample-level verification judges each individual step before execution as correct, exploratory, ineffective, or incorrect.
for round in range(N):
catalog = update_from_rollouts(catalog, prev_rollouts)
tasks = sample_tasks(catalog) # single, composite, scenario
tasks = [t for t in tasks if validity_gate(t)]
rollouts = [execute(t) for t in tasks]
for r in rollouts:
keypoints = extract_keypoints(r.task)
label = sgv_judge(r, keypoints) # completed/partial/infeasible/failed
step_labels = [judge_step(s, r.task) for s in r.steps]
catalog = writeback(catalog, r, label, step_labels)
prev_rollouts = rollouts
The prevailing move in open GUI agents is to scale one axis (bigger model, or more trajectories, or a better verifier) and hope the others catch up. UI-Venus-2’s thesis is that environment coverage, task construction, and reward verification are one system: a weak verifier turns extra environments into reward-hackable noise, and rich verification without diverse environments just overfits. The evidence to watch is not the headline scores but the safety and long-horizon benchmarks, where the gap over same-scale bases is much larger than raw grounding accuracy would predict.
The most load-bearing result is on OSBlind, a benchmark where benign-looking instructions hide harmful outcomes. Every baseline, including strong commercial models, has attack success rates between 79.4% and 93.6%. UI-Venus-2-9B and 27B cut this to 48.8% and 47.9% — roughly halving harmful execution on their base Qwen models (79.4% and 89.3%). This is the finding that suggests the training pipeline actually changes agent behavior rather than just chasing benchmark numbers.
Headline benchmarks then confirm the picture:
•
MobileWorld (long-horizon cross-app, 50-step GUI-only): 27B reaches 76.1%, ahead of Kimi-K3 (74.4%) and GPT-5.6-Sol (70.1%), behind the specialized Qwen-UI-Agent-27B (82.1%).
•
OSWorld-Verified: 27B at 80.5%, above Seed-2.1-Pro and GPT-5.5 (~78.7–78.8%), below the source-reported Claude-Opus-4.8 (83.4%). The authors flag that scaffolds differ, so treat as reference not controlled comparison.
•
WebVoyager (refreshed 595 tasks): 27B hits 93.4%, best in the table.
•
Odysseys (long-horizon cross-site, rubric-scored): 27B at 80.4 avg / 66.3 perfect, beating the strongest baseline by 11.5 and 21.8 points.
•
CAPTCHA: on NextGen-CAPTCHAs, 27B reaches 54.5% vs 20.4% for Seed-2.0-Pro; on Open CaptchaWorld the margin collapses to 0.7 points, showing frontier VLMs already handle common CAPTCHA formats.
•
OSWorld 2.0: everyone struggles. 27B gets 2.8% Binary Accuracy / 13.2% Partial. GPT-5.5 leads at 13.0% Binary. Long-horizon desktop remains open.
Reach for this when you’re building a browser or desktop agent and your bottleneck is data, not modeling. The lesson to steal is the capability-catalog loop: don’t hand-write task lists, mine app functionality from docs and forums into a structured registry, generate tasks against it under a validity gate, and let failed rollouts update the catalog’s sampling weights. Pair that with SGV-style keypoint verification instead of “did the final screen look right,” because partial-vs-failed is the distinction your RL reward needs.
The team releases weights for UI-Venus-2-9B and 27B, code, and a project page. They also introduce VenusBench-CAPTCHA (219 screenshots, 8 interaction types with executable action scoring), which is directly usable as a regression test for any agent that has to get through login walls. The unified action space (Click, Drag, Swipe, Type, Hotkey, plus platform-specific extensions with coordinates normalized to [0, 999]) is a reasonable template if you’re designing your own. If your agent stalls in the real world, the fix is rarely a bigger model. It’s usually that your environments, your task generator, and your verifier are out of sync with each other, and the weakest of the three is silently capping the other two.
•
OSWorld 2.0 numbers are brutal for everyone (best is 13% binary). Long-horizon desktop workflows spanning hundreds of actions are not solved here, only advanced.
•
Baseline comparisons throughout the paper mix action scaffolds and reasoning-effort settings; the authors flag this, but it means most tables are “benchmark-level references,” not controlled ablations of the training recipe.
•
The paper does not ablate the capability-catalog loop or SGV against a simpler pipeline, so we can’t tell from this report how much of the gain comes from verification vs. sheer environment scale vs. MOPD.