Xiaomi-Robotics-1 is a vision-language-action model pre-trained on 100k+ hours of handheld-gripper trajectories, then post-trained onto real robots. Its news is the scaling curve: bigger pre-training data and model size both keep lifting out-of-the-box success in unseen rooms.
Suppose you’re building a home robot that has to pack a suitcase, load laundry, and tidy a sofa in a customer’s apartment you’ve never seen. The dominant recipe today is Teleoperation: pay people to puppeteer robots through demos, which caps you at a narrow slice of tasks and homes. This paper takes a different route. It piggybacks on UMI (Universal Manipulation Interface) handheld grippers so humans can record manipulation data in the wild without a robot present, then teaches a real robot to imitate that behavior. The closest prior reference points in the paper are Physical Intelligence’s π0.5 and the authors’ own earlier Xiaomi-Robotics-0.
The model has two brains stitched together with a Mixture-of-Transformers pattern. A pre-trained vision-language model (Qwen3-VL) reads the camera and the instruction. A smaller Diffusion Transformer (DiT) then denoises a chunk of future actions using Flow matching, conditioned on the robot’s current joint state plus the VLM’s cached representation of the scene.
The interesting move is how they got labels for 100k hours of raw video. Manually writing an instruction for every clip is impossible at that scale. Instead, they slice each trajectory into fixed-length windows and ask a VLM to describe the state transition it sees: what did the gripper and the objects look like before, what do they look like after. The policy then learns “produce actions that turn scene A into scene B.” That reframing is what makes auto-labeling viable, because state-transition captions don’t require understanding the human’s intent.
Training happens in two stages. Pre-training uses only UMI handheld data with these state-transition captions. Post-training then swaps in ~10k hours of actual robot data (mobile manipulators, dual-arm rigs, plus open datasets like DROID and Bridge V2) with human-style imperative instructions like “pack the shoes.” This second stage does two alignments at once: handheld-gripper actions to real-robot action spaces, and descriptive captions to command-style prompts.
# Pre-training loop (conceptual)
for clip in umi_trajectories_100k_hours:
before, after = clip.first_frame, clip.last_frame
caption = vlm.describe_transition(before, after) # auto-label
loss = flow_matching(policy, obs=before, lang=caption,
target_actions=clip.actions)
loss += aux_action_head_loss(vlm_tokens) # speeds convergence
loss.backward()
The prevailing intuition in robot learning is that you need robot-collected demonstrations with human-written task labels, because that’s what matches deployment. This paper shows the opposite: you can pre-train on non-robot handheld data with machine-generated state-transition captions, and the resulting representations transfer cleanly to real robots after a smaller alignment stage. The load-bearing evidence is the transfer curve: pre-training scale on captioned handheld data monotonically raises real-robot success in unseen homes.
The finding that anchors the thesis is the transfer of pre-training scale to real-robot post-training. With no action pre-training at all, out-of-the-box success in unseen environments is 26%. Pre-training on just 12.5% of a 20k-hour slice more than doubles that to 53%. Full 20k-hour pre-training reaches 75%, and the curve is not saturating.
•
Model scale helps too but less than data. On the same 20k hours, 2B → 5B → 10B gives 61% → 75% → 79% out-of-the-box success.
•
Fine-tuning to four brand-new tasks with under 10 hours of data per task hits 75% average success, versus 40% for π0.5.
•
On simulator benchmarks the model tops the tables the paper reports: 57.6% on RoboCasa365 (previous best 46.6%) and 20.07 average on RoboDojo (previous best 13.07).
•
Suitcase packing runs end-to-end for over 10 minutes autonomously.
Reach for this line of work if you’re building a manipulation product and staring at a data-collection budget. The concrete lesson: you don’t have to fund only teleop hours on your target hardware. Handheld-gripper capture is cheaper, faster, more diverse, and, per this paper, transfers if you spend a smaller stage aligning to your embodiment. The auto-labeling trick (“caption the state change, not the intent”) sidesteps the annotation bottleneck that usually kills large trajectory corpora.
The paper promises code and checkpoints on the project page; at time of writing the artifacts aren’t released yet, so you can’t drop this in directly. The 100k-hour dataset is also not released.
In robot learning, cheap diverse data with machine-generated labels beats expensive narrow data with human labels, as long as you spend a second stage aligning to the real embodiment.
•
The out-of-the-box “unseen environment” tests are on task categories that were in post-training data; only rooms and object instances are new. Genuinely novel tasks still need the fine-tuning stage.
•
Model scaling helped much less than data scaling at the sizes tested, so the story is really about data. Whether returns hold past 100k hours is unproven.
•
Weights and the trajectory corpus aren’t released as of this paper, so external replication of the scaling claims isn’t yet possible.