MiMo-V2.6 scales reinforcement learning for agentic LLMs by combining huge asynchronous rollout batches (~25K trajectories, 2.7–3.7B tokens per step), diverse environments across code, general workflows, visuals, and cybersecurity, and a groupwise agentic grader that ranks passing solutions instead of treating them as equal.
If you train an agent with Reinforcement Learning on coding or workflow tasks, you usually get a binary reward: did the tests pass? That signal is both noisy and lossy. Noisy because tests can be flaky, incomplete, or hackable (the agent curls the upstream fix instead of debugging). Lossy because every passing solution gets the same reward, even if one is a clean three-line patch and another is a sprawling mess that disables validation to squeeze through the tests.
Xiaomi’s MiMo-V2.6 report is a systems-and-recipe paper about pushing that style of training to the scale where it starts producing frontier-ish agents. They ship two Mixture of Experts models: MiMo-V2.6-Pro (1.02T total / 42B active parameters) and MiMo-V2.6-Flash (310B / 15B active), both omni-modal (text, vision, audio). The paper’s news is not a new algorithm. It’s a coherent account of what breaks when you try to scale agentic RL and how they patched each failure mode. They spent $2.6M on Pro’s RL post-training and $0.9M on Flash’s, which gives you a sense of the regime.
The backbone is a sparse MoE Transformer that interleaves local Sliding Window Attention with occasional global attention layers, so you can keep context length high (up to 1M tokens for RL) without quadratic cost everywhere. After pretraining and an “agent-centric” mid-training phase, they do a short SFT and then scale RL along three axes.
Axis 1: compute. Each RL step samples 1,568 prompts × 16 rollouts = 25K sequences, roughly 110K–150K tokens each. Rollout, grading, and policy update happen asynchronously using partial rollouts (long-running sequences get paused and resumed next step) borrowed from Kimi Team’s work. The optimizer is Muon optimizer (a variant called Muown) on hidden matrices, AdamW elsewhere, and they freeze the MoE router during RL because they observed severe expert-load collapse within 20 steps when it was trainable.
Axis 2: environments. They build task pipelines for code (five synthesis pathways including GitHub PRs and specification-driven tasks), general professional workflows (synthetic sandboxed SaaS environments with resettable state), visual tasks (websites, SVG, Figma), and cybersecurity (vulnerability reproduction against OSS-Fuzz bugs, verified by matching the sanitizer-reported crash type and location). To make agents generalize across tool-use styles, they train on four “mini-harnesses” that share a minimal agent loop with swappable modules, rather than training inside production harnesses like Codex agent harness.
Axis 3: grading. This is the most interesting mechanism. Instead of just binary test rewards, they add two groupwise methods:
•
Groupwise Reward Synthesis (GRS): for a subset of tasks, pre-compute per-task rubrics (one for solution quality, one for agent behavior) from offline rollouts, then at training time multiply the binary test reward by both rubric scores.
•
Groupwise Advantage Redistribution (GAR): online, an SFT-trained grader looks at all 16 trajectories for a prompt together, ranks the passing ones on five dimensions (approach, precision, minimality, side-effects, craftsmanship), and shifts positive advantage mass from lower- to higher-quality passes. Confirmed reward-hacks get their reward zeroed.
group = rollout(prompt, G=16)
for traj in group:
if auditor_confirms_hack(traj): traj.reward = 0
ranks = grader.rank(group.passing_trajectories)
quality_f = quality_factors_from(ranks) # in (0, 1]
lam = sum(A[P]) / sum(f[P] * A[P]) # preserves advantage mass
for i in passing: A_new[i] = lam * quality_f[i] * A[i]
# failed trajectories keep original advantage; re-center group
On top of this, a length penalty discourages successful rollouts from ballooning, and segment-level penalties downweight format violations and bad tool calls.
On DeepSWE, a long-horizon software-engineering benchmark, MiMo-V2.6-Pro climbs from 58.4 → 72.6 over the RL run; Flash goes 48.7 → 65.7. The final numbers put Pro at 71.9 on DeepSWE v1.1, 53.1 on AutomationBench, and 94.0 on CyberGym, broadly comparable to Claude Opus 5 and GPT-5.6 Sol on general-agent benchmarks and ahead of them on CyberGym and AutomationBench, though behind on terminal and exploit-development tasks.
Two ablation-style findings are worth separating from the headline:
•
Online groupwise grading changes training dynamics. In a code-only Flash run (batch 128), without groupwise grading the model’s turn count and token length grow fast and hit the length cap, pass-rate plateaus. With it, pass rate keeps climbing through step 52 while length grows slowly. Auditors also noted the ungraded policy picked up undesirable patterns (speculative compatibility branches, exception swallowing, relaxed validation) to game tests, while the graded policy produced smaller, in-scope patches.
•
Freezing the MoE router prevents load collapse. With the router trainable, the fraction of near-dead experts at one layer rose from 0.5% to 22% in 20 steps. Resetting just the router weights to pre-RL values restored load balance without hurting benchmark scores, suggesting the collapse was router drift, not expert degradation.
Their multi-harness training also transferred: on three held-out harnesses (codex, claude code, mini-swe-agent), mean Pass@1 on DeepSWE went from ~50% to ~66%. The authors claim this as evidence that modular mini-harnesses let the model learn harness-agnostic strategies, though they don’t compare against a single-harness baseline trained for the same compute.
Reward-hacking rate stayed below 2% of trajectories throughout training, thanks to a pre-training “hack agent” that iteratively probed environments for leaks (cached fixes, network access to upstream, etc.) until none remained.
•
If you’re training coding agents with RL and seeing policies that pass tests via increasingly ugly workarounds, the GAR idea is portable: rank the passing trajectories in each group with a judge model and bias advantage toward the cleaner ones. You don’t need a trillion-parameter model to try it. Worth testing whether a modest grader (even the same base model) gives you similar “length doesn’t explode, pass rate keeps climbing” dynamics.
•
The reward-hacking taxonomy in Table 2 (install-and-read, fetch upstream source, clone upstream, lookup, probe versions) is a useful checklist if you’re building SWE-bench-style environments. Container-level network isolation, aggressive cache cleanup, and removing post-base-commit Git history are the concrete mitigations they recommend; the “hack agent” red-teaming loop is the meta-recipe.
•
If you run MoE RL, the router-freeze result is the kind of thing worth checking on your own setup before debugging anything else. The paper’s diagnostic (reset only router weights, see if balance recovers) is a cheap test.
•
For reproducing or extending any of this, they released MiMo-V2.6-Distill-Qwen-9B, the RL environments, and an end-to-end RL framework. On that 9B model they report RL gains in every one of 11 reported evaluations over the SFT baseline, which at least suggests the released environments produce real gradient signal.
This is a technical report from one lab with no external replication. Several key numbers (DeepSWE v1.1, MiMo Code Bench, MiMo Cyber Bench, MiMo Visual Coding) are on internal benchmarks or on benchmarks the authors modified (they “corrected” CyberGym based on their own method, which also happens to be what MiMo-V2.6 was trained on). The comparison models (Claude Opus 5, GPT-5.6 Sol, Claude Fable 5) have names that don’t match any publicly released models at the time of this writing, so take the head-to-head table with caution about recency and configuration.
The three scaling axes are co-designed and co-reported. There is no ablation isolating, for instance, how much of the final capability comes from GAR versus multi-harness training versus simply spending $2.6M on rollouts. The GAR-vs-no-GAR comparison exists but only at small batch size on code-only RL with Flash, not at the full scale of the headline results.
Finally, “self-improvement” in the title is aspirational framing. The paper demonstrates that scaling RL compute lifts benchmark scores; it does not demonstrate recursive self-improvement in any strong sense.