RSI-Anything wraps an agent loop around a predictive world model: predictions rank candidate actions, observed outcomes update both the policy and the predictor, and only policy-model pairs that pass compatibility checks are kept for the next round.
You’ve built an agent that proposes several candidate actions per step (code patches, GUI clicks, lab reagents, reaction conditions) and picks one to execute. Picking well matters because the external feedback is expensive: a hidden test suite, a GUI rollout, a wet-lab experiment. Most self-improvement setups either keep a fixed verifier (which goes stale as the policy changes) or retrain the policy from feedback without ever upgrading the thing scoring the candidates.
The paper’s framing: once you also start updating the scorer (the world model), you get a coupled-update problem. A better policy proposes different candidates, so the predictor is now being asked about inputs it wasn’t trained on. A better predictor reranks candidates, so the policy’s training signal shifts. Improving either one alone does not guarantee the pair gets better together. The closest prior work the authors position against is CURE, which co-evolves a coder and a unit-test generator through Reinforcement Learning, and RLAnything, which jointly tunes policy, reward model, and environment.
The system keeps a pair: policy θ (how candidates are proposed) and predictor φ (the world model that forecasts each candidate’s outcome). Each cycle does roughly this:
1.
The policy proposes K candidate actions.
2.
The world model (built on JEPA-Anything, which predicts latent targets from context) scores each one. The score can be an outcome probability, a yield distribution, or a multistep rollout.
3.
One candidate is executed; the real outcome comes back.
4.
That prediction-observation pair feeds two optional updates: a policy update, and a predictor update that only fires if a reliability check says the current predictor is drifting on the candidates it’s actually seeing.
5.
Proposed new versions pass through three gates: a policy gate, a predictor gate, and a pair compatibility gate that re-evaluates the assembled (θ’, φ’) together. If the pair gate fails, both revert.
The pair gate is the load-bearing idea. It catches the case where the predictor got better on average but worse on the specific candidates the new policy prefers. Reliability is measured only within the covered candidate range; if coverage is thin, the result is “insufficient” rather than “reliable,” and no adaptation is triggered.
for k in cycles:
cands = policy.propose(state, goal)
scores = [world_model.predict(s, a) for a in cands]
chosen = rank_and_pick(cands, scores, budget)
obs = execute(chosen)
history.append((chosen, scores[chosen], obs))
theta_new = maybe_update_policy(theta, history)
phi_new = maybe_adapt_wm(phi, history) if reliability(phi, history)=='unreliable' else phi
if gate_policy(theta_new) and gate_wm(phi_new) and gate_pair(theta_new, phi_new, theta, phi):
theta, phi = theta_new, phi_new
For code repair the predictor is W1, a 3B execution-outcome model adapted from Qwen2.5-Coder-3B-Instruct with LoRA. For chemistry it’s a ReactionT5-JEPA-Anything adapter that outputs a yield distribution. For immune science it’s an action-conditioned model over cellular, tissue, and individual states.
The code-repair numbers are the cleanest ablation because the coder is held fixed and only the scorer changes.
•
On 200 CodeContests tasks (three seeds, up to three feedback rounds, four candidate patches per round), final hidden-test success went from 26.3% with a capacity-matched static verifier to 29.2% with a fixed world model, then 31.2% when the world model was also adapted from execution feedback. Online test-suite calls per task stayed near 3.72 across conditions, so the gain isn’t from spending more test budget. The paired gain over the static verifier was +2.83 pp (95% interval [+0.17, +5.50]).
•
Candidate-ranking accuracy rose from 66.5% to 72.8%, and the five-class Brier score fell from 0.488 to 0.358, linking the success gain to better calibrated ranking.
•
On MiniWoB++ checkbox-clicking, success climbed from 51% (baseline) to 99% (final), with mean actions per episode dropping 39.4% and repeated clicks going from 237 to 0. On OSWorld, RSI-Anything beat RLAnything by 5.5% relative and the initial agent by 18.4% relative.
•
On the LabHorizon F200 planning benchmark, five successive policy versions scored 21.80%, 38.03%, 41.99%, 44.56%, 46.01%, finishing roughly on par with an Actor-Simulator-Selector reference at 45.32%.
•
In two Suzuki-Miyaura reaction screens, best observed yield after 40 experiments was 96.15% versus 82.16% for Bayesian optimization at the same experiment count.
•
An immune-science case study ran the full stack end to end and ended with a prospective therapeutic hypothesis (SIRPα inhibition combined with IL-36γ-axis modulation). The paper is clear that this hypothesis is unverified and awaits experimental testing.
•
If you already run a best-of-K candidate loop with a static verifier (code agents, GUI agents, retrieval rerankers), the specific lesson worth testing is: replace the verifier with a predictor you also update from execution outcomes, and gate updates on whether the predictor’s error distribution is still representative of the candidates your current policy proposes. The code-repair ablation isolates this swap at a fixed candidate budget.
•
The pair-compatibility gate is the piece most self-improvement pipelines skip. If you’re seeing “my reward model got better on holdout but online performance dropped,” that’s the failure mode this gate is designed to catch. Worth adopting even without the world-model framing.
•
For search over expensive experiments (chemistry screens, hyperparameter sweeps with costly evaluations), the paper’s comparison with Bayesian optimization at matched budget suggests a learned yield predictor plus observed-outcome updates can beat BO when you have a reasonable pretrained backbone. The two screens shown are both Suzuki-Miyaura couplings, so transfer to other reaction families is a plausible next experiment, not an established result.
•
Code and model weights are released at GitHub and Hugging Face.
•
The code-repair paired interval ([+0.17, +5.50] pp) barely excludes zero. The headline 26.3 → 29.2 → 31.2 progression is real but not large, and the second step (world-model adaptation on top of fixed world model) is reported without its own paired interval.
•
“Domain generality” here means the same mechanism was instantiated in five different domains, not that one trained system transfers across them. Each domain has its own predictor, policy, and evaluation protocol.
•
The LabHorizon result is protocol-aligned planning on a benchmark, not physical lab execution. The immune-science hypothesis is a prospective nomination, not a validated therapy.
•
The chemistry comparison covers two Suzuki-Miyaura screens under reported experiment budgets. Bayesian optimization implementations vary; the paper cites one specific prior setup as its comparator.
•
Several referenced components (JEPA-Anything, RLAnything, Agent Genesis, the immune world model) are from concurrent work by overlapping authors. The RSI-Anything contribution is the update-and-retention protocol wrapped around them, not those components themselves.