Mental World Modeling (MWM) argues that world models predicting human behavior must carry a coupled physical-and-mental state, and shows that stripping either channel drops accuracy by 12–16 F1 points across eight LLMs, with the biggest hits on interpersonal scenes.
You’re building an assistant that decides when to interrupt a user, hand back control, or ask before acting. Your current stack tracks the physical scene: what’s on screen, what files exist, which step just failed. It doesn’t track what the user thinks is happening, whether they noticed your last action, or whether an action they’d accept from a peer feels intrusive from a bot. Existing world models in AI (representation models like Dreamer, video generators like Sora, 3D world models) are all about physical dynamics. Prior work on the mental side, mostly Theory of Mind benchmarks, treats belief inference as isolated Q&A rather than as state you carry forward through actions. This paper is the attempt to fuse the two.
The intuition: represent a scene as a third-person state with two halves, physical (objects, positions, who’s where) and mental (each agent’s beliefs, goals, emotions, norms, relationships). Then render a first-person view for a specific target agent that hides what they can’t see or know. When you evaluate a candidate action, simulate how it changes both halves in parallel, not just the physical scene. Formally the authors dress this up as a POMDP where the hidden state now includes mental variables, and an action is a pair: a physical carrier (say, speaking) and a mental content (reassuring, deceiving, refusing).
They implement this in Mentis, a training-free pipeline where each stage is a separate LLM call producing inspectable JSON. Pseudo-code:
state = parse_scene(scene) # physical + mental JSON
obs = render_observation(state, target) # hide what target can't access
for option in options: # 6 candidate actions
act = decompose(option) # physical carrier + mental content
next_phy = transit_physical(state, act)
next_ment = transit_mental(state, act, next_phy)
score[option] = evaluate(state, obs, act, next_phy, next_ment)
return argmax(score) # deterministic, outside the LLM
Every intermediate artifact is logged, so a wrong final answer can be blamed on a specific stage.
The default in AI world modeling is to make the object of prediction physical: pixels, geometry, object positions, affordances. Agents in the scene are just moving objects. This paper argues the opposite. For any decision where a human is the actor, the mental variables (what they believe, want, feel, and consider permissible) are state, not narration, and they belong inside the transition function alongside the physical dynamics. The load-bearing evidence isn’t the headline benchmark lift; it’s that removing the mental channel costs more than any test-time tweak recovers, on every model tested.
Across eight LLMs from OpenAI and Anthropic, on the paper’s own Menti-Bench (448 situated-decision items, text plus image plus sounding video), the full pipeline beats direct answering by +21 to +28 F1 and beats six-sample self-consistency by ~10 F1 on average. The finding that carries the thesis is the ablation table: removing the mental channel costs 12.1 F1 on average, removing the physical channel costs 16.5, and predicting the two transitions independently instead of coupled costs 6.4. All three losses hold for every model. Even the strongest model with six-sample Self-consistency (83.6) loses to the weakest model running full MWM (84.9), so more sampling does not substitute for structure.
Oracle interventions localize the remaining gap to human performance (98.5 vs. best-config 90.7). Handing the pipeline gold transitions recovers +3.5, gold states +2.8, gold observations +1.7, gold action-decomposition only +0.7. Transition simulation is the binding bottleneck, not scene parsing or option interpretation. Gains concentrate on interpersonal scenes (+26.4 vs. +14.0 on object/resource scenes), which is what the theory predicts.
Reach for this framing when you’re building an agent that decides whether to act, ask, or defer around a human, a tutoring system, a care or scheduling assistant, or an NPC in an interactive world. The concrete transferable move is small: before scoring candidate actions, force the model to write down two things as separate JSON blobs, what the user currently believes and wants, and what they can actually perceive right now. Then score each action against both a physical and a mental successor state. The Chain-of-Thought and self-consistency baselines in the paper are the honest comparison points, and structured state beats them well before any branch simulation happens.
Artifacts: the project homepage hosts Menti-Bench (448 records with full process-level gold: state, observation, per-option successor states, final action), released research-only with a request not to train on the gold blocks. Mentis itself is described as an inspectable baseline with per-stage JSON schemas; the paper walks through the module contracts in enough detail to reimplement.
If your agent acts on a person, model what the person thinks is happening as state, not as narration inside a prompt. The payoff shows up exactly where you’d expect: interpersonal decisions where the physical scene under-determines the right action. It also shows up largest on weaker base models, so this is a structure-compensates-for-scale result, not a frontier-only trick.
•
Menti-Bench is 448 items, English, authored by the paper’s team, and the image and video subsets are 100 and 28 records; treat modality-specific numbers as directional, and expect the norm judgments to reflect one cultural frame.
•
The pipeline is training-free and prompt-driven, so “the mental channel matters” is demonstrated with LLMs that already carry strong social priors from pretraining; a smaller or non-instruction-tuned model may not fill the mental-state schema usefully at all.
•
Every mental variable is a hypothesis the LLM invented from context. The paper flags but does not solve the calibration problem, so deploying this on real users without uncertainty handling turns confident belief-attribution into a manipulation and privacy risk.