WorldClaw generates explorable 3D worlds from a text prompt by planning globally first, then filling regions locally: an agent builds a semantic terrain foundation, then paints objects into 2D renders of that terrain and lifts them back to placed 3D meshes, keeping every asset independently editable.
You’re building a tool where a designer types “snowy valley with a Hobbit village along a river” and expects a walkable 3D scene they can open in Blender or Unreal, move a house, swap a tree, and re-render. Today you either stitch together video-generated frames (pretty from one angle, geometry falls apart when you walk around) or use Procedural Content Generation rules (structurally sound but visually repetitive and hard to steer with natural language). Systems like Infinigen give you the geometric backbone; video-lifting products like Marble give you the visual richness. Neither gives you both plus a scene you can actually edit as separate assets.
The pipeline is three stages, each run by an LLM agent that calls specialized tools. Stage 1 turns the prompt into a structured spec: regions, terrain types, object categories, spatial relations, visual style. Stage 2 builds the global terrain. The planner emits a colored 2D semantic layout map where each color is a region type (forest, dune, river). A height field is then composed per region as a weighted sum of noise components and geomorphic operators (peak, terrace, erosion), blended smoothly across region boundaries. Materials are assigned per region using either generated textures or procedural Blender node graphs, and reusable prop assets (rocks, vegetation) are scattered based on slope and elevation.
Stage 3 is the interesting part. Rather than generating 3D objects directly, WorldClaw renders the local terrain from a chosen camera as a 2D image, feeds that image plus the regional prompt to an image editor (GPT-image-2), and gets back a composite where objects have been painted onto the terrain in context. It then uses SAM 3 (Segment Anything 3) to segment individual objects out of that composite, runs each crop through SAM3D to reconstruct a mesh, and back-projects each object’s pixel location through the terrain camera ray to find where on the terrain it should sit. A per-object scale calibration checks that the reconstructed mesh, when re-rendered, occupies roughly the same image area as the segmented crop. Low-quality meshes get a second pass through Hunyuan3D using the coarse mesh as a structural prior.
spec = plan_agent(prompt) # regions, styles, objects
layout_map = gen_layout(spec) # colored 2D partition
terrain = build_heightfield(layout_map, spec)
terrain = scatter_props(terrain, spec)
for region in select_regions(spec, terrain):
img_terrain, cam = render(terrain, region)
img_comp = image_edit(img_terrain, spec[region]) # paint objects in
for obj_crop, mask in sam3_segment(img_comp):
mesh = sam3d_reconstruct(obj_crop, mask)
place = raycast_to_terrain(cam, obj_crop.center, terrain)
scene.add(calibrate_scale(mesh, place))
refine_agent(scene) # fix floating, penetration, scale
A final refinement loop uses BlenderMCP to re-render from diagnostic viewpoints, detect floating objects or terrain clipping, and locally deform the terrain or nudge objects until contact checks pass.
The prevailing move in text-to-3D-world work is to generate the whole scene in one shot: one video rollout, one panoramic lift, one big diffusion pass over a voxel grid. This paper argues the opposite. Global coherence and local richness should be produced by different mechanisms at different times, communicating through a shared structured terrain, so that each part can be inspected, edited, and re-run without redoing the world. The evidence for this stance is not a benchmark score. It is the qualitative demonstration that the outputs remain independently editable textured meshes usable in a game engine, which the video-lifting and native-3D-diffusion baselines cannot deliver.
This is a systems-and-qualitative-results report; there are no benchmark tables. The load-bearing evidence is the head-to-head comparison against five recent text-to-scene systems (SynCity, Marble, MajutsuCity, WorldGen, and a coding-agent baseline GPT-5.6 Sol) on shared medieval-village prompts:
•
Terrain expressiveness. Marble renders individual views beautifully but has no explicit region structure, and geometry degrades as the camera moves. WorldClaw produces continuous terrain with real elevation change and semantically distinct connected regions.
•
Editability. SynCity and Marble output scene-level representations with no addressable object assets. WorldClaw keeps every object as an independent textured mesh with a known placement transform.
•
Content variety. Video-lifting baselines skew heavily toward buildings and vegetation; the coding-agent baseline produces blockout-quality geometry. WorldClaw mixes settlements, animals, vehicles, and environmental props per region.
•
Range of worlds shown. The appendix demonstrates the pipeline on prompts as varied as a tropical pirate island, a canyon with tribal settlements, a desert battlefield, a snowy mountain valley with futuristic facilities, a volcanic demon lair, and a Hobbit-style village. All hold together as a single terrain rather than a collage of disconnected local scenes.
The authors are candid that no quantitative metric is reported; the claim rests on side-by-side walk-view comparisons plus instance/depth/normal renders showing the explicit geometry.
Reach for this pattern when you’re building a text-to-scene tool for game or simulation content and your users need to open the result in an engine and move things around. The concrete recipe worth stealing, even if you don’t use WorldClaw itself: render your current 3D scene to 2D, use an image editor to add objects in-context, segment them out, lift each back to 3D via image-to-3D, and re-project pixels through the camera ray to find placements. This bypasses the hard problem of native-3D multi-object generation by offloading composition to a 2D image model that already knows what a plausible village looks like.
The paper does not link a code repository or release weights. It is a technical report describing the system, with dependencies on Claude Opus 4.8 as the agent brain, plus GPT-Image-2, SAM3, SAM3D, and Hunyuan3D as tool models, all orchestrated inside Blender 5.1.1 on 4 H20 GPUs. If you want to reproduce it you are reimplementing from the description.
•
No quantitative evaluation. Every claim in the paper is qualitative. There is no user study, no geometric fidelity metric, no head-to-head score. The comparison figures use prompts adapted per method, so the setup is not strictly controlled.
•
Strong dependency on frontier proprietary models. The authors state plainly that open-source LLMs and image models failed to produce usable layouts or executable procedural code in their tests. The pipeline works because Claude Opus 4.8, GPT-Image-2, and Hunyuan3D work; swap them out and the results may not hold.
•
Latency and cost scale with scene complexity. Every object is reconstructed and refined individually with multiple render-inspect-repair loops. For a dense scene this is many minutes of tool-calling and GPU inference, and the authors flag that holistic methods will beat this pipeline on simple scenes.