Meshy T2 generates artist-style 3D meshes from a single image by running Flow matching over an unordered set of per-vertex latent tokens, jointly producing coordinates, edges, and face orientation in one parallel pass in under 10 seconds with a user-set face budget.
You’re shipping a product that turns a reference photo into a game-ready 3D asset. Today’s best pipelines give you a dense soup of hundreds of thousands of near-identical triangles from Marching Cubes on an implicit field, and your artists have to retopologize by hand before the mesh enters an engine. The alternative, sequence-model approaches following MeshGPT, serialize the mesh into a long token stream and take minutes per asset, with the risk that a single sampling slip cracks the surface. Meshy T2 targets the case where you want compact, clean, artist-shaped topology directly from an image, at interactive speed, with an explicit polygon budget you can hand to a level-of-detail system.
The core move is to treat a mesh as an unordered set of vertices and generate all of them in parallel, rather than reading the mesh out as a 1D sequence. A Mesh VAE encodes each ground-truth vertex to one latent token, and its decoder recovers three things per token in a single pass: a continuous 3D position, an edge embedding, and a face embedding. Two vertices are connected when a scored quantity between their embeddings crosses zero, and each vertex’s local fan of triangles is recovered by predicting which neighbor follows which as you walk around it, extending the SpaceMesh halfedge scheme with a NULL element so open boundaries work too. Crucially, coordinates are never snapped to a grid and coincident vertices are never merged, so artist-authored topology survives round-trip.
Generation itself is two Flow matching stages. Stage one takes the input image, encoded by a frozen DINOv3 backbone, and generates a coarse 64³ occupancy grid inside the latent space of a pretrained voxel Variational Autoencoder. Stage two conditions on the image, that occupancy scaffold, and a requested vertex count, and denoises the per-vertex latent set with a Diffusion Transformer (DiT) backbone. Because the latent tokens are unordered but the transformer needs spatial coordinates for Rotary Position Embedding (RoPE), each latent slot is assigned a fixed position from a Sobol point set via an Optimal transport assignment matching to the true vertex coordinates during training. The vertex count is a soft budget: training pads the set with dummy tokens carrying a -1 “existence” channel, and at inference any generated token with a negative existence value is discarded.
# Inference sketch
img_tokens = dinov3(image)
voxel_latent = flow1.sample(cond=img_tokens) # 16^3 grid
occ = voxel_vae.decode(voxel_latent) > 0.5 # 64^3 scaffold
slots = sobol_points(n=N_budget) # fixed positions
z = flow2.sample(cond=(img_tokens, voxel_latent, N_budget),
rope_pos=slots)
real = z[z.existence > 0]
mesh = mesh_vae.decode(real) # verts+edges+faces
The prevailing move in mesh generation has been to serialize the mesh into a token sequence and let a large autoregressive model sample it, then chase efficiency by inventing smarter traversal orders. This paper argues the opposite. A mesh is inherently unordered, so generate all vertices in parallel as a set and predict connectivity as an explicit relation between tokens, not as an emergent property of a long sample. The evidence that this is the load-bearing choice is the position-encoding ablation on the Mesh VAE, not the headline latency win.
•
The Optimal transport assignment assignment of Sobol positions to latent tokens is what makes the set formulation actually train. Swapping it for index-order positions raises validation Chamfer Distance from 0.0070 to 0.0127 and Hausdorff Distance from 0.0225 to 0.0783, and roughly doubles the non-manifold edge ratio. Even pairing Sobol points to vertices by a cheaper Morton order sort leaves Chamfer 23% and Hausdorff 58% worse than full transport. Without OT, the model has to average over every equally-valid permutation of the vertex latents, and convergence collapses.
•
On the retopology benchmark of 115 assets (targeting ~4,000-face outputs), Meshy T2 reaches CD 0.020, HD 0.044, NC 0.860, ahead of the nearest baseline MeshAnything V2 at CD 0.037. Median wall-clock is 3 seconds per asset, versus 49 s for MeshAnything V2, 94 s for MeshFlow, and 210 s for BPT.
•
On image-to-mesh, Meshy T2 finishes end-to-end in a median 6 s at 100% success, and leads the field on DINOv2 Fréchet Distance (2312 vs 2442 for the nearest diffusion competitor). Inception FD is close to the pack, so the gain shows up more in structural-semantic alignment than in pixel-level render similarity.
•
Practical robustness: 100% completion within a 20-minute budget for Meshy T2, versus 28.7% for DeepMesh and 49.6% for Mesh-Silksong, both of which frequently time out or emit unusable outputs.
Reach for this when you’re building an asset pipeline that ingests a reference photo and needs to hand a game engine or DCC tool a mesh under a fixed polygon budget. Because connectivity is generated explicitly, a multi-part input (say, a character plus a held prop) comes back as separated connected components in one shot, so you skip the usual segmentation-plus-stitching stage. Face-count control is not exact but range-bounded through the Euler relation F≈2V, which is enough for level-of-detail tiers.
The paper is a technical report from Meshy AI, and it does not mention a code, weight, or dataset release. Evaluation is on an in-house benchmark of 115 assets with ground truth produced by the company’s own prior product Meshy 6, so external replication would require rebuilding both the training corpus and the evaluation set.
Treat a mesh as an unordered set with explicit connectivity, and the whole thing generates in parallel. The trick that makes the set formulation trainable is giving every latent slot a fixed spatial address via optimal transport to a low-discrepancy point set, so the model stops wasting capacity on permutation ambiguity and starts learning geometry.
•
All numbers come from the authors’ own 115-asset benchmark, with ground truth generated by their prior commercial system and no independent test set. The apparent gap over baselines could partly reflect distributional alignment with that ground truth.
•
Vertex-count control is a range, not an exact target: you get somewhere between N/(1+p) and N vertices. If your downstream pipeline requires a strict polygon count, you’ll still need a decimation or subdivision post-step.
•
The face-prediction scheme assumes manifold meshes, and non-manifold training data is repaired by splitting edges beforehand. Highly irregular or intentionally non-manifold source geometry (CAD assemblies, hair cards, decals) is outside the regime the Mesh VAE was designed for, and the authors flag topological robustness on irregular meshes as future work.