MVCHead generates view-consistent 3D Gaussian head avatars from a single forward pass, trained only on unposed 2D face photos, by rewarding agreement between its own renderings from different camera angles instead of synthesizing intermediate views.
You’re building an avatar pipeline for a VR app or a synthetic-identity generator for content. The current playbook either demands a studio capture rig with dozens of synchronized cameras, or uses a Multi-view diffusion stage to hallucinate side views from one photo before lifting them into a 3D representation. Both are expensive: studio data doesn’t scale to thousands of identities, and per-asset view synthesis is slow and leaks inconsistencies (the left ear in view 3 doesn’t match the left ear in view 7). This paper asks whether you can skip both crutches and still get heads that look like the same person from every angle.
The model maps a random latent vector to 240K anisotropic 3D Gaussian Splatting (3DGS) primitives that compose a head. Training only sees ordinary 2D face photos with no multi-view pairs. Two pieces do the work.
First, a stack of HiSS blocks builds the head coarse-to-fine. Each block predicts new Gaussians as offsets anchored to the previous level’s Gaussians, so detail refines structure rather than drifting. Inside each block, tokens pass through self-attention (for global identity) and through a state-space scan called HiBiSS. The intuition for HiBiSS: when you rotate a head left or right (yaw), points on the face move mostly horizontally in the image; when you tilt up or down (pitch), they move mostly vertically. So the recurrence is run along rows and columns of a token grid, bidirectionally, matching the axes where view-to-view drift actually shows up. This adapts Mamba scans to a 2D, non-causal setting.
Second, an SE(3) Multi-view Critic supplies the consistency signal. After generating an avatar, the system renders K views from known cameras, and a separate network (a ViT augmented with Geometric Transform Attention (GTA)) scores whether those K renders look like they came from one underlying 3D head. The critic is trained as a binary classifier: positive sets are K renders of the same latent, negatives mix renders from different latents under the same cameras. The generator then maximizes this score.
for step in training:
z = sample_latent()
gaussians = HiSS_stack(z) # coarse-to-fine, HiBiSS inside
views = [render(gaussians, T_k) for T_k in cameras]
L_mvc = -critic(views, cameras) # reward consistent self-renders
L_adv = adversarial_loss(views) # texture realism per view
L_reg = knn_spacing + anchor_drift # keep Gaussians well-behaved
backprop(L_mvc + L_adv + L_reg)
No intermediate view diffusion, no 3D ground truth, no FLAME mesh required.
The prevailing fix for multi-view consistency without studio data is to first synthesize side views with a diffusion model, then reconstruct 3D from those views. This paper shows the opposite. Train a critic that judges the model’s own renderings for cross-view agreement, and let gradients through that critic shape the 3D representation directly. The model never needs to see a real multi-view pair, because the consistency signal lives in whether its self-renders mutually agree, not in matching held-out cameras. The ablation removing the critic and the one swapping HiBiSS for a standard one-way scan both degrade MEt3R, which is the evidence that consistency comes from these two pieces and not just from generic adversarial pressure.
The load-bearing finding is the ablation: dropping the multi-view critic loss degrades both Fréchet Inception Distance (FID) and MEt3R, and replacing HiBiSS with a standard unidirectional scan also degrades consistency. That confirms the architecture-plus-critic combo is doing the consistency work, not the GAN loss alone. Removing the entire state-space component (keeping only attention) also drops MVC, so attention by itself isn’t enough.
•
On FFHQ and FFHQ-C at 512×512, MVCHead reports the best FID and FID-3D among feed-forward 3D Gaussian head generators trained without multi-view supervision.
•
On multi-view consistency metrics, it beats prior feed-forward baselines on Chamfer Distance, texture metrics (cPSNR, cSSIM, cLPIPS computed between two independent 3DGS reconstructions of the same identity), and MEt3R. Depth error is comparable rather than better.
•
The authors release FaceGS-10K, 10,000 synthesized 3D Gaussian heads (240K Gaussians each plus 24 renderings) filtered for consistency and frontal realism, as a stand-in for the scarce real 3D head data the field currently lacks.
Reach for this when you need many synthetic 3D head identities for a VR product, a privacy-preserving training corpus, or a content pipeline, and you don’t have a capture stage. The recipe: train on a 2D face dataset you already have, render K views per sample during training, score them with a learned consistency critic, and skip the intermediate-view diffusion step entirely. Single forward pass at inference means you can generate avatars at interactive rates rather than running a multi-step diffusion per asset.
Code and the project page are released. The FaceGS-10K dataset is the more immediately reusable artifact for downstream teams: it ships raw Gaussian attributes that go straight into any 3DGS rasterizer, without needing a FLAME registration or a parametric head model in the loop. Useful as supervision for reconstruction models or as a synthetic-identity bank.
Self-renders are a free multi-view dataset, if you train a critic to grade their agreement. You don’t need ground-truth other-angle photos to enforce 3D consistency; you need a network that can tell whether your own renderings from different cameras are mutually plausible, and gradients through that network do the rest.
•
Coverage is front and side views only. The model can’t produce the back of a head, so it’s not a drop-in for full 360° avatars.
•
The consistency critic learns from fairly easy negatives (different identities, same cameras). Harder negatives, like geometrically perturbed renders of the same identity, are left as future work, meaning the current consistency signal may miss subtle local geometric errors.
•
All geometric structure is learned from 2D images alone. There’s no explicit shape prior (symmetry, parametric face model), so depth accuracy is only comparable to baselines rather than better, and unusual accessories or hairstyles may still bend geometry in implausible ways.