ZipTok3D reconstructs a full 3D object from as few as one latent token by training an ordered Nested dropout prefix and unrolling a weight-shared decoder for several passes, matching a 32-token baseline while using 32× fewer tokens on ShapeNet.
If you’re shipping a text-to-3D or asset-generation pipeline, the length of the latent sequence your tokenizer emits directly sets the cost of the diffusion or autoregressive model that runs on top. Cut tokens from 32 to 2 and your stage-2 generator becomes dramatically cheaper. The catch: existing global 3D tokenizers like VecSet and COD-VAE fall apart at very short lengths because they were trained at a fixed budget, so no individual token is responsible for carrying the object-wide shape. ZipTok3D targets exactly this regime: how do you keep geometric fidelity when the latent budget collapses to a handful of tokens?
There are two coupled ideas. First, the encoder produces up to 128 latent tokens, but during training the decoder only sees a random prefix of length K, sampled from {1, 2, 4, …, 128}. Every prefix must reconstruct the full shape. This is Nested dropout: shorter prefixes are always contained in longer ones, so the leading tokens are forced to carry object-wide geometry and later tokens add detail. Second, since one or two tokens can’t be unfolded into a detailed 3D field in a single forward pass, the decoder applies the same six-layer Transformer block L times, each iteration refining a Triplane representation feature grid while re-attending to the fixed prefix. This is a Universal Transformer-style shared recurrence, so depth grows without adding parameters. An intermediate iteration is also supervised, both directly against ground-truth occupancy and via Intra-loop self-distillation from the final iteration, so every refinement step produces a valid reconstruction rather than an unconstrained intermediate.
Z = encoder(surface_points) # up to 128 tokens
K = sample_from({1,2,4,8,16,32,64,128})
Z_prefix = Z[:K] # nested dropout
H, R = selection_block(triplane_tokens, Z_prefix)
for l in range(L): # shared block, L passes
H = shared_transformer(H, Z_prefix)
T = restore(H, R) # scatter back to triplane
occ = mlp(sample_triplane(T, query_points))
The usual assumption in global 3D tokenizers is that if you want a short sequence, you retrain the tokenizer at that shorter budget and hope the decoder redistributes information. This paper shows the opposite. Train once with an ordered prefix objective so the leading tokens are the ones that must reconstruct the object, then compensate for the encoding’s compactness on the decoder side by iterating a shared block rather than making the decoder deeper. The load-bearing evidence isn’t the headline reconstruction score. It’s the ablation showing that adding nested-prefix training to the original single-pass decoder already jumps IoU from 77.7 to 92.3 at K=2.
•
The prefix objective alone recovers most of the gap. On ShapeNet at K=2, swapping COD-VAE’s fixed-budget training for nested-dropout (same decoder architecture) lifts IoU from 77.7 to 92.3 and F1 from 80.9 to 95.8. This isolates the representation-side contribution from the decoder-side contribution.
•
Shared iterative refinement closes the rest. Going from one decoder pass to five (same six-layer block, no new parameters) lifts IoU further to 96.6, and adding intermediate supervision reaches 96.9. Trainable decoder parameters actually drop from 39.3M to 23.5M vs the baseline.
•
Headline reconstruction. One ZipTok3D token matches 32-token COD-VAE on ShapeNet within 0.3 IoU points (96.8 vs 97.1) with identical CD and F1 at reported precision. Four tokens match or slightly beat 32-token COD-VAE on the more diverse TRELLIS split. A paired object-level bootstrap on TRELLIS says the CD and F1 differences favor ZipTok3D with 95% intervals excluding zero, while IoU is a statistical tie.
•
The two knobs are complementary. Longer prefixes help most when you only do one decoder pass; extra refinement helps most at K=1 or K=4. Both datasets saturate around L=5.
•
Downstream generation works from two tokens. A causal stage-2 VAE plus an EDM diffusion model trained on a $2\times 32$ latent produces class-conditioned ShapeNet samples with distribution metrics close to a 32-token COD-VAE stage-2, at higher sampling throughput.
•
Throughput caveat. End-to-end reconstruction throughput is lower than single-pass COD-VAE (72 vs 82 shapes/s) because the dense $128^3$ occupancy query dominates and five refinement passes add cost. The win is latent-sequence length, which matters for the stage-2 generator, not for the tokenizer decoder itself.
Reach for this if you’re building a text-to-3D or class-conditional 3D generator and the stage-2 diffusion or autoregressive model over latent tokens is your bottleneck. A 2-token latent means your stage-2 model has a trivial sequence length, so denoising steps and memory drop hard. The stage-1 decoder gets a bit slower per shape from the shared refinement passes, but that’s amortized against a much cheaper generator. The design also gives you a single checkpoint that operates at any prefix length from 1 to 128 with any refinement depth from 1 to 6, so you can trade quality for latency at inference without retraining.
The paper describes an anonymous code release but the manuscript doesn’t include a public repository URL. Artifacts referenced are ShapeNet (ShapeNetCore-v2, 55 categories, using the 3DShape2VecSet split) and the TRELLIS-500K asset pool, both preprocessed via the VecSet watertight pipeline. No new dataset is introduced.
•
Decoder wall-clock is worse, not better. Five shared passes plus the dense $128^3$ occupancy query mean end-to-end reconstruction throughput is below single-pass baselines. The win is only realized when a downstream generator consumes the short latent sequence.
•
The reported statistical test is object-level, not run-level. Every model is trained once with a single seed. The bootstrap intervals reflect variation across evaluation shapes, not across independent training runs, so small headline gaps could be seed noise.
•
Adaptive budgeting is oracle-only. The post-hoc diagnostic that picks the shortest (K, L) matching a 32-token baseline uses ground-truth metrics; no learned budget predictor is proposed, and on TRELLIS 32% of shapes have no operating point in the grid that matches the baseline on all three metrics.