OrbitQuant quantizes diffusion transformer weights and activations to low bits without any calibration data by first rotating everything into a basis where every coordinate follows the same known distribution, so one offline codebook works for every timestep, prompt, layer, and even switches from image to video unchanged.
You’ve shipped an image or video generator built on a Diffusion Transformer (DiT) backbone. Inference is slow and memory-hungry, so you reach for post-training quantization to drop weights and activations to 4 bits. The problem: diffusion activations aren’t stationary. They shift across denoising timesteps, across prompts, and between the conditional and unconditional branches of classifier-free guidance. Every existing strong baseline (SVDQuant, ViDiT-Q, AdaTSQ) handles this by collecting a calibration set and refitting per-checkpoint scales. Ship a new checkpoint, a new resolution, or switch from an image model to a video model, and you redo calibration.
OrbitQuant removes that step entirely. The same recipe drops onto FLUX.1, Z-Image-Turbo, Wan 2.1, and CogVideoX with no per-model tuning.
The core trick is a change of basis. A raw activation vector has some ugly, input-dependent distribution with channel outliers you’d need to calibrate against. If you normalize it to unit length and then apply a random orthogonal rotation, a classical result says each coordinate of the rotated vector follows one fixed marginal that depends only on the dimension d, not on the input. For d above 64, that marginal is basically a Gaussian with variance 1/d. So build ONE optimal scalar quantizer against that Gaussian offline (the standard Lloyd-Max quantizer construction), and it fits every activation you’ll ever see at that dimension.
Apply the same rotation to the weight rows offline. Since it’s orthogonal, W · rotation_transpose · rotation · x = W · x, so the rotation cancels inside each linear layer. At runtime, all that remains is one forward rotation on the activation.
A dense random rotation is O(d²) per token, which would dominate cost. OrbitQuant instead uses Randomized Permuted Block-Hadamard rotation: a uniform random permutation followed by block-diagonal Walsh-Hadamard blocks with random sign flips. It runs in O(d log h) with a fast Hadamard kernel. The paper proves that the leading uniform-random permutation is what keeps the rotated marginal well-behaved: it spreads any concentrated outlier across blocks so no single block dominates, with a variance bound that holds for any input vector.
# Offline, once per dimension d
rotation[d] = build_rpbh(d) # permutation + block Hadamard + signs
codebook[d, b] = lloyd_max(f_d, b) # 2^b centroids for the fixed marginal
# Offline, per weight matrix W with input dim d
W_rot = W @ rotation[d].T
row_norms = norm(W_rot, axis=1)
W_hat = diag(row_norms) @ nearest_centroid(W_rot / row_norms, codebook[d, bw])
# Online, per activation x (per token)
x_rot = x @ rotation[d].T
s = norm(x_rot)
x_hat = s * nearest_centroid(x_rot / (s + eps), codebook[d, ba])
The only input-dependent number at runtime is the scalar norm s per token. No scales, no zero-points, no per-channel statistics.
The prevailing approach to activation quantization is to chase the distribution: measure it per channel, per timestep, per branch, then fit scales that survive the drift. This paper shows the opposite. Stop measuring the activation distribution. Rotate the coordinate system so the distribution is fixed by geometry alone, then quantize against that. The load-bearing evidence is that a codebook derived analytically from f_d, with zero model evaluations at construction, matches or beats calibration-based methods across five different DiTs and two modalities.
The finding that makes the thesis credible is the transfer test: the identical recipe, with no per-model retuning, works on three image DiTs (FLUX.1-schnell, FLUX.1-dev, Z-Image-Turbo) and two video DiTs (Wan 2.1-1.3B, CogVideoX-2B), including huge models (Wan 14B, HunyuanVideo). Calibration-based methods can’t do this by construction.
•
On GenEval at W4A4, OrbitQuant matches or slightly exceeds full-precision Overall on FLUX.1-schnell (0.703 vs 0.664 FP16) and Z-Image-Turbo (0.767 vs 0.754 FP16), setting the state of the art among PTQ methods.
•
At W2A4 (2-bit weights, 4-bit activations), every rotation and smoothing baseline collapses to Overall ≤ 0.001 on all three image models. OrbitQuant still produces usable images, e.g. 0.604 on FLUX.1-schnell. This is the regime prior work simply couldn’t reach.
•
On VBench video at W4A4, OrbitQuant is the best PTQ on Overall Consistency on both Wan 2.1-1.3B and CogVideoX-2B.
•
The rotation ablation isolates why RPBH works: removing the leading uniform permutation (Block-RHT) drops W2A4 Overall from 0.595 to 0.558, confirming the permutation is what spreads outliers across blocks.
•
Compared to Quantization-Aware Training (QAT) baselines that fine-tune the quantized model, OrbitQuant is generally a step below the strongest (QVGen) but beats every QAT method on several VBench dimensions without a single gradient step.
Reach for this when you’re shipping a diffusion image or video generator and you don’t want a calibration pipeline in your release process. The recipe is layer-agnostic and modality-agnostic: quantize every linear projection in the transformer block (Q, K, V, output, feed-forward, cross-attention text projections). The one exception is the Adaptive Layer Normalization (AdaLN) modulation modulation projections, which produce timestep-dependent scale-and-shift and can’t be folded into neighboring weights. The paper keeps those at INT4 weight round-to-nearest, and the ablation shows dropping AdaLN below INT4 collapses the FLUX models specifically.
Artifacts: a project page is listed but the paper doesn’t link a code repo. Honest caveat on speed: their measurements use fake quantization, meaning weights and activations get dequantized to BF16 and the matmul runs in BF16. The reported latency reflects overhead, not realized low-bit speedup. They note that Lloyd-Max centroids are non-uniform, so integer tensor cores can’t consume the codes directly. A fused lookup-table GEMM kernel is future work.
When your data distribution shifts on you, don’t calibrate harder. Change coordinates until the distribution stops shifting. OrbitQuant works because rotating a normalized vector by a random orthogonal matrix produces coordinates whose distribution depends only on the dimension. That turns a moving-target calibration problem into a fixed table lookup you build once.
•
Speedup is currently theoretical. All numbers are under fake quantization in BF16. The actual wall-clock win from low-bit compute waits on a fused non-uniform-codebook GEMM kernel the authors haven’t built yet.
•
AdaLN is a load-bearing exception. The rotation-cancellation trick doesn’t apply to timestep-dependent modulation, so those projections stay at INT4 RTN. Pushing them lower collapses FLUX models. If your architecture leans heavily on similar dynamic modulations, the recipe won’t cover them.
•
Z-Image-Turbo shows the ceiling. At W2A3 even OrbitQuant degrades sharply on it, and at W2A4 seed variance on its Overall score is ±0.072. Calibration-free codebooks have a bit-width floor that depends on the model, and you’ll only find it by trying.