SoloQ makes 4-bit quantization of diffusion language models work without any calibration data by rotating weights and activations into a basis where the coordinate distribution is predictable and the same across masking states, cutting peak memory by up to 2.61× and inference latency by up to 2.24×.
Diffusion Language Model (dLLM) are a newer alternative to standard left-to-right text generation: instead of emitting one token at a time, they start from a sequence full of [MASK] tokens and iteratively unmask them over many denoising steps, using bidirectional context. Models like LLaDA and Dream show this works at 7B+ scale, and newer block-diffusion variants (Fast-dLLM v2, Nemotron-Labs-Diffusion) denoise one block at a time while caching KV states from already-committed blocks, much like autoregressive decoding.
The practical pain point: each denoising step is a full forward pass, so compute and memory costs stack up fast. The natural fix is low-bit Post-Training Quantization, which compresses weights, activations, and KV-cache to 4 bits. But standard PTQ for autoregressive LLMs assumes activation distributions you can characterize by running a small calibration set through the model once. In a dLLM, activations shift substantially depending on how many tokens are currently masked and which denoising step you’re on. The authors show that for a representative rotation-based method (QuaRot with GPTQ), swapping the calibration set from one benchmark to another changes downstream accuracy noticeably. Calibration-based methods specific to dLLMs exist (DLLMQuant, STaR-Quant), but they still depend on picking the right calibration data.
The core trick comes from a known fact: if you take any vector and apply a random rotation to it, the resulting coordinates are distributed on the unit sphere in a way that only depends on the dimension, not on what the original vector looked like. So instead of chasing a moving activation distribution with calibration data, SoloQ rotates weights and activations into this predictable “spherical” space, then designs a quantizer analytically for that known distribution. No data needed.
The practical rotation is called K-RPBH rotation. It builds on an earlier rotation, Randomized Permuted Block-Hadamard rotation, which does fast structured mixing by chopping the vector into blocks, applying a sign-flipped Hadamard transform inside each block, and shuffling. That’s cheap but leaves residual imbalance across blocks, especially when there are many of them (Dream’s feed-forward dimension is 18944 = 512 × 37 blocks). K-RPBH adds a small random orthogonal mixer that acts across corresponding coordinates in different blocks, redistributing leftover imbalance. The authors prove that averaged over the mixer, each output coordinate has variance exactly 1/d, matching the ideal spherical target.
Once the representation is in this predictable basis, SoloQ offers two quantizers on the same rotated tensor. SoloQ-C uses distribution-matched codebooks: a Lloyd–Max scalar quantizer for activations and an E8P lattice codebook for weights, both tuned to the known post-rotation distribution. SoloQ-N uses NVFP4, the hardware-native 4-bit float format on recent NVIDIA GPUs, with its standard tensor- and block-wise scales. Both paths get a lightweight scale-only bias correction: quantizing a vector shrinks its projection along the original direction, and the correction rescales to exactly restore that projection, absorbed into existing per-row scales at no bit cost.
For block-diffusion models, which keep a persistent KV cache across committed blocks, SoloQ quantizes KV states only at commit time. The actively denoising block stays in full precision, so repeated denoising steps don’t accumulate quantization error on in-flight tokens. Keys use per-channel asymmetric quantization in the style of KIVI; values go through the same K-RPBH path as activations.
# Offline
for layer in target_layers:
W_rot = W @ Pi.T # fold K-RPBH into weights
W_q = quantize_weight(W_rot) # SoloQ-C or SoloQ-N
# Online, per forward pass
x_rot = Pi @ x # rotate activation
x_q = quantize_act(x_rot) # same codebook/NVFP4
y = W_q @ x_q # low-bit matmul
# Block-diffusion only: on block commit
K_q = asym_per_channel_quant(K_block)
V_q = quantize_act(Pi @ V_block)
kv_cache.append((K_q, V_q))
At W4A4 (4-bit weights and 4-bit activations) on three full-sequence dLLMs across nine benchmarks (MMLU, GSM8K, HellaSwag, HumanEval, etc.), SoloQ without any calibration either matches or beats calibration-based baselines. On LLaDA-1.5, SoloQ-N averages 68.48% versus 66.93% for the best calibration-based competitor (STaR-Quant) and 69.23% for the full-precision model. On LLaDA-Base, SoloQ-C averages 57.74% versus 57.07% for STaR-Quant. On Dream-7B the picture is more mixed: SoloQ-N reaches 62.53% while STaR-Quant hits 63.59%, so SoloQ is competitive but not strictly best there.
For block-diffusion models, where KV-cache quantization matters, the gap gets wider. On Fast-dLLM v2 at W4A4, naive round-to-nearest and AWQ collapse to near-zero on code and math tasks; QuaRot isn’t even applicable because the model’s hidden dimension isn’t supported by its rotation. SoloQ-C holds 57.00% average versus 60.35% full precision. On Nemotron-Labs-Diffusion, SoloQ-N reaches 70.76% versus 78.67% full precision, while QuaRot+GPTQ sits at 59.39%. Adding 4-bit KV cache (W4A4KV4) costs only a point or less on average.
Ablations isolate the pieces: dropping either the permutation or the cross-block mixer from K-RPBH roughly doubles the KL divergence to the target distribution. The scale-only bias correction reliably reduces KL between full-precision and quantized logits across all input masking ratios on LLaDA-1.5. On end-to-end inference measured against BF16, SoloQ-N delivers 2.31×–2.61× memory reduction and 1.94×–2.24× speedup across five dLLMs on H200 and RTX Pro 6000 hardware. Code is on GitHub.
If you’re deploying a diffusion LLM and want to cut memory or latency, SoloQ gives you a path that doesn’t require you to assemble a representative calibration set. That matters more than it sounds: picking calibration data for a dLLM is genuinely hard because activations drift across denoising steps and mask ratios, and the paper shows calibration choice changes which benchmark wins.
If you’re on NVIDIA hardware with native NVFP4 support (RTX Pro 6000, Blackwell-class), SoloQ-N is the obvious variant to try: it rides the hardware FP4 path and gets the full speedup. On older GPUs without NVFP4, SoloQ-C runs on 8-bit tensor cores and gets smaller but still real speedups.
For block-diffusion models specifically, this is one of the few quantization paths that explicitly handles the persistent KV cache, and the commit-time design means you don’t pay re-quantization cost on every denoising step.
A deployment caveat worth testing: activation rotation is applied online and adds measurable latency per matmul. You can “fold” it into adjacent weights offline for models with static RMSNorm gains (LLaDA, Fast-dLLM v2), but the appendix shows this folded topology degrades accuracy noticeably on Fast-dLLM v2 (57.00% → 50.94% average). Treat folding as a speed-vs-accuracy knob, not a free optimization.
SoloQ isn’t strictly best everywhere: on Dream-7B, calibration-based STaR-Quant edges it out by about a point on average at W4A4. The authors don’t claim universal dominance, only that calibration-free quantization is competitive while sidestepping the calibration-data problem.
The speedup numbers are measured on specific hardware (H200 for SoloQ-C, RTX Pro 6000 for SoloQ-N). SoloQ-N’s advantage depends on hardware NVFP4 support; on devices without it, the practical acceleration drops. SoloQ-C uses non-uniform codebooks represented in INT8, so it can’t ride native 4-bit tensor cores and gets smaller speedups than SoloQ-N even on supported hardware.
The K-RPBH mixer helps most when the dimension decomposes into many blocks; the “K-RPBH reduces to RPBH” case (single block) means the contribution is less meaningful for models whose dimensions already fit RPBH well. And as with any PTQ work, benchmark averages hide per-task variance: the qualitative examples in the appendix show individual generations can shift between full precision and quantized versions, sometimes fixing an FP16 error, sometimes introducing a new one.