DistillAlign fixes a hidden failure in fast video generators: the initialization stage and the Distribution Matching Distillation (DMD) refinement stage often chase different distributions, so refinement collapses diversity. Aligning their targets and adding a coverage-preserving loss lets a 1.3B teacher beat 14B-teacher baselines.
Say you’re shipping a text-to-video product. A big diffusion model generates gorgeous clips but takes 25+ denoising steps per video. The standard fix is to distill it into a few-step causal generator that produces frames autoregressively for real-time streaming. Pipelines like Self Forcing and Causal Forcing do this in stages: warm up the student with ODE distillation or Consistency distillation, then polish with Distribution Matching Distillation (DMD) against a stronger teacher. Teams pick the intermediate checkpoint by eyeballing VBench scores. This paper argues that habit is quietly broken. A student that looks sharper after warmup can actually be worse raw material for the DMD stage, because it has silently collapsed onto a narrow slice of the teacher’s distribution.
The intuition: DMD is a sharpening operator. It pulls the student toward high-density regions of the teacher, but it cannot invent modes the student doesn’t already reach. So if warmup has thrown away half the teacher’s modes to look pretty, DMD has nothing to sharpen there. Formally, DMD minimizes reverse-KL between student and teacher, which is mode-seeking. Consistency and ODE-style warmup, by contrast, regress onto teacher trajectories, which is mode-covering (broad but blurry).
The authors build two things on top of that observation. First, a teacher-normalized evaluation protocol: take the initializer’s video, re-noise it partway, denoise it back with a fixed reference teacher, then compare against reference samples in V-JEPA 2 feature space using k-NN precision and coverage. The re-noising step scrubs away sharpness differences so you’re measuring which semantic modes the initializer reaches, not how crisp its pixels are. Second, a joint distillation loss that runs DMD and consistency distillation simultaneously in the final stage, with the CD term acting as an anchor against distribution drift.
# Final-stage training, per step
x = student(noise, prompt)
loss_dmd = reverse_kl_via_score_diff(x, teacher) # mode-seeking
loss_cd = consistency_loss(student, cd_teacher) # mode-covering anchor
loss = loss_dmd + 0.01 * loss_cd
loss.backward()
They also argue the warmup data and the DMD teacher should be sampled from the same distribution. Mixing a warmup trained on 1.3B samples with a 14B DMD teacher creates unsupported gradients.
The prevailing habit is to pick warmup checkpoints by VBench and swap in the strongest available DMD teacher at the end. This paper shows the opposite. What matters is whether the warmup already covers the modes the DMD teacher will sharpen; a blurrier but broader initialization beats a crisper but collapsed one, and the two stages should share a target distribution. The load-bearing evidence is not the leaderboard number: it is the controlled swap where matching a weaker teacher across stages outperforms mismatching with a stronger one.
The finding that carries the thesis: in the controlled teacher-source swap, matched initialization-plus-DMD pairings achieve higher teacher coverage than mismatched pairings, both before and after DMD. Concretely, a mode-covering warmup like Causal CD Init keeps broad support through refinement, while mode-seeking warmups (AR Init, Causal DMD Init) start with high precision but low coverage and never recover it.
Secondary results reinforce this:
•
On the headline text-to-video comparison, DistillAlign with a Wan-1.3B DMD teacher already beats all baselines refined with Wan-14B. Distributional alignment substitutes for teacher scale.
•
Pure DMD shows late-stage drift: VBench rises then falls while diversity monotonically drops. Joint distillation with λ=0.01 holds coverage and diversity steady while matching or exceeding VBench.
•
Ablating λ recovers the expected knob: larger λ favors coverage, λ=0 reduces to pure DMD and collapses diversity.
•
The re-noising protocol matters. Raw coverage of a blurry ODE initializer scores 0.020, but after teacher-normalized re-noising it jumps to 0.326, matching its true post-DMD potential.
Reach for this when you’re building a distilled few-step video generator and picking your intermediate checkpoint by visual score alone. The concrete change: stop comparing warmup checkpoints on VBench. Instead, project both student and teacher videos into a frozen video-encoder space, run the k-NN precision/coverage estimator, and pick the checkpoint whose coverage against your final DMD teacher is highest, even if its raw samples look blurrier. Then, in the DMD stage, keep the consistency loss on at a small weight rather than dropping it.
Artifacts: the project page is at DistillAlign. The evaluation protocol is described in enough detail to reimplement: V-JEPA 2 features, k=5, N=M=256, re-noise at ρ=0.9 with a shared normalization teacher. The paper doesn’t explicitly state whether training code or checkpoints are released beyond the project page.
Pick your warmup by what modes it reaches, not by how sharp it looks. Sharpening is what the next stage does; coverage is the one thing sharpening cannot add back. If your fast generator feels samey across seeds, the fix is upstream of the final loss.
•
The whole framework assumes DMD-style pipelines with a mode-covering warmup followed by a mode-seeking refinement. If you’re doing single-stage distillation or GAN-based finetuning, the diagnosis doesn’t transfer directly.
•
The coverage metric depends on the V-JEPA 2 encoder, prompt set, and chosen reference teacher. Precision and coverage across different reference teachers are not on the same absolute scale, so cross-experiment ranking needs care.
•
Experiments run on 81-frame, 832×480 clips from Wan2.1 with 16 prompts × 16 seeds. Whether alignment matters as much at longer horizons, higher resolutions, or with very different base models is untested here.