DCOCO corrects parallel diffusion drafts in Speculative Decoding by scoring all adjacent candidate-token pairs in one batched pass, then walking the grid left-to-right, delivering up to 6.6× speedup over vanilla decoding without a sequential correction module.
Serving an LLM is bottlenecked by the fact that each token waits on the previous one. Speculative decoding fixes this by having a cheap drafter propose several tokens that the big verifier model then checks in parallel. The verifier accepts the longest correct prefix, so throughput depends on both how many drafted tokens survive and how fast the drafter runs.
The usual drafter is a small autoregressive model (e.g., EAGLE), which keeps token-to-token dependencies but pays a sequential cost. A newer line, including DFlash, uses a Diffusion language model to predict a whole block of future tokens in one forward pass. That is fast but weakens coherence: each slot is predicted without seeing the actual tokens chosen at earlier slots, so individually plausible picks can combine into junk (the paper’s example: a model that likes both “New York” and “San Diego” may emit “New Diego”).
Two recent methods, Domino and DSpark, patch this by bolting a lightweight sequential correction module onto the parallel drafter. That restores coherence but puts a recurrence back on the critical path of every drafting cycle. The paper’s question: can you recover the lost dependencies without reintroducing sequential compute?
Two observations drive the design. First, the parallel drafter usually already contains the right token among its top candidates, it just failed to rank it first. On Qwen3-8B, the top-16 candidates contain the verifier’s chosen token at ~60% of slots where the top-1 is wrong, and this gap grows deeper into the block. So correction can be re-selection among existing candidates, not generation of new ones. Second, verification stops at the first rejected token, so fixing slot 5 is worthless if slot 2 is wrong. Correction should prioritize getting the early prefix right.
DCOCO keeps the top-K candidates (K=16) at each of the B drafted slots, forming a B×K grid. For every pair of adjacent slots and every pair of candidates in them, it computes a score: the drafter’s own logit for the successor plus a learned correction from a small shared MLP that reads the two slots’ hidden states, the two candidate embeddings, and the two draft logits. Crucially, all (B-1)·K² transition scores are computed in one batched pass. No per-slot recurrence, no extra drafter forward pass.
Then comes selection. Rather than maximizing total path score, DCOCO uses prefix-first greedy selection: keep the drafter’s top pick at slot 1, then at each later slot pick the highest-scoring successor conditional on the predecessor just chosen. This mirrors how verification consumes the draft and avoids trading an early win for a late one. The MLP’s output layer is zero-initialized, so at the start of training DCOCO reproduces the drafter’s own argmax exactly.
# One DFlash pass gives hidden states h, candidate ids c, draft logits L
for p in range(B - 1):
s_p = W_pair @ concat(h[p], h[p+1]) # pair state, once per slot-pair
for a in range(K):
for b in range(K): # all K*K done in one batched op
S[p, a, b] = L[p+1, b] + MLP(s_p, z[p,a], z[p+1,b], L[p,a], L[p+1,b])
k[0] = argmax(L[0]) # start from drafter's top-1
for p in range(B - 1):
k[p+1] = argmax_b(S[p, k[p], b]) # greedy walk, O(BK)
Training jointly optimizes the drafter and corrector with a cross-entropy over the K² transitions whose label is the verifier’s argmax pair, with an exponential weight that emphasizes earlier positions. At inference the softmax is dropped; raw scores feed the greedy walk.
The paper evaluates four verifier backbones (Qwen3-4B, Qwen3-8B, Qwen3-Coder-30B-A3B, Llama-3.1-8B) across ten benchmarks spanning math, code, and chat, under greedy and sampling decoding, with and without extended thinking, using vLLM on a single B200 GPU.
•
Throughput at single-stream. DCOCO is the fastest in every one of twelve verifier×thinking×temperature cells, beating the strongest baseline by 8–19% on average. Up to 6.6× over vanilla autoregressive decoding on math with Qwen3-4B. On Qwen3-8B without thinking at T=0, average throughput is 959.7 tok/s vs 839.3 for Domino and 783.1 for uncorrected DFlash.
•
Where the gain comes from. Against Domino on Qwen3-8B, DCOCO accepts 7–8% longer prefixes and runs 6% shorter cycles. Against uncorrected DFlash, it pays 3–4% per cycle for 18–27% longer accepted prefixes. Both axes contribute.
•
Concurrent serving. This is the headline result. From 1 to 32 concurrent streams on Qwen3-8B, DCOCO’s throughput stays 19–25% above DFlash under greedy decoding. Domino and DSpark, whose correction modules run sequentially, lose ground as load grows and eventually fall below the uncorrected DFlash drafter; their per-cycle overhead eats the acceptance gain. DCOCO’s margin over the strongest baseline reaches 23% under greedy, 18% under sampling at some concurrency levels.
•
Selection rule matters. Over identical trained scores, prefix-first greedy matches or beats a geometrically discounted global-sum rule on throughput on every suite tested, averaging +2.8% throughput over the geometric variant, with no discount hyperparameter.
•
Ablations. Removing the pair-state input costs 7.3%/5.6% of accepted length (T=0/T=1); removing candidate representations costs 4.8%/3.7%. Freezing the drafter and training only the corrector as a drop-in still improves DFlash by 4.4%, with joint training worth another ~15%.
The authors frame the concurrency result as the key practical point: parallel correction is the only kind that keeps winning once the GPU is actually loaded.
•
If you serve a model and already use (or are evaluating) a parallel diffusion drafter like DFlash, this is the correction scheme to try first, especially if your production load is more than a handful of concurrent requests. Code and trained correctors are available: GitHub, Hugging Face.
•
The prefix-first greedy rule is a cheap idea worth borrowing on its own. If you have any pipeline that produces per-position candidate distributions and is consumed left-to-right with a hard stop at the first error, maximizing a global sum over the sequence will trade early quality for late quality that never gets read. Pick greedily from the front.
•
The coverage analysis is a useful diagnostic lens for other non-autoregressive generation setups: before building a corrector, check whether the right token is already in your top-K. If it is, re-ranking beats regeneration.
•
Caveat on scope: the method is tested with the specific DFlash backbone and verifiers listed, trained on ~800K prompts from the Nemotron Post-Training Dataset v2 and CodeAlpaca-20k. The authors do not evaluate it as a drop-in on autoregressive drafters like EAGLE, and the gains are tied to parallel block drafters whose top-K already contains the right token often enough.
•
All experiments run on a single NVIDIA B200 with vLLM 0.27.0 and tensor parallelism 1. Multi-GPU, multi-node, or alternative serving stacks are not evaluated.
•
The baselines (Domino, DSpark) were re-implemented and retrained by the authors on their own corpus with a shared drafter backbone, not taken as released artifacts. Each method is trained with one seed (42), so training-run variance is unmeasured.
•
Sampling-mode (T=1) gains are smaller than greedy gains and shrink further on Llama-3.1-8B (+16% at T=0 drops to +8% at T=1 vs the best baseline). On LiveCodeBench with Llama-3.1-8B at T=1, DCOCO is actually slower than DFlash in the per-dataset table, so dataset-level behavior is not uniform.
•
The method only helps when the drafter’s top-K already contains the verifier’s token; coverage reported here is for DFlash specifically, and the headroom will differ for other parallel drafters.
•
DCOCO’s lossless stochastic verification requires that the verifier treat the deterministic DCOCO proposal as a point-mass distribution, which may need changes in a serving stack that assumes a probabilistic drafter.