Get Started
Home
Topics
Search
Library
Research questionHow can autoregressive LLM decoding generate multiple tokens in parallel at large batch sizes without sacrificing quality?Autoregressive LLMs repeatedly predict one next token at a time, limiting throughput and increasing serving costs as batch sizes grow. Parallelizing generation is difficult because it must retain the quality and distribution of the original autoregressive model.
AI
Diffusion Models
Evaluation & Benchmarks
Inference Optimization
Reasoning
Technology
Latest papersRecent research connected to this question, newest first.Unlocking Lossless Speedups in LLMs via Discrete DiffusionThe evidence concerns Uno, a diffusion-augmented LLM that retains autoregressive weights, adds lightweight diffusion weights, and uses Ψ-Spec samplers without a separate draft model. Reported results cover evaluated batch sizes, with speedups of up to 3× over the base autoregressive model and comparisons across agentic tool use, coding, and long-context reasoning.research paper · Sep 3, 2026
Related questions
How can autoregressive language models add recurrent cross-token memory without breaking KV caching or one-forward-per-token decoding?How can lossy speculative decoding accelerate LLM inference without distorting token distributions or degrading generation quality?How can large language models cut training and inference costs without materially harming accuracy?How can grammar-constrained decoding preserve syntactic validity without distorting an LLM’s output probabilities?