Get Started
Home
Topics
Search
Library
Research questionHow can open-ended LLM decoding avoid repetitive text without losing coherence?Open-ended LLM generation can collapse into repetitive or lexically dull continuations when unreliable probability tails and changing logit scales are handled poorly. Increasing diversity can also disrupt semantic coherence, making decoding choices difficult across generation and reasoning tasks.
AI
Inference Optimization
Natural Language Processing
Reasoning
Latest papersRecent research connected to this question, newest first.Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model DecodingThe source concerns a training-free, pre-decoding intervention that reshapes LLM token probabilities before truncation and can integrate with existing decoding strategies. Evidence comes from experiments on open-ended generation, factual QA, and mathematical reasoning, with reported negligible computational overhead.research paper · Sep 2, 2026
Related questions
How can grammar-constrained decoding preserve syntactic validity without distorting an LLM’s output probabilities?How can generative LLM rerankers produce valid item permutations without sequential decoding for real-time ranking?How can lossy speculative decoding accelerate LLM inference without distorting token distributions or degrading generation quality?How can autoregressive LLM decoding generate multiple tokens in parallel at large batch sizes without sacrificing quality?