Get Started
Home
Topics
Search
Library
Research questionHow can repetition penalties remain invariant to logit shifts without corrupting structured LLM output?Sign- branched penalties treat positive and negative raw logits differently, even though adding a constant to all logits leaves the model’s probabilities unchanged. As a result, equivalent logit representations can produce different token choices and substantially reduce structured-output validity.
AI
Evaluation & Benchmarks
Inference Optimization
Natural Language Processing
Research Paper
Latest papersRecent research connected to this question, newest first.Gauge dependence and structured-output corruption in sign-branched repetition penalties: measurements across models, inference stacks, and alternative repetition controlsThe evidence covers five models up to 7B parameters, two code models, 200 JSON schemas, and replications in vLLM and llama.cpp. It also reports that Hugging Face beam search applies penalties to log-probabilities, so the same setting can represent different operators across decoding strategies; the measurements do not establish behavior for all models or deployments.research paper · Sep 2, 2026
Related questions
How can grammar-constrained decoding preserve syntactic validity without distorting an LLM’s output probabilities?How can autoregressive LLM decoding generate multiple tokens in parallel at large batch sizes without sacrificing quality?How can open-ended LLM decoding avoid repetitive text without losing coherence?How can generative LLM rerankers produce valid item permutations without sequential decoding for real-time ranking?