Get Started
Home
Topics
Search
Library
Research questionHow can generative LLM rerankers produce valid item permutations without sequential decoding for real-time ranking?Generative LLM rerankers emit ordinal tokens sequentially, making latency grow with the output and potentially producing invalid permutations. The underlying problem is generating an ordered list efficiently while preserving ranking quality.
AI
Inference Optimization
Information Retrieval
Machine Learning
Natural Language Processing
Latest papersRecent research connected to this question, newest first.SPD: Single Pass Decoding for Generative RerankingThe source studies ranking N items with an LLM and reports a single-forward-pass approach that extracts item-position scores from prefill states and solves the resulting assignment problem. It provides evidence from LoRA adaptation, autoregressive ranking distillation, ablations, and reported 28 ms inference with ranking quality comparable to the teacher.research paper · Sep 4, 2026
Related questions
How can lossy speculative decoding accelerate LLM inference without distorting token distributions or degrading generation quality?How can autoregressive LLM decoding generate multiple tokens in parallel at large batch sizes without sacrificing quality?How can open-ended LLM decoding avoid repetitive text without losing coherence?How can grammar-constrained decoding preserve syntactic validity without distorting an LLM’s output probabilities?