Get Started
Home
Topics
Search
Library
Research questionHow can LLM serving adapt KV-cache capacity as attention demand changes during long-output reasoning?Fixed per-request KV-cache capacity cannot reflect either differences between requests or changing attention demand during generation. This limits memory efficiency when reasoning outputs are long.
AI
AI Memory
Inference Optimization
Reasoning
Technology
Latest papersRecent research connected to this question, newest first.GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning ServingThe source describes GrowPage, which uses PagedAttention’s page-level memory abstraction and is evaluated across reasoning benchmarks and multiple models using performance–throughput trade-offs. The evidence concerns LLM serving with KV-cache budgeting and does not establish applicability beyond the reported workloads and systems.research paper · Sep 3, 2026
Related questions
How can long-context LLM inference reduce KV-cache memory without losing attention-head-specific information?How can long-reasoning KV caches retain useful context while reducing memory and eviction overhead?How can we robustly compress LLM KV caches across open-domain inputs without input-specific budget thresholds?How can batched LLM serving handle request-specific read-write adaptation state without corrupting streams or losing throughput?