Get Started
Topic · 64 recaps
Inference Optimization
Techniques that make trained models faster and cheaper to serve: quantization, speculative decoding, KV-cache tricks, kernel fusion, batching, and architectural choices for low-latency generation.
Play all
...
Posts
Questions
Home
Topics
Search
Library
Sort
Newest
Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training
Inference Optimization · Sep 7
0
Two Cache Risks: Replay Failures and Hidden Grounding Loss
Evaluation · Sep 5 · 13:25
0
BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
Inference Optimization · Sep 4
0
GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation
Inference Optimization · Sep 4
0
Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference
Inference Optimization · Sep 4
0
Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
Inference Optimization · Sep 3
0
Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction
Inference Optimization · Sep 3
0
Rethinking On-Policy Distillation of Large Language Models II: One Training Example
Inference Optimization · Sep 3
0
Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
Inference Optimization · Sep 3 · 5:58
0
Compile by Training: Turning Natural-Language Specifications into Local Neural Functions
Code Generation · Sep 3 · 8:23
0