Topic · 40 recaps

Inference Optimization

Techniques that make trained models faster and cheaper to serve: quantization, speculative decoding, KV-cache tricks, kernel fusion, batching, and architectural choices for low-latency generation.

Sort

Newest

Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding

Inference Optimization · Aug 28

Sliding-window beats linear attention

Inference Optimization · Aug 28

Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within $5090

Inference Optimization · Aug 27

TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming

Inference Optimization · Aug 21

BDH-CQ: In-Context Learning with Recurrent Latent Reasoning

Inference Optimization · Aug 10

LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers

Agents · Aug 7

SimWAM: A Simple World Action Model for End-to-End Autonomous Driving

Inference Optimization · Aug 7

JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion

Diffusion · Aug 4

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

Inference Optimization · Jul 29

Metis: Memory Foundation Model

Inference Optimization · Jul 29