Get Started
Research questionHow can LLM serving preserve reproducible agent trajectories when prefix caching interacts with weight quantization?Prefix caching reuses computed prompt representations, but its hidden state can make otherwise identical requests produce different agent trajectories. This reproducibility problem becomes more pronounced when model weights are quantized.
AI
AI Agents
Inference Optimization
Latest papersRecent research connected to this question, newest first.Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM ServingThe evidence covers an 80-episode multi-turn agentic tool-use workload run serially at batch size one across two serving engines and four weight formats. Requests used fixed models, decoding parameters, seeds, and order, with cache-enabled and recompute runs plus controlled cache-state experiments; the results concern trajectory divergence and its connection to task outcomes, not a general claim about all serving systems.research paper · Sep 4, 2026
Related questions
How can LLM agents reuse execution traces without losing temporal and outcome-dependent behavior?How can LLM trajectory evaluations distinguish genuine prefix value and early outcome information from compute and difficulty confounds?How can stateful LLM agents behave as if they never saw revoked information?How can LLM serving adapt KV-cache capacity as attention demand changes during long-output reasoning?
Home
Topics
Search
Library