Get Started
Research questionHow can multi-shell Leech-lattice 2-bit weights be decoded efficiently for batch-one GPU serving?Multi-shell Leech-lattice weights require decoding more than one codebook shell during inference. Their GPU representation affects memory traffic and latency, while the aggressive quantization can reduce model quality.
AI
Inference Optimization
Technology
Latest papersRecent research connected to this question, newest first.Unfolding the Leech Lattice: Fused Multi-Shell Decoding and VRAM Layouts for 2-Bit LLM WeightsThe evidence covers decode-phase GEMV at batch size 1 using the full 301-class codebook, with offline-expanded GPU layouts and fused dequantization with matrix-vector multiplication. It reports bit-exact VRAM layout measurements, comparisons with FP16, AWQ, and QTIP kernels, results across two memory hierarchies, and end-to-end measurements for 4B, 8B, and 14B models alongside perplexity and MMLU quality results.research paper · Sep 2, 2026
Related questions
How can extreme low-bit LLM quantization adapt its basis across layers, calibration data, and quantizers?How can lossy speculative decoding accelerate LLM inference without distorting token distributions or degrading generation quality?How can LLM serving preserve reproducible agent trajectories when prefix caching interacts with weight quantization?How can on-device LLM inference overlap weight I/O and computation under tight DRAM without stale sparsity decisions?
Home
Topics
Search
Library