Get Started
Home
Topics
Search
Library
Research questionCan post-training ternarization make language models smaller without unacceptable capability loss or slower inference?Ultra-low-bit weights can shrink model storage, but nominal bit counts may not reflect the stored representation, uneven task degradation, or actual inference speed. Compression may therefore improve footprint without improving end-to-end deployment performance.
AI
Evaluation & Benchmarks
Inference Optimization
LLM Pretraining & Post-training
Machine Learning
Research Paper
Small / On-device Models
Technology
Latest papersRecent research connected to this question, newest first.Post-Training Ternarization of Qwen3-4B Capability, Effective Bit Budget, Storage Compression, and DeploymentThe evidence concerns an instruction-tuned Qwen3-4B model converted with weight-only ternarization while activations remain at 16-bit precision. The conversion uses 1.641 effective bits per targeted weight and reduces a packed artifact from 8.29 GiB to 3.96 GiB with essentially unchanged perplexity; scored capability accuracy falls from 64.5% to 54.7% across ten comparisons, with uneven task-level degradation. The packed artifact lacks end-to-end task-accuracy and generation-throughput benchmarks, and a preliminary Triton GEMV test was 4.6× slower than FP16 cuBLAS on one shape. A separate lossy packing attempt is excluded from the primary artifact claim.research paper · Sep 2, 2026
Related questions
How can large language models cut training and inference costs without materially harming accuracy?How can post-training make language-model refusals robust without sacrificing general capability?How can we distill capable small language models with fewer training tokens without losing teacher behavior?How can large language models allocate reasoning computation to preserve accuracy under limited training and inference budgets?