Get Started
Home
Topics
Search
Library
Research questionHow can inference prune activated MoE experts without confounding compute savings with output rescaling?Reducing the number of selected experts changes both which computations are performed and how router probabilities are normalized. Quality loss may therefore result from altered output strength rather than expert removal alone.
AI
Inference Optimization
Machine Learning
Natural Language Processing
Latest papersRecent research connected to this question, newest first.Training-Free Halving of Activated Experts in Fine-Grained Mixture-of-Experts ModelsThe source studies inference-time expert reduction and alternative normalization reference sets in two fine-grained MoE language models, examining language-model and downstream evaluation behavior. Its evidence is limited to those architectures, routing configurations, and evaluated tasks, and does not establish generality across MoE designs.research paper · Sep 4, 2026
Related questions
How should experts be pruned in over-dispersed MoE routing when router importance and perplexity mislead?How can residual sparsification reduce MoE LLM memory use without accuracy loss from coupled expert projections?How does sparse mixture-of-experts routing trade off approximation, learning error, and computation under misspecification and evolving experts?When can factual recall in sparse MoE language models be attributed to one expert rather than an expert set?