Get Started
Home
Topics
Search
Library
Research questionHow can independently trained, frozen LoRA reasoning experts be combined without disrupting their learned update scales?Each LoRA expert is trained under a particular update scale, so combining its update with others can change the behavior it learned. Soft mixtures may add unnecessary routing complexity while still concentrating on one expert.
AI
Evaluation & Benchmarks
LLM Pretraining & Post-training
Machine Learning
Reasoning
Research Paper
Technology
Latest papersRecent research connected to this question, newest first.Learning to Select, Not Relearn: Hard-Routed Mixtures of Reasoning LoRAsThe source studies frozen reasoning LoRA experts trained independently with reinforcement learning from verifiable feedback. It trains a shared router and a small attention LoRA using distilled reasoning traces, evaluates across five benchmarks, multiple model scales, and additional model families, and reports evidence about behavior preservation and trainable-parameter efficiency.research paper · Sep 3, 2026
Related questions
Should reasoning LLMs combine on-policy distillation and verifiable-reward RL jointly or sequentially during post-training?How can large language models allocate reasoning computation to preserve accuracy under limited training and inference budgets?How can inference prune activated MoE experts without confounding compute savings with output rescaling?How can multi-domain MoE fine-tuning prevent negative transfer when token-level routing separates domains but LoRA updates share subspaces?