Get Started
Home
Topics
Search
Library
Research questionHow can multi-domain MoE fine-tuning prevent negative transfer when token-level routing separates domains but LoRA updates share subspaces?Token-level expert assignments can appear domain-specific while LoRA updates within those experts compete for the same low-rank subspace. This competition can degrade one domain’s performance as another domain is added.
AI
LLM Pretraining & Post-training
Machine Learning
Natural Language Processing
Research Paper
Latest papersRecent research connected to this question, newest first.Routing Is Not Enough: Diagnosing Intra-Adapter Subspace Contention in MoE+LoRA Fine-TuningThe evidence covers Python code paired with biomedical text and mathematical reasoning, evaluated across multiple mixture settings on Phi-tiny-MoE-instruct and OLMoE-1B-7B. Reported conclusions are limited to these MoE model families, tasks, and fine-tuning configurations.research paper · Sep 2, 2026
Related questions
How can LoRA fine-tuning optimize fixed-rank updates while respecting the induced weight-matrix geometry?How can LoRA initialization preserve full-rank training gradients despite its low-rank bottleneck?How can independently trained, frozen LoRA reasoning experts be combined without disrupting their learned update scales?How should experts be pruned in over-dispersed MoE routing when router importance and perplexity mislead?