MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
arXiv:2603.24044v1 Announce Type: new
Abstract: Standard LoRA fine-tuning of Mixture-of-Experts (MoE) models applies adapters to every expert, yet our profiling shows that per-layer expert routing is highly skewed: a small subset of experts handles mo…