The Myth of Expert Specialization in MoEs: Why Routing Reflects Geometry, Not Necessarily Domain Expertise
arXiv:2604.09780v1 Announce Type: new
Abstract: Mixture of Experts (MoEs) are now ubiquitous in large language models, yet the mechanisms behind their “expert specialization” remain poorly understood. We show that, since MoE routers are linear maps, h…