Piper: Efficient Large-Scale MoE Training via Resource Modeling and Pipelined Hybrid Parallelism
arXiv:2605.05049v1 Announce Type: cross
Abstract: Frontier models increasingly adopt Mixture-of-Experts (MoE) architectures to achieve large-model performance at reduced cost. However, training MoE models on HPC platforms is hindered by large memory f…