N-vium: Mixture-of-Exits Transformer for Accelerated Exact Generation
arXiv:2605.13190v1 Announce Type: new
Abstract: Improving the inference efficiency of autoregressive transformers typically means reducing FLOPs per token, usually through approximations that degrade model quality. We introduce N-vium, a mixture-of-ex…