Adaptive Head Budgeting for Efficient Multi-Head Attention
arXiv:2604.22583v1 Announce Type: new
Abstract: Transformers have become the dominant architecture across a wide range of domains, largely due to the effectiveness of multi-head attention in capturing diverse representation subspaces. However, standar…