Expressivity of Transformers: A Tropical Geometry Perspective
arXiv:2604.14727v1 Announce Type: new
Abstract: To quantify the geometric expressivity of transformers, we introduce a tropical geometry framework to characterize their exact spatial partitioning capabilities. By modeling self-attention as a vector-va…