Perceptrons and localization of attention’s mean-field landscape
arXiv:2601.21366v2 Announce Type: replace
Abstract: The forward pass of a Transformer can be seen as an interacting particle system on the unit sphere: time plays the role of layers, particles that of token embeddings, and the unit sphere idealizes la…