The Topological Trouble With Transformers
arXiv:2604.17121v1 Announce Type: new
Abstract: Transformers encode structure in sequences via an expanding contextual history. However, their purely feedforward architecture fundamentally limits dynamic state tracking. State tracking — the iterative…