Author name: Leon Lufkin, Tom\'as Figliolia, Beren Millidge, Kamesh Krishnamurthy

arXiv:2603.22325v2 Announce Type: replace-cross
Abstract: Recurrent neural networks (RNNs) and self-attention are both widely used sequence-mixing layers that maintain an internal memory. However, this memory is constructed using two orthogonal mechan…