Orthogonal Quadratic Complements for Vision Transformer Feed-Forward Networks
arXiv:2604.09709v1 Announce Type: cross
Abstract: Recent bilinear feed-forward replacements for vision transformers can substantially improve accuracy, but they often conflate two effects: stronger second-order interactions and increased redundancy re…