Can an MLP Absorb Its Own Skip Connection?
arXiv:2604.23705v1 Announce Type: new
Abstract: We study when a skip connection around a single-hidden-layer MLP can be absorbed into a residual-free MLP of the same width. We first show that for any architecture whose skip branch is an invertible lin…