Why Geometric Continuity Emerges in Deep Neural Networks: Residual Connections and Rotational Symmetry Breaking
arXiv:2605.04971v1 Announce Type: new
Abstract: Weight matrices in deep networks exhibit geometric continuity — principal singular vectors of adjacent layers point in similar directions. While this property has been widely observed, its origin remain…