Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
arXiv:2605.07870v1 Announce Type: cross
Abstract: We study the evolution of hidden-weight spectra in wide neural networks trained by (stochastic) gradient descent. We develop a two-level dynamical mean-field theory (DMFT) that jointly tracks bulk and …