Approximating Simple ReLU Networks based on Spectral Decomposition of Fisher Information
arXiv:2505.17907v2 Announce Type: replace-cross
Abstract: Properties of Fisher information matrices of 2-layer neural ReLU networks with random hidden weights are studied. For these networks, it is known that the eigenvalue distribution highly concent…