Where You Place the Norm Matters: From Prejudiced to Neutral Initializations
arXiv:2505.11312v4 Announce Type: replace
Abstract: Normalization layers were introduced to stabilize and accelerate training, yet their influence is critical already at initialization, where they shape signal propagation and output statistics before …