On the Asymptotics of Self-Supervised Pre-training: Two-Stage M-Estimation and Representation Symmetry
arXiv:2603.27631v1 Announce Type: new
Abstract: Self-supervised pre-training, where large corpora of unlabeled data are used to learn representations for downstream fine-tuning, has become a cornerstone of modern machine learning. While a growing body…