Pretraining Induces a Reusable Spectral Basis for Downstream Task Adaptation
arXiv:2605.07302v1 Announce Type: new
Abstract: Finetuning pretrained models occurs in a low-dimensional subspace of the full parameter space. Prior work has focused on characterizing this optimization subspace, but largely ignored the complementary q…