(How) Learning Rates Regulate Catastrophic Overtraining
arXiv:2604.13627v1 Announce Type: new
Abstract: Supervised fine-tuning (SFT) is a common first stage of LLM post-training, teaching the model to follow instructions and shaping its behavior as a helpful assistant. At the same time, SFT may harm the fu…