OASIS: Online Activation Subspace Learning for Memory-Efficient Training
arXiv:2604.09406v1 Announce Type: new
Abstract: Training large language models (LLMs) is constrained by memory requirements, with activations accounting for a substantial fraction of the total footprint. Existing approaches reduce memory using low-ran…