AdaFRUGAL: Adaptive Memory-Efficient Training with Dynamic Control
arXiv:2601.11568v2 Announce Type: replace-cross
Abstract: Training Large Language Models (LLMs) is highly memory-intensive due to optimizer state overhead. The FRUGAL framework mitigates this with gradient splitting, but its static hyperparameters — …