Nora: Normalized Orthogonal Row Alignment for Scalable Matrix Optimizer
arXiv:2605.03769v1 Announce Type: new
Abstract: Matrix-based optimizers have demonstrated immense potential in training Large Language Models (LLMs), however, designing an ideal optimizer remains a formidable challenge. A superior optimizer must satis…