Optimal Brain Decomposition for Accurate LLM Low-Rank Approximation
arXiv:2604.00821v1 Announce Type: new
Abstract: Low-rank decomposition has emerged as an important problem in Large Language Model (LLM) fine-tuning and inference. Through Singular Value Decomposition (SVD), the weight matrix can be factorized into lo…