Accelerating Zeroth-Order Spectral Optimization with Partial Orthogonalization from Power Iteration
arXiv:2605.09034v1 Announce Type: new
Abstract: Zeroth-order (ZO) optimization has become increasingly popular and important in fine-tuning large language models (LLMs), especially on edge devices due to its ability to adjust the model to local data w…