Privacy Amplification in Differentially Private Zeroth-Order Optimization with Hidden States
arXiv:2506.00158v2 Announce Type: replace
Abstract: Zeroth-order optimization has emerged as a promising approach for fine-tuning large language models under differential privacy (DP) and memory constraints. While privacy amplification by iteration (P…