OptEMA: Adaptive Exponential Moving Average for Stochastic Optimization with Zero-Noise Optimality
arXiv:2603.09923v2 Announce Type: replace
Abstract: The Exponential Moving Average (EMA) is a cornerstone of widely used optimizers such as Adam. However, existing theoretical analyses of Adam-style methods have notable limitations: their guarantees c…