Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization
arXiv:2605.04269v1 Announce Type: new
Abstract: We provide a theoretical analysis of Adam under non-stationary stochastic objectives, separating two regimes: Euclidean tracking under adaptive strong monotonicity of the Adam-preconditioned mean-gradien…