Author name: Thomas Pethick, Wanyun Xie, Roman Machacek, Volkan Cevher

arXiv:2605.11172v1 Announce Type: new
Abstract: We introduce SODA, a generalization of Optimistic Dual Averaging, which provides a common perspective on state-of-the-art optimizers like Muon, Lion, AdEMAMix and NAdam, showing that they can all be view…