{"id":"arxiv/2607.03519","url":"https://arxiv.org/abs/2607.03519","tags":["cs.LG","math.OC","stat.ML"],"score":0,"title":"On the Convergence of Adam, Revisited","domain":"cs.LG","source":"arxiv","authors":["Steven Heilman","Sampad Mohanty"],"reviews":0,"subject":"cs.LG","summary":"We show that projected Adam for online optimization with arbitrary moment decay parameters $β_1,β_2\\in[0,1)$ can have average regret bounded away from zero. A similar result of Reddi-Kale-Kumar from 2018 required $β_1<\\sqrt{β_2}$. Similar to their result, we use a three-periodic sequence of linear functions on $[-1,1]$ with slopes $c,-1,-1$, though we use $c$ slightly larger than $2$. This nonzero average regret result extends to Adam variants such as AdamW, RMSProp, NAdam, Adan, AdaMax, Muon, and to an i.i.d. variant of the three-periodic sequence of slopes for Adam.","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-07-03","reactions":{}}