{"id":"arxiv/2603.02092","url":"https://arxiv.org/abs/2603.02092","tags":["cs.LG","math.OC"],"score":0,"title":"Adam Converges Without Any Modification On Update Rules","domain":"cs.LG","source":"arxiv","authors":["Yushun Zhang","Bingran Li","Congliang Chen","Zhi-Quan Luo","Ruoyu Sun"],"reviews":0,"subject":"cs.LG","summary":"Adam is the default algorithm for training neural networks, including large language models (LLMs). However, \\citet{reddi2019convergence} provided an example that Adam diverges, raising concerns for its deployment in AI model training. We identify a key mismatch between the divergence example and practice: \\citet{reddi2019convergence} pick the problem after picking the hyperparameters of Adam, i.e., $(β_1,β_2)$; while practical applications often fix the problem first and then tune $(β_1,β_2)$. In this work, we prove that Adam converges with proper problem-dependent hyperparameters. First,…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-03-02","reactions":{}}