{"id":"arxiv/2602.07425","url":"https://arxiv.org/abs/2602.07425","tags":["cs.LG","cs.CL","math.OC"],"score":0,"title":"Sign-Based Optimizers Are Effective Under Heavy-Tailed Noise","domain":"cs.LG","source":"arxiv","authors":["Dingzhi Yu","Hongyi Tao","Yuanyu Wan","Luo Luo","Lijun Zhang"],"reviews":0,"subject":"cs.LG","summary":"While adaptive gradient methods are the workhorse of modern machine learning, sign-based optimization algorithms such as Lion and Muon have recently demonstrated superior empirical performance over AdamW in training large language models (LLM). However, a theoretical understanding of why sign-based updates outperform variance-adapted methods remains elusive. In this paper, we aim to bridge the gap between theory and practice through the lens of heavy-tailed gradient noise, a phenomenon frequently observed in language modeling tasks. Theoretically, we introduce a novel generalized…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-02-07","reactions":{}}