{"id":"arxiv/2604.15416","url":"https://arxiv.org/abs/2604.15416","tags":["cs.LG","cs.AI","math.OC"],"score":0,"title":"StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models","domain":"cs.LG","source":"arxiv","authors":["Dingzhi Yu","Rui Pan","Yuxing Liu","Difan Zou","Tong Zhang"],"reviews":0,"subject":"cs.LG","summary":"Sign-based optimization algorithms, such as SignSGD, have garnered attention for their performance in distributed learning and training large foundation models. Despite their empirical superiority, SignSGD is known to diverge on non-smooth objectives, which are ubiquitous due to ReLUs, max-pools, and mixture-of-experts. To overcome this limitation, we propose StoSignSGD, an algorithm that injects structural stochasticity into the sign operator while maintaining an unbiased update step. In the regime of (online) convex optimization, StoSignSGD rigorously resolves the non-convergence issues of…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-04-16","reactions":{}}