{"id":"arxiv/2609.11751","url":"https://arxiv.org/abs/2609.11751","tags":["math.OC"],"score":0,"title":"Stochastic Gradient Methods with Online Scaling","domain":"math.OC","source":"arxiv","authors":["Wanyu Zhang","Wenzhi Gao","Yinyu Ye","Madeleine Udell"],"reviews":0,"subject":"math.OC","summary":"This paper introduces Stochastic Online Scaled Gradient Methods (SOSGM), a generalization of the recently developed adaptive preconditioning framework in arXiv:2505.23081 and arXiv:2509.11007 to stochastic optimization. Under standard assumptions, we establish convergence guarantees for SOSGM using large batchsize or variance reduction. SOSGM is compatible with popular diagonal and/or low-rank preconditioners as well as heavy-ball momentum, while maintaining memory and computation cost comparable to Adam. Extensive numerical experiments demonstrate the strong empirical performance of SOSGM.…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-09-10","reactions":{}}