{"id":"arxiv/2601.20399","url":"https://arxiv.org/abs/2601.20399","tags":["math.OC","cs.LG","stat.ML"],"score":0,"title":"Convergence Analysis of Randomized Subspace Normalized SGD under Heavy-Tailed Noise","domain":"math.OC","source":"arxiv","authors":["Gaku Omiya","Pierre-Louis Poirion","Akiko Takeda"],"reviews":0,"subject":"math.OC","summary":"Randomized subspace methods reduce per-iteration cost; however, in nonconvex optimization, most analyses are expectation-based, and high-probability bounds remain scarce even under sub-Gaussian noise. We first prove that randomized subspace SGD (RS-SGD) admits a high-probability convergence bound under sub-Gaussian noise, achieving the same order of oracle complexity as prior in-expectation results. Motivated by the prevalence of heavy-tailed gradients in modern machine learning, we then propose randomized subspace normalized SGD (RS-NSGD), which integrates direction normalization into…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-01-28","reactions":{}}