{"id":"arxiv/2602.01777","url":"https://arxiv.org/abs/2602.01777","tags":["cs.LG","cs.AI","math.ST","stat.ML"],"score":0,"title":"Stein-Rule Shrinkage for Stochastic Gradient Estimation in High Dimensions","domain":"cs.LG","source":"arxiv","authors":["M. Arashi","M. Amintoosi"],"reviews":0,"subject":"cs.LG","summary":"Stochastic gradient methods are central to large-scale learning, but they treat mini-batch gradients as unbiased estimators, which classical decision theory shows are inadmissible in high dimensions. We formulate gradient computation as a high-dimensional estimation problem and introduce a framework based on Stein-rule shrinkage. We construct a gradient estimator that adaptively contracts noisy mini-batch gradients toward a stable estimator derived from historical momentum. The shrinkage intensity is determined in a data-driven manner using an online estimate of gradient noise variance,…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-02-02","reactions":{}}