Stochastic Gradient Descent: A Nonlinear Time Series Perspective

Jiaqi Li, Zhipeng Lou, Stefan Richter, Wei Biao Wu

arXivarxiv/2610.20894Oct 14, 2026stat.MLscore 0

This paper investigates the statistical behavior of Stochastic Gradient Descent (SGD) with constant step size under the framework of iterated random functions. Compared to previous studies establishing the convergence of SGD in probability measure, e.g., Wasserstein distance, our approach directly provides the convergence in Euclidean distance by showing the Geometric Moment Contraction (GMC) of SGD. This new convergence can address the non-stationarity of SGD due to fixed initial points and can provide a more refined asymptotic analysis of SGD. Specifically, we prove quenched central limit…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2610.20894
curl -X POST /api/v1/records/arxiv/2610.20894/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2610.20894/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'