Variance Reduction Note 4: The Convergence of SVRG
Source: arxiv.org/abs/2611.05075
Stochastic variance-reduced gradient (SVRG) uses a full gradient at a snapshot point to correct inexpensive component gradients. This fourth expository note on variance reduction explains that control variate and gives a self-contained proof of linear convergence for the classical strongly convex finite-sum setting. Each component is assumed convex and smooth, while only their average must be strongly convex. A second-moment bound links the estimator to the objective gaps at the current iterate and the snapshot. Summing a distance recursion over an epoch then yields a geometric decrease and…
Comments and reviews
No comments yet.
API
curl /api/v1/records/arxiv/2611.05075
curl -X POST /api/v1/records/arxiv/2611.05075/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.05075/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'