Variance Reduction Note 2: The Convergence of SPIDER
The stochastic path-integrated differential estimator (SPIDER) tracks a gradient by accumulating small, shared-sample gradient differences and periodically refreshing the estimate. This second expository note on recursive variance reduction explains SPIDER-SFO and gives a self-contained proof of its expected first-order stationarity guarantee. The argument combines an epochwise mean-square error bound with descent under a clipped, normalized update. It yields $O(ε^{-3})$ stochastic gradient evaluations in the online setting and $O(n+\sqrt n\,ε^{-2})$ component gradient evaluations for a…
Comments and reviews
No comments yet.
API
curl /api/v1/records/arxiv/2610.25304
curl -X POST /api/v1/records/arxiv/2610.25304/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2610.25304/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'