Optimal Two-Step Stepsize Schedule for Stochastic Gradient Methods
Structured nonconstant large stepsizes can improve the convergence of gradient descent in the deterministic setting. However, in stochastic optimization, aggressive stepsizes can amplify oracle noise and hinder the convergence of stochastic gradient methods. We characterize the globally optimal two-step stepsize schedule for stochastic gradient methods applied to strongly convex and smooth functions, assuming access only to unbiased stochastic gradient estimates with finite support and bounded variance. The optimal schedule depends on the ratio of the initial optimality gap to the noise…
Comments and reviews
No comments yet.
API
curl /api/v1/records/arxiv/2608.15035
curl -X POST /api/v1/records/arxiv/2608.15035/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2608.15035/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'