{"id":"arxiv/2607.06314","url":"https://arxiv.org/abs/2607.06314","tags":["math.OC"],"score":0,"title":"Gradient descent with exponentially increasing stepsizes and restarts","domain":"math.OC","source":"arxiv","authors":["François Clément","Stefan Steinerberger"],"reviews":0,"subject":"math.OC","summary":"Let $f:\\mathbb{R}^d \\rightarrow \\mathbb{R}$. We consider gradient descent $x_{n+1} = x_n - τ_n \\nabla f(x_n)$, where the stepsize $τ_n = τ\\cdot e^{rn}$ is exponentially growing (with $τ> 0$ and $0 < r \\ll 1$). This diverges for almost all initial values. We show that restarting the algorithm whenever $\\|x_{n+1} - x_n\\| \\geq e^r\\|x_n - x_{n-1}\\|$ has good properties: it works very well in practice; we determine the limiting convergence rate in the case of convergence to a non-degenerate local minimum: it improves on classic gradient descent even though computational cost is comparable. The…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-07-07","reactions":{}}