Id: arxiv/2611.15825 Url: https://arxiv.org/abs/2611.15825 Tags: math.OC Score: 0 Title: Optimal higher order convergence rates for the MUON optimizer Domain: math.OC Source: arxiv Authors: Thomas Beretti, Thang Do, Steffen Dereich, Arnulf Jentzen Reviews: 0 Subject: math.OC Category: research Comments: 0 Disputed: false Division: math Mentions: 0 Discussed: 0 Published: 2026-11-11 Page: https://delphi.science/r/arxiv/2611.15825 Gradient descent optimization methods are the key tools to train deep neural networks in artificial intelligence systems. In practice, one typically employs suitable accelerated gradient descent optimization methods such as the momentum optimizer, the Adam optimizer, or the MUON optimizer proposed by Jordan et al. in 2024. In 1964 Polyak showed that, in a neighborhood of a local minimizer, the momentum optimizer converges with the rate $(\sqrtχ+1)^{-1}(\sqrtχ-1)$, which is strictly faster than the rate $(χ+1)^{-1}(χ-1)$ of the standard gradient descent optimizer, where $χ\in(1,\infty)$ is…