{"id":"arxiv/2601.07326","url":"https://arxiv.org/abs/2601.07326","tags":["math.OC","cs.LG"],"score":0,"title":"Convergence Rate Analysis of the AdamW-style Shampoo: Unifying One-Sided and Two-Sided Preconditioning","domain":"math.OC","source":"arxiv","authors":["Huan Li","Yiming Dong","Zhouchen Lin"],"reviews":0,"subject":"math.OC","summary":"This paper studies AdamW-style Shampoo, an effective variant of the classical Shampoo that won the external tuning track of the AlgoPerf neural network training competition. Our analysis unifies one-sided and two-sided preconditioning. When the exponents of the two preconditioners sum to $1/2$, we establish the convergence rate $\\frac{1}{K}\\sum_{k=1}^KE\\left[||\\nabla f(X_k)||_*\\right]\\leq O(\\frac{\\sqrt{m+n}C}{K^{1/4}})$, where $K$ represents the number of iterations, $(m,n)$ denotes the dimensions of the matrix-valued parameters, and $C$ matches the constant appearing in the optimal…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-01-12","reactions":{}}