{"id":"arxiv/2604.01472","url":"https://arxiv.org/abs/2604.01472","tags":["math.OC","cs.AI","cs.LG"],"score":0,"title":"The Newton-Muon Optimizer","domain":"math.OC","source":"arxiv","authors":["Zhehang Du","Weijie Su"],"reviews":0,"subject":"math.OC","summary":"The Muon optimizer has received considerable attention for its strong performance in training large language models, yet the design principle behind its matrix-gradient orthogonalization remains largely elusive. In this paper, we introduce a surrogate model that not only sheds new light on the design of Muon, but more importantly leads to a new optimizer. In the same spirit as the derivation of Newton's method, the surrogate approximates the loss as a quadratic function of the perturbation to a weight matrix $W$ using only three matrices: the gradient $G$, an output-space curvature matrix…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-04-01","reactions":{}}