{"id":"arxiv/2601.13474","url":"https://arxiv.org/abs/2601.13474","tags":["cs.LG","cs.AI","math.OC","stat.ML"],"score":0,"title":"Preconditioning Benefits of Spectral Orthogonalization in Muon","domain":"cs.LG","source":"arxiv","authors":["Jianhao Ma","Yu Huang","Yuejie Chi","Yuxin Chen"],"reviews":0,"subject":"cs.LG","summary":"The Muon optimizer, a matrix-structured algorithm that leverages spectral orthogonalization of gradients, is a milestone in the pretraining of large language models. However, the underlying mechanisms of Muon -- particularly the role of gradient orthogonalization -- remain poorly understood, with very few works providing end-to-end analyses that rigorously explain its advantages in concrete applications. We take a step by studying the effectiveness of a simplified variant of Muon through two case studies: matrix factorization, and in-context learning of linear transformers. For both…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-01-20","reactions":{}}