{"id":"arxiv/2606.27216","url":"https://arxiv.org/abs/2606.27216","tags":["math.NA","cs.LG"],"score":0,"title":"Hierarchical Muon: Tiled Newton-Schulz Updates for Efficient Muon Optimization","domain":"math.NA","source":"arxiv","authors":["Ziyuan Tang","Tianshi Xu","Yousef Saad","Yuanzhe Xi"],"reviews":0,"subject":"math.NA","summary":"Muon-type optimizers construct update directions for dense neural-network weights by applying a finite Newton-Schulz map to momentum-gradient matrices. For an $H \\times W$ matrix, with $r=\\min\\{H,W\\}$ and $s=\\max\\{H,W\\}$, $K$ steps of the full-matrix Newton-Schulz update require $O(r^2 s K)$ work and couple all rows and columns through repeated Gram matrix products. We introduce Hierarchical Muon (HiMuon), a tiled Newton-Schulz scheme for Muon-type optimization. HiMuon partitions each momentum-gradient matrix into $T \\times T$ tiles, applies the same finite Newton-Schulz map independently to…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-06-25","reactions":{}}