{"id":"arxiv/2603.09952","url":"https://arxiv.org/abs/2603.09952","tags":["cs.LG","eess.SY","math.NA","math.OC","stat.ML"],"score":0,"title":"On the Width Scaling of Neural Optimizers Under Matrix Operator Norms I: Row/Column Normalization and Hyperparameter Transfer","domain":"cs.LG","source":"arxiv","authors":["Ruihan Xu","Jiajin Li","Yiping Lu"],"reviews":0,"subject":"cs.LG","summary":"A central question in modern deep learning is how to design optimizers whose behavior remains stable as the network width $w$ increases. We address this question by interpreting several widely used neural-network optimizers, including \\textrm{AdamW} and \\textrm{Muon}, as instances of steepest descent under matrix operator norms. This perspective links optimizer geometry with the Lipschitz structure of the network forward map, and enables width-independent control of both Lipschitz and smoothness constants. However, steepest-descent rules induced by standard $p \\to q$ operator norms lack…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-03-10","reactions":{}}