{"id":"arxiv/2602.09006","url":"https://arxiv.org/abs/2602.09006","tags":["cs.LG","cs.AI","math.OC"],"score":0,"title":"ARO: A New Lens On Matrix Optimization For Large Models","domain":"cs.LG","source":"arxiv","authors":["Wenbo Gong","Javier Zazo","Qijun Luo","Puqian Wang","James Hensman","Chao Ma"],"reviews":0,"subject":"cs.LG","summary":"Matrix-based optimizers have attracted growing interest for improving LLM training efficiency, with significant progress centered on orthogonalization/whitening based methods. While yielding substantial performance gains, a fundamental question arises: can we develop new paradigms beyond orthogonalization, pushing the efficiency frontier further? We present \\textbf{Adaptively Rotated Optimization (ARO}, a new matrix optimization framework that treats gradient rotation as a first class design principle. ARO accelerates LLM training by performing normed steepest descent in a rotated coordinate…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-02-09","reactions":{}}