{"id":"arxiv/2602.10905","url":"https://arxiv.org/abs/2602.10905","tags":["cs.LG","math.OC","stat.ML"],"score":0,"title":"Natural Hypergradient Descent: Algorithm Design, Convergence Analysis, and Parallel Implementation","domain":"cs.LG","source":"arxiv","authors":["Deyi Kong","Zaiwei Chen","Shuzhong Zhang","Shancong Mou"],"reviews":0,"subject":"cs.LG","summary":"In this work, we propose Natural Hypergradient Descent (NHGD), a new method for solving bilevel optimization problems. To address the computational bottleneck in hypergradient estimation--namely, the need to compute or approximate Hessian inverse--we exploit the statistical structure of the inner optimization problem and use the empirical Fisher information matrix as an asymptotically consistent surrogate for the Hessian. This design enables a parallel optimize-and-approximate framework in which the Hessian-inverse approximation is updated synchronously with the stochastic inner…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-02-11","reactions":{}}