{"id":"arxiv/2604.15554","url":"https://arxiv.org/abs/2604.15554","tags":["cs.LG","cs.AI","math.NA","math.OC"],"score":0,"title":"Natural gradient descent with momentum","domain":"cs.LG","source":"arxiv","authors":["Anthony Nouy","Agustín Somacal"],"reviews":0,"subject":"cs.LG","summary":"We consider the problem of approximating a function by an element of a nonlinear manifold which admits a differentiable parametrization, typical examples being neural networks with differentiable activation functions or tensor networks. Natural gradient descent (NGD) for the optimization of a loss function can be seen as a preconditioned gradient descent where updates in the parameter space are driven by a functional perspective. In a spirit similar to Newton's method, a NGD step uses, instead of the Hessian, the Gram matrix of the generating system of the tangent space to the approximation…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-04-16","reactions":{}}