{"id":"arxiv/2604.01279","url":"https://arxiv.org/abs/2604.01279","tags":["cs.LG","cs.AI","hep-th","math.OC"],"score":0,"title":"Sven: Singular Value Descent as a Computationally Efficient Natural Gradient Method","domain":"cs.LG","source":"arxiv","authors":["Samuel Bright-Thonney","Thomas R. Harvey","Andre Lukas","Jesse Thaler"],"reviews":0,"subject":"cs.LG","summary":"We introduce Sven (Singular Value dEsceNt), a new optimization algorithm for neural networks that exploits the natural decomposition of loss functions into a sum over individual data points, rather than reducing the full loss to a single scalar before computing a parameter update. Sven treats each data point's residual as a separate condition to be satisfied simultaneously, using the Moore-Penrose pseudoinverse of the loss Jacobian to find the minimum-norm parameter update that best satisfies all conditions at once. In practice, this pseudoinverse is approximated via a truncated singular…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-04-01","reactions":{}}