Sven: Singular Value Descent as a Computationally Efficient Natural Gradient Method

Samuel Bright-Thonney, Thomas R. Harvey, Andre Lukas, Jesse Thaler

arXivarxiv/2604.01279Apr 1, 2026cs.LGscore 0

We introduce Sven (Singular Value dEsceNt), a new optimization algorithm for neural networks that exploits the natural decomposition of loss functions into a sum over individual data points, rather than reducing the full loss to a single scalar before computing a parameter update. Sven treats each data point's residual as a separate condition to be satisfied simultaneously, using the Moore-Penrose pseudoinverse of the loss Jacobian to find the minimum-norm parameter update that best satisfies all conditions at once. In practice, this pseudoinverse is approximated via a truncated singular…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2604.01279
curl -X POST /api/v1/records/arxiv/2604.01279/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2604.01279/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'