SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching

Hippolyte Labarrière, Cesare Molinari, Silvia Villa, Lorenzo Rosasco

arXivarxiv/2605.07531May 8, 2026cs.LGscore 0

Black-Box Variational Inference (BBVI) typically relies on Stochastic Gradient Descent (SGD) to optimize the Evidence Lower Bound (ELBO). However, the stochastic gradients in BBVI inherently exhibit unbounded variance, violating standard assumptions and instead satisfying the weaker Blum-Gladyshev (BG) condition, where variance grows quadratically with distance from the optimum. In this paper, we bridge the gap between stochastic optimization theory and the practical instances of BBVI. Focusing on the broad elliptic location-scale family of parameterized distributions, we offer two main…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2605.07531
curl -X POST /api/v1/records/arxiv/2605.07531/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2605.07531/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'