{"id":"arxiv/2601.22307","url":"https://arxiv.org/abs/2601.22307","tags":["cs.LG","math.NA"],"score":0,"title":"Exact Gaussian Moment Matching for Residual Networks: a Second-Order Method","domain":"cs.LG","source":"arxiv","authors":["Simon Kuang","Xinfan Lin"],"reviews":0,"subject":"cs.LG","summary":"We study the problem of propagating the mean and covariance of a general multivariate Gaussian distribution through a deep (residual) neural network using layer-by-layer moment matching. We close a longstanding gap by deriving exact moment matching for the probit, GeLU, ReLU (as a limit of GeLU), Heaviside (as a limit of probit), and sine activation functions; for both feedforward and generalized residual layers. On random networks, we find orders-of-magnitude improvements in the KL divergence error metric, up to a millionfold, over popular alternatives. On a variational Bayes neural…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-01-29","reactions":{}}