{"id":"arxiv/2610.37388","url":"https://arxiv.org/abs/2610.37388","tags":["cs.LG","math.PR"],"score":0,"title":"Para-Gaussian Approximations of Random Deep Neural Networks","domain":"cs.LG","source":"arxiv","authors":["Mihai Nica","Janosch Ortmann"],"reviews":0,"subject":"cs.LG","summary":"It is known that randomly initialized neural networks have Gaussian behaviour in the $n\\to \\infty$ limit. Unfortunately, this approximation can be poor for finite $n$ and gets worse as one goes deeper into the network. In this work, we develop a more accurate approximation that is \\emph{slightly} non-Gaussian, which we call the para-Gaussian distribution and is constructed from the Hermite polynomials. We show how this para-Gaussian approximation can lead to a practical recursive method to approximate the deeper layers of the network. Finally, we apply this theory to linear neural networks…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-27","reactions":{}}