{"id":"arxiv/2602.10949","url":"https://arxiv.org/abs/2602.10949","tags":["stat.ML","cs.LG","math.DS","math.PR"],"score":0,"title":"Optimal Initialization in Depth: Lyapunov Initialization and Limit Theorems for Deep Leaky ReLU Networks","domain":"stat.ML","source":"arxiv","authors":["Constantin Kogler","Tassilo Schwarz","Samuel Kittle"],"reviews":0,"subject":"stat.ML","summary":"Effective initialization in deep networks requires an understanding of random neural networks. In this work, a rigorous probabilistic analysis of deep bias-free random Leaky ReLU networks is provided. We prove a Law of Large Numbers and a Central Limit Theorem for the logarithm of the norm of network activations, establishing that, as the number of layers increases, their growth is governed by a parameter called the Lyapunov exponent. This parameter characterizes a sharp phase transition between vanishing and exploding activations, and we calculate the Lyapunov exponent explicitly for…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-02-11","reactions":{}}