{"id":"arxiv/2606.17013","url":"https://arxiv.org/abs/2606.17013","tags":["math.OC","cs.LG"],"score":0,"title":"Exploding and vanishing gradients in deep neural networks: the effect of residual connections","domain":"math.OC","source":"arxiv","authors":["Vivek S Borkar"],"reviews":0,"subject":"math.OC","summary":"The well known phenomenon of exploding and vanishing gradients in deep neural networks is analyzed using multiplicative ergodic theory. The effect of adding a residual connection is explained in this context. Specifically, a characterization of Liapunov exponents due to Furstenberg and Kifer is exploited in order to make a precise statement about the Liapunov spectrum and the effect of residual connections on it.","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-06-15","reactions":{}}