{"id":"arxiv/2605.10775","url":"https://arxiv.org/abs/2605.10775","tags":["math.OC","cs.LG"],"score":0,"title":"On the global convergence of gradient flow for wide shallow models beyond homogeneous nonlinearities","domain":"math.OC","source":"arxiv","authors":["Romain Petit","Clarice Poon","Gabriel Peyré"],"reviews":0,"subject":"math.OC","summary":"A surprising phenomenon in the training of neural networks is the ability of gradient descent to find global minimizers of the training loss despite its non-convexity. Following earlier work, we investigate this behavior for wide shallow models. Existing global convergence results primarily concern models with positively one-homogeneous nonlinearities, such as ReLU activations, and models with scalar output weights and bounded nonlinearities, such as sigmoid activations. We study a broader class of models, including multi-head attention layers and two-layer networks with bounded or…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-05-11","reactions":{}}