{"id":"arxiv/2605.17660","url":"https://arxiv.org/abs/2605.17660","tags":["math.OC","cs.AI","cs.LG","stat.ML"],"score":0,"title":"Training Infinitely Deep and Wide Transformers","domain":"math.OC","source":"arxiv","authors":["Raphaël Barboni","Maarten V. de Hoop","Takashi Furuya","Gabriel Peyré"],"reviews":0,"subject":"math.OC","summary":"Transformers have become the dominant architecture in modern machine learning, yet the theoretical understanding of their training dynamics remains limited. This paper develops a rigorous mathematical framework for analyzing gradient-based training of transformers in the mean-field regime, where both the depth (number of layers) and width (number of attention heads) tend to infinity. While ResNet training can be understood as controlling a neural ODE, transformer training corresponds to controlling a neural PDE, due to the coupling of multiple token distributions through the attention…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-05-17","reactions":{}}