{"id":"arxiv/2606.23235","url":"https://arxiv.org/abs/2606.23235","tags":["math.OC","math.DS","stat.ML"],"score":0,"title":"A First-Order Mean Field Control Analysis of Transformer Layers under Cross-Entropy Training","domain":"math.OC","source":"arxiv","authors":["Cheng Huan","Hongwei Yuan"],"reviews":0,"subject":"math.OC","summary":"We study Transformer-type residual layers under cross-entropy training through a continuous-depth mean field control viewpoint. Depth is treated as time, layer parameters as controls, and the residual Transformer recursion as an explicit Euler scheme for a controlled hidden-state flow. For fixed controls, we prove an $O(\\varepsilon)$ pathwise approximation of finite-depth trajectories by the continuous flow and combine this with high-probability sampling bounds for the empirical cross-entropy risk. We formulate the limiting population problem as a first-order transport control problem for…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-06-22","reactions":{}}