{"id":"arxiv/2606.04476","url":"https://arxiv.org/abs/2606.04476","tags":["cs.LG","math.OC","math.ST","stat.ML"],"score":0,"title":"When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks","domain":"cs.LG","source":"arxiv","authors":["Berk Tinaz","Changzhi Xie","Mahdi Soltanolkotabi"],"reviews":0,"subject":"cs.LG","summary":"In this paper, we study the gradient descent dynamics for jointly training both layers of a one-hidden-layer ReLU network to fit a linear target function. Concretely, we consider a realizable setting where inputs are drawn i.i.d. from a Gaussian distribution and labels follow a planted linear model. This stylized framework captures salient features of end-to-end training in inverse problems and certain auto-encoder models. Despite its apparent simplicity, the dynamics remain poorly understood, in part because the loss landscape contains multiple non-strict saddle points, making it unclear…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-06-03","reactions":{}}