{"id":"arxiv/2605.16622","url":"https://arxiv.org/abs/2605.16622","tags":["cs.LG","math.OC","stat.ML"],"score":0,"title":"Does Weight Decay Enhance Training Stability?","domain":"cs.LG","source":"arxiv","authors":["Marius Saether","Amir Kolic","Tomaso Poggio","Pierfrancesco Beneventano"],"reviews":0,"subject":"cs.LG","summary":"In modern deep learning, weight decay is often credited with \"stabilizing\" training dynamics, diverging from its classical role as a static regularization penalty. We investigate a fundamental question: *does weight decay stabilize training dynamics, and if so, through which mechanism?* Indeed, training stability is understood through different but related notions in the literature. We consider how weight decay affects the parameter-space dynamics and loss sharpness by analyzing its effects at the \\emph{Edge of Stability} (EoS). We show that weight decay robustly slows *progressive…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-05-15","reactions":{}}