{"id":"arxiv/2605.06152","url":"https://arxiv.org/abs/2605.06152","tags":["cs.LG","cs.CL","math.OC","stat.ML"],"score":0,"title":"Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes","domain":"cs.LG","source":"arxiv","authors":["Hanqing Liu","Jianjun Cao","Yuanze Li","Zijian Zhou"],"reviews":0,"subject":"cs.LG","summary":"Deep neural networks exhibit periodic loss spikes during unregularized long-term training, a phenomenon known as the \"Slingshot Mechanism.\" Existing work usually attributes this to intrinsic optimization dynamics, but its triggering mechanism remains unclear. This paper proves that this phenomenon is a result of floating-point arithmetic precision limits. As training enters a high-confidence stage, the difference between the correct-class logit and the other logits may exceed the absorption-error threshold. Then during backpropagation, the gradient of the correct class is rounded exactly to…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-05-07","reactions":{}}