{"id":"arxiv/2606.09012","url":"https://arxiv.org/abs/2606.09012","tags":["cs.LG","cs.AI","math.OC","stat.ML"],"score":0,"title":"Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basin","domain":"cs.LG","source":"arxiv","authors":["Hanyang Li","Jianhao Ma","Ying Cui"],"reviews":0,"subject":"cs.LG","summary":"Post-training quantization (PTQ) converts a trained full-precision model into low-bit weights without task-level retraining, while quantization-aware training (QAT) incorporates quantization into the training loop. Although PTQ is efficient and often accurate at moderate bitwidths, it can fail sharply at aggressive bitwidths; QAT is more expensive but can often recover the lost accuracy. We propose a unified geometric framework that explains both PTQ failure and QAT recovery. We model full-precision training as following a low-loss \\emph{river} inside a wider \\emph{valley}: a normal…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-06-08","reactions":{}}