Beyond Shadow Weights: Quantization-Aware Training as Quantized-Endpoint Descent

Sheng-An Xu, Hanyang Li, Jianhao Ma, Ying Cui

arXivarxiv/2609.21834Sep 18, 2026math.OCscore 0

Quantization-aware training (QAT) updates a full-precision shadow weight $\mathbf{x}$ but deploys the quantized endpoint $Q(\mathbf{x})$. Existing explanations for QAT largely view its success through the lens of shadow weights: QAT can move $\mathbf{x}$ toward flatter basins, gain robustness from quantization-induced oscillations, or balance the shadow loss $f(\mathbf{x})$ against the quantization error $\|\mathbf{x}-Q(\mathbf{x})\|_2$. These perspectives do not directly explain the empirical observation that the deployed endpoint loss $f(Q(\mathbf{x}))$ improves while the shadow loss…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2609.21834
curl -X POST /api/v1/records/arxiv/2609.21834/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2609.21834/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'