Beyond ECE: Calibrated Size Ratio, Risk Assessment, and Confidence-Weighted Metrics

Fernando Martin-Maroto, Nabil Abderrahaman, Gonzalo G. de Polavieja

arXivarxiv/2605.01796May 3, 2026cs.LGscore 0

Confidence calibration has been dominated by the Expected Calibration Error (ECE), a linear metric that counts calibration offset equally regardless of the confidence level at which it occurs. We show that ECE can remain small even under arbitrarily large overconfidence risk, so we propose Calibrated Size Ratio (CSR) instead, an interpretable metric that equals 1 under perfect calibration, from which we derive the risk probability $P_{\mathrm{risk}}$ that quantifies the statistical evidence for overconfidence. We further argue that overconfidence risk assessment must be complemented by a…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2605.01796
curl -X POST /api/v1/records/arxiv/2605.01796/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2605.01796/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'