{"id":"arxiv/2602.03061","url":"https://arxiv.org/abs/2602.03061","tags":["cs.LG","cs.AI","math.ST","stat.ME","stat.ML"],"score":0,"title":"Evaluating LLMs When They Do Not Know the Answer: Statistical Evaluation of Mathematical Reasoning via Comparative Signals","domain":"cs.LG","source":"arxiv","authors":["Zihan Dong","Zhixian Zhang","Yang Zhou","Can Jin","Ruijia Wu","Linjun Zhang"],"reviews":0,"subject":"cs.LG","summary":"Evaluating mathematical reasoning in LLMs is constrained by limited benchmark sizes and inherent model stochasticity, yielding high-variance accuracy estimates and unstable rankings across platforms. On difficult problems, an LLM may fail to produce a correct final answer, yet still provide reliable pairwise comparison signals indicating which of two candidate solutions is better. We leverage this observation to design a statistically efficient evaluation framework that combines standard labeled outcomes with pairwise comparison signals obtained by having models judge auxiliary reasoning…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-02-03","reactions":{}}