{"id":"arxiv/2603.10960","url":"https://arxiv.org/abs/2603.10960","tags":["cs.LG","math.ST"],"score":0,"title":"Ranking Reasoning LLMs under Test-Time Scaling","domain":"cs.LG","source":"arxiv","authors":["Mohsen Hariri","Michael Hinczewski","Jing Ma","Vipin Chaudhary"],"reviews":0,"subject":"cs.LG","summary":"Test-time scaling evaluates reasoning LLMs by sampling multiple outputs per prompt, but ranking models in this regime remains underexplored. We formalize dense benchmark ranking under test-time scaling and introduce Scorio, a library that implements statistical ranking methods such as paired-comparison models, item response theory (IRT) models, voting rules, and graph- and spectral-based methods. Across $20$ reasoning models on four Olympiad-style math benchmarks (AIME'24, AIME'25, HMMT'25, and BrUMO'25; up to $N=80$ trials), most full-trial rankings agree closely with the Bayesian gold…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-03-11","reactions":{}}