{"id":"arxiv/2601.13458","url":"https://arxiv.org/abs/2601.13458","tags":["stat.ML","cs.AI","cs.LG","math.ST"],"score":0,"title":"Labels or Preferences? Budget-Constrained Learning with Human Judgments over AI-Generated Outputs","domain":"stat.ML","source":"arxiv","authors":["Zihan Dong","Xiaotian Hou","Ruijia Wu","Linjun Zhang"],"reviews":0,"subject":"stat.ML","summary":"The increasing reliance on human preference feedback to judge AI-generated pseudo labels has created a pressing need for principled, budget-conscious data acquisition strategies. We address the crucial question of how to optimally allocate a fixed annotation budget between ground-truth labels and pairwise preferences in AI. Our solution, grounded in semi-parametric inference, casts the budget allocation problem as a monotone missing data framework. Building on this formulation, we introduce Preference-Calibrated Active Learning (PCAL), a novel method that learns the optimal data acquisition…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-01-19","reactions":{}}