{"id":"arxiv/2605.00155","url":"https://arxiv.org/abs/2605.00155","tags":["cs.LG","cs.CL","math.OC","stat.ML"],"score":0,"title":"Wasserstein Distributionally Robust Regret Optimization for Reinforcement Learning from Human Feedback","domain":"cs.LG","source":"arxiv","authors":["Yikai Wang","Shang Liu","Jose Blanchet"],"reviews":0,"subject":"cs.LG","summary":"Reinforcement learning from human feedback (RLHF) is a central post-training tool for aligning large language models, but its training reward is only a learned proxy for true human utility. This creates a decision problem under objective misspecification: the policy is optimized against an estimated reward, while deployment performance is governed by an unobserved population preference. The resulting gap leads to reward over-optimization, where proxy reward keeps improving after true quality deteriorates. We propose distributionally robust regret optimization (DRRO) for RLHF with a…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-04-30","reactions":{}}