{"id":"arxiv/2603.20521","url":"https://arxiv.org/abs/2603.20521","tags":["cs.LG","cs.AI","math.OC","stat.ML"],"score":0,"title":"Delightful Distributed Policy Gradient","domain":"cs.LG","source":"arxiv","authors":["Ian Osband"],"reviews":0,"subject":"cs.LG","summary":"Distributed reinforcement learning trains on data from stale, buggy, or mismatched actors, producing actions with high surprisal (negative log-probability) under the learner's policy. The core difficulty is not surprising data per se, but \\emph{negative learning from surprising data}. High-surprisal failures can dominate finite-batch updates through large perpendicular components, while high-surprisal successes reveal opportunities the current policy would otherwise miss. The \\textit{Delightful Policy Gradient} (DG) separates these cases by gating each update with delight, the product of…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-03-20","reactions":{}}