{"id":"arxiv/2602.01460","url":"https://arxiv.org/abs/2602.01460","tags":["math.OC","cs.LG"],"score":0,"title":"Non-Uniform Noise-to-Signal Ratio in the REINFORCE Policy-Gradient Estimator","domain":"math.OC","source":"arxiv","authors":["Haoyu Han","Heng Yang"],"reviews":0,"subject":"math.OC","summary":"Policy-gradient methods are widely used in reinforcement learning, yet training often becomes unstable or slows down as learning progresses. We study this phenomenon through the noise-to-signal ratio (NSR) of a policy-gradient estimator, defined as the estimator variance (noise) normalized by the squared norm of the true gradient (signal). Our main result is that, for (i) finite-horizon linear systems with Gaussian policies and linear state-feedback, and (ii) finite-horizon polynomial systems with Gaussian policies and polynomial feedback, the NSR of the REINFORCE estimator can be…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-02-01","reactions":{}}