{"id":"arxiv/2603.03523","url":"https://arxiv.org/abs/2603.03523","tags":["cs.LG","math.OC"],"score":0,"title":"Q-Measure-Learning for Continuous State RL: Efficient Implementation and Convergence","domain":"cs.LG","source":"arxiv","authors":["Shengbo Wang"],"reviews":0,"subject":"cs.LG","summary":"We study reinforcement learning in infinite-horizon discounted Markov decision processes with continuous state spaces, where data are generated online from a single trajectory under a Markovian behavior policy. To avoid maintaining an infinite-dimensional, function-valued estimate, we propose the novel Q-Measure-Learning, which learns a signed empirical measure supported on visited state-action pairs and reconstructs an action-value estimate via kernel integration. The method jointly estimates the stationary distribution of the behavior chain and the Q-measure through coupled stochastic…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-03-03","reactions":{}}