{"id":"arxiv/2602.06146","url":"https://arxiv.org/abs/2602.06146","tags":["cs.LG","math.OC","stat.ML"],"score":0,"title":"Optimistic Training and Convergence of Q-Learning -- Extended Version","domain":"cs.LG","source":"arxiv","authors":["Prashant Mehta","Sean Meyn"],"reviews":0,"subject":"cs.LG","summary":"In recent work it is shown that Q-learning with linear function approximation is stable, in the sense of bounded parameter estimates, under the $(\\varepsilon,κ)$-tamed Gibbs policy; $κ$ is inverse temperature, and $\\varepsilon>0$ is introduced for additional exploration. Under these assumptions it also follows that there is a solution to the projected Bellman equation (PBE). Left open is uniqueness of the solution, and criteria for convergence outside of the standard tabular or linear MDP settings. The present work extends these results to other variants of Q-learning, and clarifies prior…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-02-05","reactions":{}}