{"id":"arxiv/2601.00151","url":"https://arxiv.org/abs/2601.00151","tags":["cs.LG","math.OC"],"score":0,"title":"Reinforcement Learning with Function Approximation for Non-Markov Processes","domain":"cs.LG","source":"arxiv","authors":["Ali Devran Kara"],"reviews":0,"subject":"cs.LG","summary":"We study reinforcement learning methods with linear function approximation under non-Markov state and cost processes. We first consider the policy evaluation method and show that the algorithm converges under suitable ergodicity conditions on the underlying non-Markov processes. Furthermore, we show that the limit corresponds to the fixed point of a joint operator composed of an orthogonal projection and the Bellman operator of an auxiliary \\emph{Markov} decision process. For Q-learning with linear function approximation, as in the Markov setting, convergence is not guaranteed in general. We…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-01-01","reactions":{}}