{"id":"arxiv/2607.11626","url":"https://arxiv.org/abs/2607.11626","tags":["math.OC"],"score":0,"title":"On the Policy Convergence of Policy Mirror Descent Methods","domain":"math.OC","source":"arxiv","authors":["Wenye Li","Ke Wei"],"reviews":0,"subject":"math.OC","summary":"We study the policy convergence of unregularized policy mirror descent (PMD) with arbitrary constant step sizes for finite discounted Markov decision processes. We focus on decomposable mirror maps of the form $h(p)=\\sum_a ψ(p(a))$, where $ψ$ satisfies standard Legendre-type assumptions. Under these conditions, we prove that the policy sequence generated by PMD converges in the policy domain to a limiting optimal policy, even when the optimal policy set is not a singleton. This result covers a broad class of commonly used mirror maps, including the squared Euclidean mirror map underlying…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-07-13","reactions":{}}