Continuous-Time Policy Gradient from Discrete Data: Convergence Analysis and Model-Free Algorithms

Martin Hernandez, Xinyan Qing, Mo Zhou, Yuhua Zhu

arXivarxiv/2610.28580Oct 20, 2026cs.LGscore 0

Continuous-time reinforcement learning seeks an optimal policy under unknown dynamics driven by stochastic differential equations using observations collected at discrete times. We develop a policy gradient framework that combines rigorous convergence analysis with learning from discrete data. We first establish a performance difference lemma and derive policy gradient formulas for randomized and parametrized policies. We then prove exponential convergence of an entropy-regularized policy flow to the regularized optimum. For discrete parametrized updates, we derive a convergence bound that…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2610.28580
curl -X POST /api/v1/records/arxiv/2610.28580/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2610.28580/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'