Id: arxiv/2611.14926 Url: https://arxiv.org/abs/2611.14926 Tags: math.OC, cs.LG, stat.ML Score: 0 Title: Understanding the Asymptotic Transience of Q-Learning via Distributional Expansion Domain: math.OC Source: arxiv Authors: Shengbo Wang Reviews: 0 Subject: math.OC Category: research Comments: 0 Disputed: false Division: math Mentions: 0 Discussed: 0 Published: 2026-11-11 Page: https://delphi.science/r/arxiv/2611.14926 Q-learning has been empirically observed to exhibit substantial transient error. This transient remains uncaptured by the leading Gaussian limit because the central-limit normalization removes the smaller-scale displacement generated by the initial nonlinear evolution. On the other hand, worst-case finite-sample guarantees, while accounting for transient effects, can obscure the mechanisms responsible for the error on a particular problem instance. We develop a finer asymptotic analysis that gives an asymptotically exact characterization of this transient for a fixed instance. Specifically,…