{"id":"arxiv/2610.01843","url":"https://arxiv.org/abs/2610.01843","tags":["math.OC","cs.LG"],"score":0,"title":"Optimal Stochastic Bilevel Optimization with First-Order Oracles","domain":"math.OC","source":"arxiv","authors":["Linxuan Pan","Junchi Yang"],"reviews":0,"subject":"math.OC","summary":"We study nonconvex--strongly-convex bilevel optimization under a stochastic first-order oracle. We introduce MRT-FD, a single-loop first-order method that simultaneously tracks the upper-level variable, the lower-level solution, and the auxiliary response arising from implicit differentiation of the hyperobjective. MRT-FD performs one update of each variable per iteration and approximates the second-order derivative actions using order-$p$ finite differences. For any fixed finite smoothness order $p\\ge1$ in the lower-level variable, MRT-FD finds an $\\varepsilon$-stationary point using…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-10-01","reactions":{}}