{"id":"arxiv/2601.05444","url":"https://arxiv.org/abs/2601.05444","tags":["math.ST","cs.LG","stat.ML"],"score":0,"title":"What Functions Does XGBoost Learn?","domain":"math.ST","source":"arxiv","authors":["Dohyeong Ki","Adityanand Guntuboyina"],"reviews":0,"subject":"math.ST","summary":"This paper establishes a rigorous theoretical foundation for the function class implicitly learned by XGBoost, bridging the gap between its empirical success and our theoretical understanding. We introduce an infinite-dimensional function class $\\mathcal{F}^{d, s}_{\\infty-\\text{ST}}$ that extends finite ensembles of bounded-depth regression trees, together with a complexity measure $V^{d, s}_{\\infty-\\text{XGB}}(\\cdot)$ that generalizes the $L^1$ regularization penalty used in XGBoost. We show that every optimizer of the XGBoost objective is also an optimizer of an equivalent penalized…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-01-09","reactions":{}}