{"id":"arxiv/2601.15014","url":"https://arxiv.org/abs/2601.15014","tags":["stat.ML","cs.LG","math.ST"],"score":0,"title":"Efficient and Minimax Optimal In-context Nonparametric Regression with Transformers","domain":"stat.ML","source":"arxiv","authors":["Michelle Ching","Ioana Popescu","Nico Smith","Tianyi Ma","William G. Underwood","Richard J. Samworth"],"reviews":0,"subject":"stat.ML","summary":"We study in-context learning for nonparametric regression with $α$-Hölder smooth regression functions, for some $α>0$. We prove that, with $n$ in-context examples and $d$-dimensional regression covariates, a pretrained transformer with $Θ(\\log n)$ parameters and $Ω\\bigl(n^{2α/(2α+d)}\\log^3 n\\bigr)$ pretraining sequences can achieve the minimax optimal rate of convergence $O\\bigl(n^{-2α/(2α+d)}\\bigr)$ in mean squared error. Our result requires substantially fewer transformer parameters and pretraining sequences than previous results in the literature. This is achieved by showing that…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-01-21","reactions":{}}