{"id":"arxiv/2602.17744","url":"https://arxiv.org/abs/2602.17744","tags":["cs.LG","cs.CL","math.ST","stat.ML"],"score":0,"title":"Bayesian Optimality of In-Context Learning with Selective State Spaces","domain":"cs.LG","source":"arxiv","authors":["Di Zhang","Jiaqi Xing"],"reviews":0,"subject":"cs.LG","summary":"We propose Bayesian optimal sequential prediction as a new principle for understanding in-context learning (ICL). Unlike interpretations framing Transformers as performing implicit gradient descent, we formalize ICL as meta-learning over latent sequence tasks. For tasks governed by Linear Gaussian State Space Models (LG-SSMs), we prove a meta-trained selective SSM asymptotically implements the Bayes-optimal predictor, converging to the posterior predictive mean. We further establish a statistical separation from gradient descent, constructing tasks with temporally correlated noise where the…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-02-19","reactions":{}}