{"id":"arxiv/2602.17565","url":"https://arxiv.org/abs/2602.17565","tags":["math.ST","cs.LG","stat.ML"],"score":0,"title":"Optimal Unconstrained Self-Distillation in Ridge Regression: Strict Improvements, Precise Asymptotics, and One-Shot Tuning","domain":"math.ST","source":"arxiv","authors":["Hien Dang","Pratik Patil","Alessandro Rinaldo"],"reviews":0,"subject":"math.ST","summary":"Self-distillation (SD) is the process of retraining a student on a mixture of ground-truth labels and the teacher's own predictions using the same architecture and training data. Although SD has been empirically shown to often improve generalization, its formal guarantees remain limited. We study SD for ridge regression in unconstrained setting in which the mixing weight $ξ$ may be outside the unit interval. Conditioned on the training data and without any distributional assumptions, we prove that for any squared prediction risk (including out-of-distribution), the optimally mixed student…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-02-19","reactions":{}}