{"id":"arxiv/2602.14029","url":"https://arxiv.org/abs/2602.14029","tags":["stat.ML","cs.LG","math.ST"],"score":0,"title":"Why Self-Training Helps and Hurts: Denoising vs. Signal Forgetting","domain":"stat.ML","source":"arxiv","authors":["Mingqi Wu","Archer Y. Yang","Qiang Sun"],"reviews":0,"subject":"stat.ML","summary":"Iterative self-training (self-distillation) repeatedly refits a model on pseudo-labels generated by its own predictions. We study this procedure in overparameterized linear regression: an initial estimator is trained on noisy labels, and each subsequent iterate is trained on fresh covariates with noiseless pseudo-labels from the previous model. In the high-dimensional regime, we derive deterministic-equivalent recursions for the prediction risk and effective noise across iterations, and prove that the empirical quantities concentrate sharply around these limits. The recursion separates two…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-02-15","reactions":{}}