{"id":"arxiv/2602.02855","url":"https://arxiv.org/abs/2602.02855","tags":["cs.LG","cond-mat.dis-nn","math.ST"],"score":0,"title":"When pre-training hurts LoRA fine-tuning: a dynamical analysis via single-index models","domain":"cs.LG","source":"arxiv","authors":["Gibbs Nwemadji","Bruno Loureiro","Jean Barbier"],"reviews":0,"subject":"cs.LG","summary":"Pre-training on a source task is usually expected to facilitate fine-tuning on similar downstream problems. In this work, we mathematically show that this naive intuition is not always true: excessive pre-training can computationally slow down fine-tuning optimization. We study this phenomenon for low-rank adaptation (LoRA) fine-tuning on single-index models trained under one-pass SGD. Leveraging a summary statistics description of the fine-tuning dynamics, we precisely characterize how the convergence rate depends on the initial fine-tuning alignment and the degree of non-linearity of the…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-02-02","reactions":{}}