{"id":"arxiv/2605.24879","url":"https://arxiv.org/abs/2605.24879","tags":["cs.LG","math.OC"],"score":0,"title":"Efficient DP-SGD for LLMs with Randomized Clipping","domain":"cs.LG","source":"arxiv","authors":["Enayat Ullah","Sai Aparna Aketi","Devansh Gupta","Huanyu Zhang","Meisam Razaviyayn"],"reviews":0,"subject":"cs.LG","summary":"Large language models (LLMs) are trained on vast datasets that may contain sensitive information. Differential privacy (DP), the de facto standard for formal privacy guarantees, provides a principled framework for training LLMs with provable privacy protection. However, state-of-the-art DP training implementations rely on fast gradient clipping techniques with memory overhead $O(B \\min\\{T^2, d^2\\})$, where $B$ is the batch size, $T$ is the sequence length, and $d$ is the model width. This becomes prohibitive as both model size and context length grow. We propose DP-SGD-RC, a novel variant of…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-05-24","reactions":{}}