{"id":"arxiv/2609.18656","url":"https://arxiv.org/abs/2609.18656","tags":["cs.LG","math.OC","stat.ML"],"score":0,"title":"Revisiting Distributed Sign-Based Variance Reduction","domain":"cs.LG","source":"arxiv","authors":["Wei Jiang","Zechao Li","Lijun Zhang"],"reviews":0,"subject":"cs.LG","summary":"Sign-based methods reduce communication costs in distributed environments, but aggregating local signs can introduce bias when data are heterogeneous. As a result, existing sign-based variance reduction methods fail to obtain the optimal convergence rates. In this paper, we solve this problem and obtain optimal rates for both nonconvex stochastic and finite-sum optimization. We first give a counterexample showing that majority voting can fail to approach stationary points even with exact local gradients. Motivated by this limitation, we propose tracking the global gradient at the server…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-09-16","reactions":{}}