Constrained Nonconvex Stochastic Optimization with One Projection

Yuyang Deng, Mohammadreza M. Kalan, Eitan J. Neugut, Mehrdad Mahdavi

arXivarxiv/2609.34099Sep 28, 2026math.OCscore 0

Constrained nonconvex optimization has seen increasing application in modern machine learning, such as safe LLM alignment/finetuning, transfer learning and rank-constrained continual learning. The commonly used approach is Projected SGD which requires projection at every iteration. However, projection onto a functional constraint can cost substantially more than a stochastic first-order update. We study whether this operation can be deferred until the end of nonconvex stochastic optimization, and only do it once. For weakly convex, possibly nonsmooth objectives with regular convex…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2609.34099
curl -X POST /api/v1/records/arxiv/2609.34099/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2609.34099/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'