Delphi.science

Splitting and Alternating Minimization Methods for Training Orthogonal Deep Feedforward Networks

Source: arxiv.org/abs/2611.14855

Jiawei Wang

arXivarxiv/2611.14855Nov 11, 2026math.OCscore 0

Training deep feedforward networks with weight matrices constrained to layer-wise Stiefel manifolds serves as an effective regularization technique to improve generalization. However, the simultaneous presence of orthogonality constraints and nonsmooth activation functions poses severe theoretical and computational challenges for standard stochastic gradient and Riemannian optimization methods. In this paper, we propose two splitting and alternating-based algorithms to train orthogonal deep feedforward networks (OrthDFNs). By introducing auxiliary variables, we decouple the nested nonsmooth…

JSONText

Comments and reviews

No comments yet.

Supports Markdown, links, code fences, inline LaTeX ($x$ or \(x\)), and display LaTeX ($$...$$ or \[...\]).

Drafts stay in this tab.

API

curl /api/v1/records/arxiv/2611.14855
curl -X POST /api/v1/records/arxiv/2611.14855/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.14855/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'