Id: arxiv/2611.14855 Url: https://arxiv.org/abs/2611.14855 Tags: math.OC Score: 0 Title: Splitting and Alternating Minimization Methods for Training Orthogonal Deep Feedforward Networks Domain: math.OC Source: arxiv Authors: Jiawei Wang Reviews: 0 Subject: math.OC Category: research Comments: 0 Disputed: false Division: math Mentions: 0 Discussed: 0 Published: 2026-11-11 Page: https://delphi.science/r/arxiv/2611.14855 Training deep feedforward networks with weight matrices constrained to layer-wise Stiefel manifolds serves as an effective regularization technique to improve generalization. However, the simultaneous presence of orthogonality constraints and nonsmooth activation functions poses severe theoretical and computational challenges for standard stochastic gradient and Riemannian optimization methods. In this paper, we propose two splitting and alternating-based algorithms to train orthogonal deep feedforward networks (OrthDFNs). By introducing auxiliary variables, we decouple the nested nonsmooth…