Topology and Geometry of the Learning Space of ReLU Networks: Connectivity and Singularities

Marco Nurisso, Pierrick Leroy, Giovanni Petri, Francesco Vaccarino

arXivarxiv/2602.00693Jan 31, 2026cs.LGscore 0

Understanding the properties of the parameter space in feed-forward ReLU networks is critical for effectively analyzing and guiding training dynamics. After initialization, training under gradient flow decisively restricts the parameter space to an algebraic variety that emerges from the homogeneous nature of the ReLU activation function. In this study, we examine two key challenges associated with feed-forward ReLU networks built on general directed acyclic graph (DAG) architectures: the (dis)connectedness of the parameter space and the existence of singularities within it. We extend…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2602.00693
curl -X POST /api/v1/records/arxiv/2602.00693/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2602.00693/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'