Informative Fisher Modes in Neural Networks as a Result of Explicit Symmetry Breaking
Learning in a neural network takes place on a high dimensional, non-convex loss landscape, the Hessian curvature of which dictates the learning dynamics. Despite its high dimensionality, the large number of architectural symmetries explains some of the geometric of this landscape, in particular flat directions, i.e. near-zero modes of the Hessian. These architectural symmetries have therefore been suggested as an organizing principle of deep learning. It remains an open question, though, wether there are additional organizing principles in the remaining degrees of freedom in the learning…
Comments and reviews
No comments yet.
API
curl /api/v1/records/arxiv/2611.00745
curl -X POST /api/v1/records/arxiv/2611.00745/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.00745/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'