Muon in Associative Memory Learning: Training Dynamics and Scaling Laws

Binghui Li, Kaifei Wang, Han Zhong, Pinyan Lu, Liwei Wang

arXivarxiv/2602.05725Feb 5, 2026cs.LGscore 0

Muon updates matrix parameters via the matrix sign of the gradient and has shown strong empirical gains, yet its dynamics and scaling behavior remain unclear in theory. We study Muon in a linear associative memory model with softmax retrieval and a hierarchical frequency spectrum over query-answer pairs, with and without label noise. In this setting, we show that Gradient Descent (GD) learns frequency components at highly imbalanced rates, leading to slow convergence bottlenecked by low-frequency components. In contrast, the Muon optimizer mitigates this imbalance, leading to faster and more…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2602.05725
curl -X POST /api/v1/records/arxiv/2602.05725/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2602.05725/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'