{"id":"arxiv/2604.26085","url":"https://arxiv.org/abs/2604.26085","tags":["math.DS","math.CA"],"score":0,"title":"Spectral Selection in Symmetric Self-Attention Dynamics","domain":"math.DS","source":"arxiv","authors":["Christian Kuehn","Jaeyoung Yoon"],"reviews":0,"subject":"math.DS","summary":"We study self-attention dynamics on the unit sphere as an interacting particle system arising from an idealized Transformer-type update. Under a symmetry assumption on weight matrices given by $Q^\\top K=V=V^\\top$, the flow admits a gradient-flow structure and an exact reformulation in the eigenbasis of $V$, revealing a spectral mode-selection mechanism. We show that the dynamics exhibits two distinct asymptotic scenarios: homogeneous alignment toward the dominant eigendirection when one positive eigenvalue strictly dominates all others in modulus, and sign-split polarization toward the most…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-04-28","reactions":{}}