{"id":"arxiv/2604.08764","url":"https://arxiv.org/abs/2604.08764","tags":["cs.CL","math.DG"],"score":0,"title":"Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamics","domain":"cs.CL","source":"arxiv","authors":["Raphael Bernas","Fanny Jourdan","Antonin Poché","Céline Hudelot"],"reviews":0,"subject":"cs.CL","summary":"Since their introduction, Transformer architectures have dominated Natural Language Processing (NLP). However, recent research has highlighted an inherent anisotropy phenomenon in these models, presenting a significant challenge to their geometric interpretation. Previous theoretical studies on this phenomenon are rarely grounded in the underlying representation geometry. In this paper, we extend them by deriving geometric arguments for how frequency-biased sampling attenuates curvature visibility and why training preferentially amplify tangent directions. Empirically, we then use…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-04-09","reactions":{}}