{"id":"arxiv/2602.02224","url":"https://arxiv.org/abs/2602.02224","tags":["cs.LG","cs.AI","math.SP","stat.ML"],"score":0,"title":"Spectral Superposition: A Theory of Feature Geometry","domain":"cs.LG","source":"arxiv","authors":["Georgi Ivanov","Narmeen Oozeer","Shivam Raval","Tasana Pejovic","Shriyash Upadhyay","Amir Abdullah"],"reviews":0,"subject":"cs.LG","summary":"Neural networks represent more features than they have dimensions via superposition, forcing features to share representational space. Current methods decompose activations into sparse linear features but discard geometric structure. We develop a theory for studying the geometric structre of features by analyzing the spectra (eigenvalues, eigenspaces, etc.) of weight derived matrices. In particular, we introduce the frame operator $F = WW^\\top$, which gives us a spectral measure that describes how each feature allocates norm across eigenspaces. While previous tools could describe the…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-02-02","reactions":{}}