{"id":"arxiv/2605.09887","url":"https://arxiv.org/abs/2605.09887","tags":["cs.LG","cs.AI","math.DG"],"score":0,"title":"The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws","domain":"cs.LG","source":"arxiv","authors":["Eslam Zaher","Maciej Trzaskowski","Quan Nguyen","Fred Roosta"],"reviews":0,"subject":"cs.LG","summary":"Sparse autoencoders (SAEs) operationalise the linear representation hypothesis: they reconstruct model activations as sparse linear combinations of interpretable dictionary atoms, on the implicit assumption that activation space is well approximated by a globally linear structure. Their reconstruction error varies sharply across layers in ways that existing scaling laws, fitted at single layers, do not explain. We argue that this variation is the empirical trace of a geometric mismatch: where the activation manifold is curved and its intrinsic dimension varies across layers, no sparse linear…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-05-11","reactions":{}}