{"id":"arxiv/2604.09175","url":"https://arxiv.org/abs/2604.09175","tags":["cs.LG","cs.AI","math.ST","stat.ML"],"score":0,"title":"Generalization and Scaling Laws for Mixture-of-Experts Transformers","domain":"cs.LG","source":"arxiv","authors":["Mansour Zoubeirou a Mayaki"],"reviews":0,"subject":"cs.LG","summary":"We develop a theory of generalization and scaling for Mixture-of-Experts (MoE) Transformers that cleanly separates \\emph{active} per-input capacity from routing combinatorics. By conditioning on fixed routing patterns and union-bounding across them, we derive a sup-norm covering-number bound whose metric entropy scales with the active parameter budget and incurs a MoE-specific routing overhead. Combined with a standard ERM analysis for squared loss, this yields a generalization bound under a $d$-dimensional manifold data model and $C^β$ targets, showing that approximation and estimation…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-04-10","reactions":{}}