{"id":"arxiv/2605.02124","url":"https://arxiv.org/abs/2605.02124","tags":["cs.LG","cs.AI","math.PR"],"score":0,"title":"Soft-to-Hard Routing in Sparse Mixture-of-Experts Models","domain":"cs.LG","source":"arxiv","authors":["Reza Rastegar"],"reviews":0,"subject":"cs.LG","summary":"Softmax routing approaches hard top-1 routing as the temperature tends to zero, but the limiting passage is singular at router ties. This paper develops a boundary-layer calculus for this soft-to-hard limit in population squared-loss mixture-of-experts regression. For a router with logits $a_k(x;φ)$, the relevant local quantity is the top-two margin $Δ(x;φ)$, and the relevant global quantity is the boundary mass $\\mathbb{P}(Δ(X;φ)\\le w)$. Under smoothness and transversality assumptions, coarea and tubular-neighborhood estimates show how this mass scales with the slab width; in the binary…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-05-04","reactions":{}}