{"id":"arxiv/2602.12390","url":"https://arxiv.org/abs/2602.12390","tags":["cs.LG","cs.AI","math.NA"],"score":0,"title":"Rational Neural Networks have Expressivity Advantages","domain":"cs.LG","source":"arxiv","authors":["Maosen Tang","Alex Townsend"],"reviews":0,"subject":"cs.LG","summary":"We study neural networks with trainable low-degree rational activation functions and show that they are more expressive and parameter-efficient than modern piecewise-linear and smooth activations such as ELU, LeakyReLU, LogSigmoid, PReLU, ReLU, SELU, CELU, Sigmoid, SiLU, Mish, Softplus, Tanh, Softmin, Softmax, and LogSoftmax. For an error target of $\\varepsilon>0$, we establish approximation-theoretic separations: Any network built from standard fixed activations can be uniformly approximated on compact domains by a rational-activation network with only…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-02-12","reactions":{}}