{"id":"arxiv/2608.09558","url":"https://arxiv.org/abs/2608.09558","tags":["cs.LG","math.NA","math.ST"],"score":0,"title":"Training-Free Universal Approximation by Prompting Random Transformers","domain":"cs.LG","source":"arxiv","authors":["Alexander Hsu","Rongjie Lai"],"reviews":0,"subject":"cs.LG","summary":"How expressive is prompting a transformer? Answering this question is important for separating the roles of prompting, architecture, and pretraining in transformer models, and for determining whether task-specific behavior must be stored in model weights or can instead be induced at inference time through the prompt. We show, in an approximation-theoretic sense, that pretraining is optional: a single-layer softmax attention network with random, untrained weights can approximate any Hölder function on a compact manifold when steered by an appropriate soft prompt. Guided by the connection…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-08-10","reactions":{}}