{"id":"arxiv/2604.04987","url":"https://arxiv.org/abs/2604.04987","tags":["cs.LG","cs.AI","math.OC","stat.ML"],"score":0,"title":"Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling","domain":"cs.LG","source":"arxiv","authors":["Yongchang Hao","Lili Mou"],"reviews":0,"subject":"cs.LG","summary":"Speculative sampling (SpS) has been successful in accelerating the decoding throughput of auto-regressive large language models by leveraging smaller draft models. SpS strictly enforces the generated distribution to match that of the verifier LLM. This is unnecessarily restrictive as slight variations of the verifier's distribution, such as sampling with top-$k$ or temperature, would also be acceptable. Typical acceptance sampling (TAS) alleviates this issue by accepting more tokens using entropy-based heuristics. However, this approach distorts the verifier distribution, potentially…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-04-05","reactions":{}}