{"id":"arxiv/2601.14175","url":"https://arxiv.org/abs/2601.14175","tags":["cs.LG","cs.AI","cs.CL","hep-th"],"score":0,"title":"A model of errors in transformers","domain":"cs.LG","source":"arxiv","authors":["Suvrat Raju","Praneeth Netrapalli"],"reviews":0,"subject":"cs.LG","summary":"We study the error rate of LLMs on tasks like arithmetic that require a deterministic output, and repetitive processing of tokens drawn from a small set of alternatives. We argue that incorrect predictions arise when small errors in the attention mechanism accumulate to cross a threshold, and use this insight to derive a quantitative two-parameter relationship between the accuracy and the complexity of the task. The two parameters vary with the prompt and the model; they can be interpreted in terms of an elementary noise rate, and the number of plausible erroneous tokens that can be…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-01-20","reactions":{}}