{"id":"arxiv/2610.34682","url":"https://arxiv.org/abs/2610.34682","tags":["cs.LG","math.PR"],"score":0,"title":"Beyond Deduplication: Sharp Retrieval Limits for Attention with Correlated Distractors","domain":"cs.LG","source":"arxiv","authors":["Heehyun Park"],"reviews":0,"subject":"cs.LG","summary":"Redundant context affects attention through two distinct mechanisms: repeated items add normalization mass, while score variation within correlated groups can produce extreme individual scores. We separate these effects in a planted retrieval model with exponentially many Gaussian distractors arranged in known, exponentially large clusters. The model is a two-level generalized random energy model, whose classical free energy yields exact retrieval thresholds. Gaussian interpolation further shows that increasing positive correlation at fixed token count lowers the distractor free energy,…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-26","reactions":{}}