{"id":"arxiv/2609.02027","url":"https://arxiv.org/abs/2609.02027","tags":["cs.PF","math.PR"],"score":0,"title":"Multi-Turn LLM Conversations under the Least-Recently-Used Policy: Mean-Field Asymptotics and Hit Ratio Approximation","domain":"cs.PF","source":"arxiv","authors":["Heyuan Yao","Chutong Gao","Yuan Lyu","Izzy Grosof","David Simchi-Levi"],"reviews":0,"subject":"cs.PF","summary":"The major workloads in modern large language model (LLM) serving systems have shifted from single-shot LLM calls to multi-turn conversations, where new responses are generated based on the whole conversation history across all previous turns. The hit ratio, i.e., the average fraction of KV caches accessed directly from existing caches stored in high-bandwidth memory (HBM), is hence a crucial metric that governs system performance. Estimating the hit ratio is a highly nontrivial task due to the complex system dynamics, where the KV cache prefixes grow with turns and some must be evicted due…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-09-02","reactions":{}}