{"id":"arxiv/2602.02987","url":"https://arxiv.org/abs/2602.02987","tags":["cs.DC","math.OC"],"score":0,"title":"Large-Scale LLM Inference with Heterogeneous Workloads: Prefill-Decode Contention and Asymptotically Optimal Control","domain":"cs.DC","source":"arxiv","authors":["Ruihan Lin","Zezhen Ding","Zean Han","Jiheng Zhang"],"reviews":0,"subject":"cs.DC","summary":"Large Language Models (LLMs) are rapidly becoming critical infrastructure for enterprise applications, driving unprecedented demand for GPU-based inference services. A key operational challenge arises from the two-phase nature of LLM inference: a compute-intensive \\emph{prefill} phase that processes user input, followed by a memory-bound \\emph{decode} phase that generates output tokens. When these phases share GPU resources, prefill tasks throttle the processing speed of concurrent decodes, creating state-dependent contention. This contention is further complicated by workload heterogeneity,…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-02-03","reactions":{}}