Pathwise-Budgeted LLM Routing: Online Allocation with K-Bit Tail-Cost Learning

Source: arxiv.org/abs/2611.05835

Hanzhang Qin, Sophie H. Yu, Yuewei Zhang, Zhi-Hai Zhang

arXivarxiv/2611.05835Nov 4, 2026cs.LGscore 0

Firms using large language models under fixed budgets must decide which model to call and whether to retry a failed task. Because a call's charge is revealed only after completion, one long response can exhaust the remaining budget and leave later tasks unserved. Our objective is to maximize the number of tasks solved before service stops. We develop a contextual bandit policy, ADP-HT, which learns each model's task-specific success probability and constructs conservative cost estimates from coarsened response-length labels on a quantile grid with $2^K$ intervals. The policy weighs predicted…

JSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2611.05835
curl -X POST /api/v1/records/arxiv/2611.05835/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.05835/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'