Sequential Sampling for Binary Classification: Two LLMs are (Almost) All You Need

Guokai Li, Alys Liang, Mo Liu, Murray Lei, Stefanus Jasin, Fenghua Yang, Preet Baxi

arXivarxiv/2604.01086Apr 1, 2026cs.DSscore 0

We study how to orchestrate multiple large language models (LLMs) for reliable binary classification when models differ in accuracy, monetary cost, and response time. A decision-maker adaptively chooses which LLM to query, updates its belief after each response, and stops once sufficient confidence has been reached. Our results reveal a sharp contrast between exact and near-optimal orchestration. In the worst case, an exactly optimal policy may need to query every available LLM. Yet under high reliability requirements, a simple policy using at most two LLMs is asymptotically optimal. The…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2604.01086
curl -X POST /api/v1/records/arxiv/2604.01086/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2604.01086/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'