AI & LLM cost
Batch API
An asynchronous request mode. It trades a longer turnaround, from minutes to hours, for a discount against real-time API calls.
Last updated
Definition
Batch processing submits a large set of requests together. It returns the results once they all finish, not one response per call in real time. The provider can schedule that work more efficiently than a latency-sensitive request. Anthropic and OpenAI both publish a 50% discount on batch requests against the standard API rate. Check the provider's price page for the current rate.
Batch fits anything that does not need an immediate answer. Examples are bulk classification, offline evaluation, and nightly summarization jobs. It is a poor fit for anything user-facing where response time matters.
Where it shows up
You submit a batch API request through its own dedicated endpoint. It is separate from the standard real-time completion endpoint. The provider's usage dashboard reports batch token volume as its own category. Batch tokens bill at their own discounted rate. The invoice shows that rate apart from the real-time total.
What makes it expensive
Two mistakes are possible. One is to route a latency-sensitive, user-facing request through the batch endpoint by default. Check first whether the workload can wait minutes to hours. The other mistake is quieter and far more common. Nightly jobs and bulk classification stay on the real-time endpoint. They pay a premium for speed nobody needed.
Related
Token spend moves every day.
Model prices and rate limits change between invoices. CostMon tracks your AI spend daily, so a change shows up before the bill does.