Batch API Explained — 50% Off for Async LLM Workloads

What the Batch API does

Instead of sending individual API requests (and waiting for each), you submit a JSON file of requests. The API processes them in bulk and returns results within 24 hours. The discount is ~50% across all providers. For non-real-time pipelines, this is free money.

Which providers offer batch APIs

OpenAI Batch API: 50% off, results in 24h. Anthropic Message Batches: 50% off for all Claude models, results in 24h. Google Gemini Batch: 50% off, results in 24h. All three support their respective model families.

Qualifying workloads

Good candidates: document summarization pipelines, content classification at scale, data extraction from large corpora, offline evaluation runs, generating embeddings, nightly report generation. Not suitable: real-time chatbots, interactive tools, anything requiring <2s latency.

Implementation sketch

Build a JSONL file where each line is a full API request object with a custom_id. Submit to the batch endpoint. Poll for completion or set up a webhook. On completion, download the results JSONL. Most pipelines add <10 lines of code over the synchronous version.