How LLM Pricing Works — A Complete Guide

The basic formula

Cost = (input_tokens / 1,000,000) × price_per_M_input + (output_tokens / 1,000,000) × price_per_M_output. Every other pricing feature is a modifier on top of this.

Why output costs 3–5× more than input

Input tokens go through the model's forward pass once. Output tokens are generated one at a time — each requires a full forward pass through the model. This is fundamentally more compute. At scale, minimizing output token length has outsized cost impact.

Prompt caching: the biggest cost lever

Anthropic (Claude), OpenAI (GPT), and Google (Gemini) all offer prompt caching — storing a reusable prefix so you don't re-process it on every call. Cached reads cost 5–10× less than standard input. If you have a long system prompt used across many calls, caching should be your first optimization.

Batch API: 50% off for async workloads

All major providers offer batch APIs (results in <24h, not real-time): OpenAI Batch API, Anthropic Message Batches, Gemini Batch. These cut costs by ~50% with no quality trade-off. Any pipeline that doesn't need real-time results should use batch.

Context window pricing surprises

You only pay for tokens actually used, not the context window size. But in practice, long contexts cost more because: (1) every turn in a conversation re-sends the full history, (2) long system prompts compound, (3) RAG adds retrieved chunks. A 20-turn conversation with a 2K system prompt uses ~50K+ tokens just for repeated context.