How to Reduce LLM API Costs — 10 Proven Techniques

1. Enable prompt caching (up to 90% off repeated context)

If you have a long system prompt or document context repeated across calls, caching cuts the cost of that prefix by 90% (Anthropic). One system prompt of 10K tokens, sent 10K times/month: $0.30/M × 10K × 10K/1M = $3,000 uncached; $30 cached. Saves $2,970/month.

2. Use the batch API (50% off for async workloads)

Any pipeline that doesn't need real-time results (data extraction, summarization, classification at scale) should use the Batch API. Flat 50% discount, no quality change, minimal code change.

3. Downgrade the model for simpler tasks

GPT-5.4 ($2.50/M input) vs GPT-5 mini ($0.25/M) = 10× difference. Claude Opus 5 ($15/M) vs Claude Sonnet 5 ($3/M) = 5×. Classification, extraction, and simple Q&A don't need the flagship model. Run your eval suite on the cheaper model first.

4. Implement conversation history management

Full history compounds quadratically: total_input = S × N + (U + A) × N(N-1)/2. A 20-turn chat with a 1K system prompt costs 5× more at turn 20 vs turn 5. Use a sliding window (last 5 turns) or summarize every 10 turns. The cost savings are often 80%+ for long conversations.

5. Reduce system prompt size

Every token in your system prompt is paid for on every call. A 5,000-token system prompt vs a 1,000-token one costs 5× more on the input side of every request. Audit your system prompt: remove examples that can be compressed, remove disclaimers, remove repeated instructions.

6. Minimize output tokens

Output tokens cost 3–5× more than input tokens. Ask for: shorter answers, JSON instead of verbose prose, bullet points instead of paragraphs, no "Sure, I'd be happy to help!" preambles. Instruction: "Be concise. Answer in under 100 words." can cut output costs by 60%.

7. Use structured output for extraction (fewer output tokens)

Asking a model to return JSON directly vs asking it to "explain and then extract" cuts output tokens by 3–5×. The model doesn't need to write "Based on the above, the fields are:" before the JSON.

8. Reduce RAG chunk count and chunk size

In RAG pipelines, retrieved chunks often dominate input token costs. 10 chunks of 1,000 tokens = 10,000 tokens per query. Cutting to 5 chunks saves 50% on input for every query. Test whether fewer chunks degrade answer quality before assuming 10 is necessary.

9. Cap agent loop steps

Agent costs grow quadratically with step count. A 40-step agent costs roughly 4× a 20-step agent (quadratic context growth). Set hard step limits. Summarize or truncate tool results. Use a cheaper model for simple tool-use steps.

10. Cache at the application layer

Identical questions get identical answers. Cache semantic-similarity queries (not just exact matches). For customer support bots, 20–40% of queries are near-duplicates. Application-layer caching costs $0; API caching costs $0.10×. This is free savings.