RAG Cost Calculator
Calculate the full monthly cost of a RAG pipeline — embedding, vector DB, and LLM generation. The surprising insight: generation is almost always 85%+ of total cost, not embedding.
Prices verified 2026-08-20 · Prompt Cost Calculator
How to Use
Set your corpus size
Enter the number of documents and average tokens per document. This determines initial embedding cost and storage.
Set query volume
Monthly queries, query token length, number of chunks retrieved per query, and chunk size. These drive generation cost.
Choose models and vector DB
Select embedding model, generation model, and vector database. pgvector is cheapest for self-hosted setups.
See the surprise
LLM generation is almost always 85%+ of total RAG cost. Most people optimize embedding cost instead of generation. This calculator shows exactly where your money goes.
The RAG cost stack
Most teams optimize for embedding cost. The actual cost driver is almost always generation. A 50,000-query/month RAG system with 5 chunks × 400 tokens each means 100M input tokens/month to the generation model. Embedding those same 50,000 queries costs less than 1M tokens total. Optimize generation first: fewer chunks, cheaper model, prompt caching on the system prompt.
How to Cut This Cost
Reduce retrieved chunks from 10 → 3. Most RAG answers use only the top 1–2 chunks. Fewer chunks = fewer generation tokens.
Cache the top retrieved chunks for identical or near-identical queries. A query cache layer (Redis) eliminates LLM calls entirely for repeated questions.
Use a smaller model for generation if your context is clean and factual. GPT-5 mini or Haiku 4.5 handles most factual retrieval tasks adequately.
Embed once, reuse forever. Batched embedding of your corpus costs the same whether done today or in a year — avoid re-embedding unchanged documents.
FAQ
Have more questions? Contact us
Last verified: 2026-08-20 · methodology · data sources