RAG Cost Calculator
Prices the three real components — indexing, retrieval, generation — instead of guessing. Prices updated 2026-08-02.
- One-time indexing
- $—
- Per query
- $—
- Generation share
- Embedding share
Includes a 200-token system prompt and 50-token question embedding per query.
Worked example: a 5,000-document knowledge base
Take a support knowledge base of 5,000 articles averaging 2,000 tokens (10M tokens total), serving 1,000 queries a day with 3,000 tokens of retrieved context and 300-token answers:
- Indexing (one-time): 10M tokens × $0.02/M = $0.20. Yes, twenty cents — indexing is never the expensive part.
- Embedding queries: 50 tokens × 30,000 queries/month ≈ 1.5M tokens ≈ $0.03/month.
- Generation on a mini-tier model ($0.75/$4.50): (3,250 in × $0.75 + 300 out × $4.50) ÷ 1M ≈ $0.0038/query → ~$115/month.
- The same pipeline on a flagship ($5/$25): ~$24 per 1,000 queries → ~$730/month.
The lesson generalizes: RAG cost ≈ generation cost. Optimize the model choice and retrieved-context size; ignore embedding prices entirely.
Frequently asked questions
› What are the cost components of a RAG system?
Three parts: (1) a one-time indexing cost to embed your document corpus; (2) a tiny per-query embedding cost for the user question; and (3) the dominant cost — generation, where retrieved chunks are stuffed into the LLM prompt as input tokens. In most production RAG systems, 90%+ of spend is the generation step, not the embeddings.
› Why is embedding so much cheaper than generation?
Embedding models are small and run one forward pass per text, with prices around $0.02–$0.15 per million tokens — roughly 100× cheaper than flagship LLM input rates. Embedding a 10-million-token corpus costs under a dollar with most models. The retrieved context you feed the LLM on every single query is what adds up.
› How many tokens of context does a typical RAG query use?
A common setup retrieves 3–8 chunks of 300–800 tokens each, so 1,500–5,000 tokens of context per query, plus the system prompt and question. Retrieving more chunks improves recall but raises cost linearly — tune chunk count against answer quality, not just cost.
› How do I reduce RAG costs?
In order of impact: (1) use a cheaper generation model — mini tiers handle grounded answering well because the facts come from your documents; (2) cache the static system prompt; (3) retrieve fewer, better chunks with reranking; (4) cap answer length. Re-embedding with a cheaper model saves almost nothing, since indexing is a rounding error.