What is a token and how many are in my prompt?

A token is the billing unit LLM APIs meter — roughly 4 characters or ¾ of an English word, so 1,000 tokens ≈ 750 words. Every request is billed on two counts: the tokens you send (input) and the tokens the model generates (output). Paste your text into our token counter for a quick estimate before you commit to a model.

Why do output tokens cost more than input tokens?

Generating text is sequential — the model produces one token at a time, each requiring a full forward pass — while reading input is processed in parallel. That compute asymmetry shows up on the price sheet: as of 2026-08-23, OpenAI GPT-5.6 Sol charges 5.0× more per output token than per input token. This is why capping response length (max_tokens) is one of the cheapest optimizations available.

Do I pay for the chat history in every message?

Yes — the API is stateless, so your app re-sends the system prompt and conversation history with every turn, and you pay input rates on all of it each time. A 20-turn conversation bills the early messages twenty times. This context-resend effect is why real chat workloads skew heavily toward input costs, and why cached-input pricing matters so much.

What is the cheapest way to run the same prompts repeatedly?

Combine the two standing discounts: cached input for repeated context (90% off the input rate on OpenAI GPT-5.6 Sol) and batch endpoints for anything that can wait minutes-to-hours (~50% off both rates). For high-volume simple tasks, also consider routing to a mini-tier model — the price gap between tiers is far larger than most quality gaps on routine work.

How LLM API Pricing Actually Works

Every LLM provider prices the same way — per million tokens, split across a handful of meters — yet most first invoices still surprise the team that gets them. This guide explains the four meters on the bill, with live prices (updated 2026-08-23), and the one mechanic — context re-sending — that makes real applications cost several times the back-of-napkin estimate.

The billing unit: tokens, in millions

A token is roughly 4 characters, or about ¾ of an English word — 1,000 tokens ≈ 750 words. Prices are quoted per 1 million tokens, which makes small numbers feel abstract: OpenAI GPT-5.6 Sol at $2.00/1M input means a 2,000-token prompt costs $0.004 to read. Estimate your own text with the token counter — code and non-English text tokenize less efficiently, so budget 20–40% more tokens for both.

The four meters on every bill

Input (prompt) tokens — everything you send: system prompt, conversation history, retrieved documents, the user's question. Usually the biggest line item in production.
Output (completion) tokens — everything the model writes back, billed at a premium: 5.0× the input rate on GPT-5.6 Sol, 6.0× on GPT-5.4 Mini. Generation is sequential compute; reading is parallel — you pay for that asymmetry.
Cached input tokens — repeated context (a long system prompt, a shared document) that the provider has seen recently bills at a deep discount — 90% off on OpenAI GPT-5.6 Sol ($0.2/1M instead of $2.00/1M). The catch is cache lifetime — the prompt caching guide covers the TTL cliffs.
Batch tokens — the same request, submitted asynchronously and answered within minutes-to-hours, at roughly half price on both meters. Free money for evals, backfills, and summaries — see the batch API guide.

One request, priced end to end

Take a typical RAG-style request: 2,000 input tokens (system prompt + retrieved context + question) and 500 output tokens of answer:

Model	Input cost	Output cost	Total / request	× 1,000 requests/day × 30
OpenAI GPT-5.6 Sol	$0.004	$0.005	$0.009	$270/mo
OpenAI GPT-5.4 Mini	$0.0015	$0.0023	$0.0038	$113/mo

Same request, 2× spread between tiers. Run your own workload through the API cost calculator — it does this math across all 290 models we track.

The trap: you re-send the conversation every turn

LLM APIs are stateless. There is no "session" on the provider's side — your application re-sends the system prompt and the full chat history with every message, and pays input rates on all of it, every time. Turn 1 sends 500 tokens; turn 10 might send 8,000, most of it text you already paid for nine times. This single mechanic is why chat products cost 3–5× the naive estimate, why input dominates real bills, and why cached-input pricing exists at all. The chatbot cost calculator models this growth (and per-provider cache behavior) instead of pretending every request is turn one.

What actually moves the bill

Model tier — the biggest lever by far, as the table above shows. Route routine traffic down-tier; the cheapest LLM APIs ranking shows how wide the floor-to-flagship spread is right now.
Output caps — output is the expensive meter; set max_tokens and ask for terse formats (JSON, bullet answers) instead of essays.
Context discipline — trim history, summarize old turns, retrieve fewer documents. Every input token you don't send is billed zero times per conversation, not once.
Caching and batch — the two standing discounts; stack them where the workload allows.

The full playbook, ranked by effort against impact, is in how to cut your LLM API bill.