What AI token pricing is — why output costs about 5× input
AI API pricing charges separately for the text a model reads (input tokens) and the text it writes (output tokens), quoted per million tokens, and output costs several times more because input is processed in a single parallel pass while output is generated one token at a time
The three lines
- Billing is per token, with separate input and output rates — output typically costs four to five times input
- A model has no memory between calls, so the whole prior conversation is resent as input each turn; cost grows closer to the square of conversation length
- List rates exclude cache discounts, batch rates and introductory expiry dates — real invoices are not computed from them alone
Key questions
- What is a token?
- It is the smallest unit a model works in — not a character and not a word but something between. In English one token runs roughly four characters; languages such as Korean use more tokens for the same content. That makes 'a million tokens equals so many books' a language-dependent estimate. If you need a precise figure, read the token usage the API returns with each response. Tokenizers differ by company, so the same sentence yields different token counts on different platforms.
- Why do input and output cost different amounts?
- Because the computation differs. Input is a single pass in which the model reads the whole prompt, and a GPU can process that in parallel. Output is sequential: the model produces one token, appends it, and produces the next. Writing 100 tokens means running the model 100 times. A workload that compresses into parallel and one that stretches out sequentially carry different costs, and that difference appears directly in the rates — published output prices are commonly four to five times input.
- Why do longer conversations cost more?
- Because the model has no memory between calls. Every request must resend the entire prior exchange as input for the model to have context. If turn one sent 500 tokens, turn ten resends all nine previous questions and answers. Cumulative cost across a conversation grows closer to the square of its length than linearly. That is why long advisory-style products insert summaries or truncate older turns to control input.
- Is the list price what gets billed?
- No. At least three things sit between the rate card and the invoice. Caching: repeated prefixes such as system prompts or long reference documents can be stored and billed at a much lower rate on reuse. Batch processing: work that does not need an immediate response can be submitted together at a separate lower rate. Introductory expiry: new-model prices frequently carry an end date, after which they can double.
- How does subscription pricing differ from API pricing?
- They are separate billing models. A subscription is a flat monthly fee with usage allowances; an API is metered per token. Individuals using a chat app are usually on subscriptions, while companies embedding a model in a product use the API. The two are not linked — API rates can fall while subscription prices hold, and that has happened explicitly more than once.
An AI model's price list usually looks like this.
`` Input $0.75 / 1M tokens Output $3.75 / 1M tokens ``
Two numbers, apparently. Turning them into an actual invoice takes at least five more facts.
1. Tokens — the billing unit
A token is the smallest unit a model handles: not a character, not a word, something between.
| Language | Rough density |
|---|---|
| English | 1 token ≈ 4 characters |
| Korean, Japanese, Chinese | more tokens for the same content |
Exact conversion depends on the tokenizer, which differs by company. The same sentence produces different token counts on different platforms.
So treat "a million tokens is about N books" as a feel, not a figure, and read the token usage the API returns alongside each response when it matters.
For anyone building a multilingual product, this is not a footnote. The same feature costs more in Korean than in English.
2. Why input and output are priced differently
This is the structural core of the rate card.
| Stage | What happens | Computation |
|---|---|---|
| Input processing | the model reads the whole prompt once | parallel — the GPU handles it together |
| Output generation | produce a token, append it, produce the next | sequential — one at a time |
Reading a 100-token prompt is close to running the model once. Writing a 100-token answer is running the model 100 times.
That difference shows up directly in prices: published output rates are commonly four to five times input.
The practical consequence is clear.
| Workload | Input | Output | Cost profile |
|---|---|---|---|
| Long-document summarisation | very long | short | input-dominated — relatively cheap |
| Short prompt → long generation | short | very long | output-dominated — expensive |
| Classification / extraction | medium | very short | among the cheapest |
| Code generation | medium | long | output-heavy |
The same million tokens can cost five times more depending on which side it falls.
3. Why conversations get expensive
This is where most budget surprises originate.
A model has no memory of the conversation. Every request must resend the whole prior exchange for the model to have context.
Say each turn involves a 200-token question and a 300-token answer.
| Turn | Input tokens | Cumulative input |
|---|---|---|
| 1 | 200 | 200 |
| 2 | 200 + 500 = 700 | 900 |
| 3 | 200 + 1,000 = 1,200 | 2,100 |
| 5 | 2,200 | 6,000 |
| 10 | 4,700 | 24,500 |
Cumulative input across a ten-turn conversation is not ten times a one-turn conversation — it is more than 120 times, because it grows roughly with the square of the length.
Products that handle long conversations use one of three mitigations.
① Summary injection — replace older turns with a short summary ② Window truncation — keep the last N turns and drop the rest ③ Caching — bill the repeated prefix at a lower rate (next section)
Why an advertised context limit differs from the length a model can actually use is set out in "What a context window is."
4. Three things the rate card omits
| Item | What it is | Effect |
|---|---|---|
| Cached input | store a repeated prefix — system prompt, long reference document — and reuse it | that portion bills at a far lower rate |
| Batch processing | submit work that does not need an immediate response together | separate lower rate |
| Introductory expiry | new-model prices often carry an end date | the price rises on that date |
Caching matters most. A service that asks 100 questions about the same 500-page document pays for that document 100 times without it. With caching in place it is billed once in full and then at a reduced rate on reuse.
Introductory expiry is the item most often dropped from price comparisons. "What does it cost now" and "what does it cost next year" are different questions for many models.
5. A real rate card — August 2026
Published rates confirmed as of August 15, 2026.
| Model | Input (1M) | Output (1M) | Note |
|---|---|---|---|
| Gemini 3.7 Flash | $0.75 | $3.75 | introductory through December 31, 2026, then $1.50/$7.50 |
| Gemini 3.6 Flash | $0.75 | $3.75 | cut to the same rate the same day, same expiry |
| Claude Sonnet 5 | $2 | $10 | made standard on August 10, 2026 (a scheduled rise to $3/$15 was cancelled) |
Three things to read out of it.
① The output-to-input ratio converges near 5×. Gemini 5.0, Sonnet 5.0. The computational structure described above pulls different vendors to a similar ratio.
② Some prices carry expiry dates and some do not. Gemini's $0.75 has an end date; Sonnet's $2 does not. For a service intended to run for years, that distinction can matter more than the rate difference.
③ Tiers differ within a vendor. Workhorse and frontier tiers usually differ by a multiple, often a large one. Routing every request to "the best model" is generally a cost-design failure.
The background to these moves is set out in "Gemini 3.7 Flash costs half the old workhorse price."
6. Working an invoice
Assume Gemini 3.7 Flash rates ($0.75/$3.75) for a customer-support response service.
| Item | Value |
|---|---|
| Requests per day | 5,000 |
| Input per request | 3,000 tokens (2,000 system prompt + 1,000 query) |
| Output per request | 500 tokens |
| Daily input tokens | 15,000,000 |
| Daily output tokens | 2,500,000 |
| Daily input cost | 15 × $0.75 = $11.25 |
| Daily output cost | 2.5 × $3.75 = $9.38 |
| Daily total | about $20.63 |
| Monthly (30 days) | about $619 |
Now apply caching. The 2,000-token system prompt is identical across all 5,000 requests — 10 million identical tokens a day. Moving that to a cache rate cuts into roughly two-thirds of the input bill.
Shifting repetition into cache usually saves more than trimming token counts. Specific discount rates differ by vendor and are not fixed here.
7. Subscriptions and APIs are different prices
| Model | Billing | Who uses it |
|---|---|---|
| Subscription | flat monthly fee + allowances | individuals, chat apps |
| API | metered per token | developers embedding a model |
The two are not linked. API rates can fall while subscription prices hold. When Anthropic made Claude Sonnet 5's introductory API rate permanent in August 2026, it stated explicitly that subscription plan prices were unchanged.
8. Cutting costs, in order of effect
- Drop a tier — do not send classification and extraction work to a frontier model. This saves multiples, not percentages
- Move repeated input into cache — system prompts and reference documents
- Constrain output length — output costs 5×, so a single instruction to be concise has real financial effect
- Manage the conversation window — summarise or truncate
- Batch anything that is not urgent
- Count tokens — only meaningful after the five steps above
9. What this article does not cover
Image and audio input billing. Non-text inputs convert to tokens under separate schemes not addressed here.
Tool calls and agent loops. When a model repeatedly searches or executes code, each result re-enters as input. Where one user request becomes dozens of internal API calls, the arithmetic above does not carry over directly.
Prices change frequently. The rates here are those published as of August 15, 2026. Check each vendor's current pricing page before relying on them.
Sources
- Google AI for Developers — Gemini API pricing
- Google AI for Developers — Gemini API release notes
- Claude Platform Docs — Pricing
- Axios — New Gemini Flash model arrives before Gemini 3.5 Pro
- Digital Applied — Gemini 3.7 Flash: Smarter Workhorse at Half the Old Price
- TechJournal — Claude Sonnet 5 Pricing Now Permanent