What an AI token is — how much text $10 per million tokens actually buys
A token is the smallest unit an AI language model uses to read and write text. It is closer to a word fragment than a word, which is why the same sentence produces different token counts in different languages — one measurement found Korean uses about 2.36 times as many tokens as English for the same meaning. Providers price input tokens and output tokens separately, and output is usually more expensive because generating text requires more computation than reading it. Ramp data reported in August 2026 illustrates the structure: Anthropic's Fable 5 accounted for 11.4 percent of dollar spend on Anthropic tools but only about 6 percent of tokens, because it runs at roughly 10 dollars per million tokens, about double GPT-5.6 Sol. A context window caps the total tokens, input plus output, that a single exchange can handle
The three lines
- Unit — a token is a word fragment, not a word, and counts differ sharply by language
- Pricing — input and output are priced separately, output costs more, headline price is not total cost
- Evidence — Fable 5 was 11.4% of spend but 6% of tokens, because its unit price is double
Key questions
- What is an AI token?
- **The smallest unit a model uses to process text.** People read in words; models read in **tokens**, which are generally smaller than words — common short words become a single token, while long or rare words are split into several pieces. The sequence runs: ① the input text is converted into a list of tokens (tokenisation), ② the model reads that list and predicts the next token one at a time, ③ the predicted tokens are converted back into characters for display. That is why billing is per token — **the token count is the amount of work the model actually did**. It is also why Jensen Huang has described tokens as the currency of AI.
- Why is Korean more expensive than English?
- **Because the same meaning consumes more tokens in Korean.** One measurement translated identical sentences across languages and found Korean used about **2.36 times** as many tokens as English. The reason is that most tokenisers were built on English-heavy data — English words often map to a single token, while Korean tends to break into smaller syllable or jamo-level pieces. Two consequences follow. ① **Cost** — at the same headline price per million tokens, the same task in Korean costs more. ② **Capacity** — a model with a one-million-token context window fits less Korean text than English text. Which means **comparing models on the advertised price per million tokens alone will mislead you.**
- How much text is a million tokens?
- **There is no fixed conversion, but there are two reliable ways to get a feel for it.** Token counts vary by language, by the kind of text, and by tokeniser, so no constant ratio of tokens to characters exists. In practice: ① **Count it yourself** — every provider offers a token counter, and running your actual documents through it is the accurate answer. ② **Look at cumulative usage**, not a single query. This matters most in long conversations or when a codebase is loaded into context, because **the entire prior content is re-sent as input tokens on every turn.** With 900,000 tokens of material loaded into a one-million-token context, a one-line question still bills 900,000 input tokens. Bills that come in far above expectation almost always trace to this.
Every AI price list uses the same unit: "$X per million tokens."
Compare two models on that number alone and you will usually be wrong. Here is what a token is, and why the headline price does not tell you the cost.
1. A token is not a word
People read in words. Models read in tokens.
| Stage | What happens |
|---|---|
| ① Tokenise | The input text is split into a list of tokens |
| ② Predict | The model reads the list and predicts the next token, one at a time |
| ③ Detokenise | Predicted tokens are converted back into characters |
Tokens are generally smaller than words. Common short words map to a single token; long or rare words split into several pieces. Treating "token" as "word" throws the arithmetic off.
Billing is per token for a simple reason: the token count is the work the model actually did.
2. Input and output are priced differently
| Type | What it is | Relative price |
|---|---|---|
| Input tokens | What the model reads — your prompt, attachments, prior turns | Cheaper |
| Output tokens | What the model writes — the answer | More expensive |
Output costs more because of how generation works. Reading can be processed in one pass; writing requires producing tokens one after another in sequence.
So two tasks that look similar can have completely different cost profiles: read a long document and answer briefly versus ask briefly and have it write at length. Summarisation is the first; generation is the second.
3. Languages count differently — the Korean case
| Language | Tokens for the same meaning |
|---|---|
| English | 1.00 (baseline) |
| Korean | about 2.36x |
Most tokenisers were built on English-heavy data. English words often become one token; Korean tends to break into small fragments.
Two consequences:
| Item | What it means for Korean users |
|---|---|
| Cost | The same task costs more at the same headline price |
| Capacity | Less Korean text fits into a given context window |
Identical advertised prices produce different real costs by language. This is why models should be measured on your own Korean text before choosing. That said, 2.36x is one measurement setup, and it shifts with the model and the kind of text.
4. Context windows — where bills spike
A context window is the total tokens, input plus output, a single processing pass can handle.
Here is the part people miss.
| Situation | Actual input tokens |
|---|---|
| 900,000 tokens of material loaded into a 1M-token context | — |
| A one-line question asked in that state | 900,000 + the question |
| One more follow-up question | 900,000 again + the prior turns + the question |
Every additional turn carries a larger input, because the whole prior content is re-sent each time. Bills that come in far above expectation almost always come from here.
Caching, summarisation and context pruning are the techniques used to manage this; the details are out of scope here.
5. A live example — 11.4 percent of spend, 6 percent of tokens
August 2026 produced a clean illustration of how unit price moves a market. The spend management firm Ramp aggregated usage across roughly 70,000 businesses.
| Metric | Anthropic Fable 5 |
|---|---|
| Share of dollar spend on Anthropic tools | 11.4% |
| Share of tokens | about 6% |
| Price per million tokens | about $10 |
| Comparison | roughly 2x GPT-5.6 Sol |
Spend share is nearly double token share for one reason — the unit price is double. The table measures "how much is the expensive model used" two ways: counted in money it is 11.4 percent, counted in work done it is 6 percent.
The follow-on observation matters more. Fable 5 has not grown since the end of June, and the cheaper Opus 5, launched in late July, has already overtaken it in enterprise spending. Businesses are concluding that cheaper models handle most of the work.
The same current showed up as price cuts: on August 21, 2026 OpenAI reduced GPT-5.6 Sol's output price from $30 to $20 per million tokens (see "GPT-5.6 Sol API price cut").
6. Frequently asked
Q. How many characters is a million tokens? There is no fixed ratio; it varies by language, text type and tokeniser. Run your actual documents through a provider's token counter — that is the accurate answer.
Q. Is the cheaper headline price always cheaper? No. The same text can produce different token counts across models, and answer lengths differ. A model with a lower listed price that writes longer answers can cost more in total.
Q. Why use the expensive model at all? Accuracy differences on hard tasks. What the Ramp data shows is companies concluding that the share of work needing that difference is smaller than assumed.
Q. Should I look at unit price or context window first? Depends on use. Many short exchanges make unit price decisive; handling long documents makes context window and caching decisive.
7. What is unresolved
- The 2.36x — one measurement setup, varying by model, tokeniser and text type.
- Price figures — Fable 5 at ~$10 per million tokens and 2x GPT-5.6 Sol come from reporting, not official price lists.
- Sample — the industry composition of Ramp's roughly 70,000 businesses was not verified.
- Period — the exact window covered by the 11.4 percent and 6 percent figures was not confirmed.
- Current prices — per-model input and output rates were not individually verified.
Sources
- OpenAI Help Center — What are tokens and how to count them
- TECH INSIGHT — What AI tokens are, and why Korean costs 2-3 times more than English
- AI Weekly — FT: Ramp Data Shows Anthropic's Fable 5 Plateaus at 11% of Anthropic Spend as Opus 5 Overtakes It
- heise online — Company data: Spending on Anthropic's Fable 5 AI model stagnates
- XDNode — What is a token? The thing Jensen Huang called the currency of AI
- Turing Post Korea — Tokens: the unit by which AI understands the world