What is a context window — a model does not read all million tokens
A context window is the maximum number of tokens a language model can handle in a single request, counting input and output together — and the advertised length is not the same as the length across which the model can accurately retrieve information
The three lines
- Context window = total tokens visible in one request, input and output combined
- Most leading 2026 models converged on 1 million tokens, but retrieval accuracy degrades markedly between the 200K and 1M ranges
- The distinction to hold onto is advertised context length versus effective context length
Key questions
- What is a context window?
- The maximum amount of text a language model can consider at once in a single request, measured in tokens — roughly word-fragment sized chunks. The important detail is that the limit covers input and output together. If a model has a 1M-token window and you have already supplied 990,000 tokens, only 10,000 remain for the answer. In a conversation, the entire prior exchange is resubmitted as input on every turn, so the window fills continuously.
- How much text is a million tokens?
- It depends on the tokenizer and the language. English runs roughly four characters per token; languages such as Korean typically consume more tokens for the same content, because tokenizers are usually optimised on English-heavy corpora. So a 'million tokens equals N books' conversion changes substantially by language. In practice, reading the token usage returned by the API is more reliable than estimating from character counts.
- Is a bigger window always better?
- It carries three costs. Price: most APIs bill by input token, so filling the window raises the cost of each request. Latency: longer inputs increase time to first response. And accuracy: a model does not attend evenly across a large window. Retrieval of specific facts degrades toward the far end of the advertised length, a pattern observed consistently across frontier models.
- How do you check the effective context length?
- Through benchmarks. The best-known family is needle-in-a-haystack testing, which plants a specific sentence inside a long document and measures whether the model finds it. Single-needle tests score generously, so multi-needle retrieval — typically eight planted facts — is closer to production reality. Suites such as RULER measure not only retrieval but reasoning over long context. The vendor's published context length is a specification; these benchmarks are the measurement.
One number appears in every model announcement now. One million tokens of context.
By 2026 it has effectively become the standard. Most leading models arrived at the same figure, and some went past it.
The benchmarks tell a different story. Accepting a million tokens and finding a specific sentence inside a million tokens are separate capabilities.
1. The definition
A context window is the maximum amount of text a model can consider at once in a single request, measured in tokens — roughly word-fragment sized chunks.
The most frequently missed detail:
The window is not an input allowance. It is the total for input and output combined.
Supply 990,000 tokens to a 1M-token model and 10,000 remain for the answer. Ask for a long document and a long response, and one of them gets cut.
Conversational products add a second effect. The model does not remember previous turns. Each new question resubmits the entire conversation as input. The window fills as the conversation grows, and once the limit is reached, the earliest content falls out. That is usually what is happening when a chatbot "forgets what you said."
2. Tokens are counted differently by language
| Language | Rough behaviour |
|---|---|
| English | ~4 characters per token |
| Korean, and many non-Latin scripts | More tokens for the same content |
Tokenizers are generally built on English-heavy corpora, so text in other scripts fragments into more tokens.
Two practical consequences. The same document fills the window faster in Korean. And the same task costs more in Korean, since most APIs bill per token.
The exact multiplier differs by tokenizer. Reading the usage figures the API returns beats estimating from character counts.
3. Three costs of a bigger window
| Cost | Detail |
|---|---|
| Price | Billed per input token — filling the window raises per-request cost |
| Latency | Longer input increases time to first token |
| Accuracy | Retrieval degrades toward the far end of the window |
The first two are predictable. The third is the problem.
4. Advertised length versus effective length
The consistent 2026 finding is this: accuracy holds well over the early portion of an advertised window and falls substantially as inputs approach the upper limit, particularly between the 200K and 1M ranges.
One benchmark summary put the drop at 30 to 60 points. Per-model source data was not cross-checked, so treat that figure as indicative. That a drop exists is agreed across sources.
Two terms are worth separating.
| Term | Meaning | Source |
|---|---|---|
| Context length | Maximum tokens the model accepts | Vendor specification |
| Effective context length | Tokens the model reliably uses | Benchmark measurement |
Vendors publish the first. The second has to be measured.
The hard problem in 2026 is no longer capacity. Building a model that accepts ten million tokens is far easier than making it locate one specific fact inside them without inventing an answer.
5. Measuring it — three benchmark types
| Benchmark | What it measures | Caveat |
|---|---|---|
| Needle in a Haystack (NIAH) | One planted sentence inside a long document | Single-needle scores are generous |
| Multi-needle retrieval | Typically eight planted facts | Much closer to production |
| RULER-style suites | Retrieval and reasoning over long context | Tests whether the model can use what it found |
The middle row matters most. Single-needle scores mislead because finding one sentence in one document is not a real task. The real task is loading an entire codebase and correcting all eight related functions — and there, models fail by finding some and reporting the rest as absent.
The silence is the danger. The model does not say it missed anything; it produces a plausible answer from the subset it retrieved.
6. Working practice
Do not fill the window because you can. Irrelevant material raises cost and latency while lowering accuracy.
Put critical instructions at the beginning or the end. Information placed in the middle of a long input is the most frequently ignored, an observation reproduced repeatedly.
Keep retrieval in front of the model. Selecting the relevant passages instead of dumping everything into the window — the RAG pattern — did not become unnecessary when windows grew. It became cheaper to justify.
Summarise long conversations as you go. Since the entire history is resubmitted each turn, periodic compression improves both cost and accuracy.
7. The 2026 landscape
| Model tier | Context (mid-2026) |
|---|---|
| Most leading models | 1,000,000 tokens |
| Some models | 500,000 tokens |
| Some experimental models | Beyond |
Convergence on one figure means context length has stopped being a differentiator. DeepSeek V4-Pro, which reached general availability on August 13, 2026, also lists 1 million tokens.
The competition has moved to the next question: who retrieves most accurately within the same million.
8. What is unresolved
The 30–60 point retrieval drop cited above comes from a single benchmark summary; per-model source data was not checked. The claim that one particular model sustains accuracy across its full window comes from the same summary, with no independent reproduction found.
No token multiplier is given for non-English languages, because it varies by tokenizer.
Context lengths quoted here reflect mid-2026 and change with each version. What stays useful in this piece is not the table but the habit of separating advertised length from effective length.
Sources
- Morph — LLM context window comparison (2026): 20 models from 200K to 10M tokens
- Digital Applied — Long-context retrieval 2026: needle-in-haystack test
- DataNorth — LLM context length & context window explained (2026)
- Zylos Research — LLM context window management and long-context strategies 2026
- DevTk.AI — LLM context windows explained: 4K to 1M tokens (2026)