What prompt caching is — from the second call, your input costs one-fortieth

Prompt caching stores the opening section that goes in unchanged on every call, so it can be reused.

Why that is needed becomes obvious once you look at how a language model actually works.

1. The model rereads everything, every time

A language model processes its entire input from scratch on every call. Services that appear to "remember" a conversation are in fact resending the whole conversation.

So this happens. Say your system instruction is 2,000 tokens and your reference document 50,000. Change one line of the question and ask ten times:

52,000 tokens × 10 calls = 520,000 tokens

You paid to read the same 52,000 tokens ten times. Only the last line changed.

Prompt caching removes that repetition. Process it once, store the result, pull from storage on subsequent calls.

2. Why only the front can be cached

The prefix restriction is not a convenience choice. It falls out of the model's architecture.

In a transformer, each token's internal representation depends only on the tokens before it. Computing token 100 looks at tokens 1–99; it never looks at token 101. Therefore:

• If the prefix is byte-identical, the intermediate values across that stretch are identical too → they can be stored and reused
• If one character differs, everything computed after that point differs → all of it must be recomputed

This is prefix caching. It is why the front, and not the back, is what gets stored.

3. The price splits into three tiers

The pricing follows the mechanics.

A read costs one-fortieth of ordinary input. When Anthropic released Fable 5.1 on September 1, 2026, this was the only line that moved — cut 75 percent from $1.00 to $0.25, while input at $10 and output at $50 stayed put (see "Anthropic's Claude Fable 5.1").

4. When does it pay off?

Take a one-million-token prefix used N times.

Three lines matter here.

First, if you will use it once, do not cache. There is no chance to recover the write premium ($2.50 for five minutes, $10 for an hour).

Second, break-even is very low. The five-minute tier clears on the second call, the one-hour tier on the third. Any repetition at all argues for switching it on.

Third, the retention choice is about interval, not count. Ten calls inside three minutes fit comfortably in the five-minute tier; ten calls spread across a working day exceed even the one-hour tier and force rewrites in between. Ask how often, not how many.

5. Four reasons the cache misses

"I turned caching on and the bill didn't move" almost always comes down to one of these.

① Variable content at the front — the most common mistake.

A timestamp, user name or request ID in the first line of the system prompt makes the prefix different on every call, so the cache breaks completely every time. You end up paying write prices repeatedly and never reading once — the worst possible combination.

The fix is to invert the order:

• Front: system instructions, reference documents, codebase, tool definitions — the fixed material
• Back: the user's question, current time, session data — whatever changes

② Below the minimum token length. Every model sets a floor below which no cache forms. Caching a short prompt does nothing at all.

③ The window expired. After five minutes or an hour the stored content is gone and the next call pays write pricing again. Whether reads refresh the timer varies by provider — check the documentation.

④ The prefix itself keeps growing. In conversational and agentic work, each step appends the previous response to the front of the next call. As the prefix grows, the newly grown portion must be written, and the cached share of each call steadily falls.

6. When the unit price falls and the bill doesn't

Anthropic said the Fable 5.1 cache read cut reduces bills by 25 to 45 percent. Outside tests were reported as disagreeing on whether cost per completed task actually falls.

Both can hold, for two reasons.

First, reuse ratios differ by workload. Load one codebase and ask fifty questions and the read cut passes straight through. Send a different document every call and write pricing dominates while the read cut barely lands.

Second, cheaper units invite more usage. A stronger model in agentic work takes more self-directed steps. Cut the unit price to a quarter and quadruple the tokens, and cost per task is unchanged.

The unit price definitely fell. The total is decided by your workload. The first thing to check is what share of your own input tokens are currently cache reads.

7. What is still open

• Prices change often. The figures here are Fable 5.1 as of September 5, 2026, used to illustrate the structure. Do real calculations against the vendor's current table.
• Time-to-live refresh rules differ by provider. Whether a read extends the window materially changes the cost of long-running work.
• Minimum token thresholds can change without notice.
• The break-even table assumes a one-million-token prefix. Output tokens ($50 per million on Fable 5.1) cost the same regardless of caching, so workloads that generate long answers will feel the saving less (see "What AI token pricing is").
• Several providers offer caching, but names, minimum lengths, retention windows and discount rates differ enough that headline prices alone do not support cross-vendor comparison.