OpenAI prompt caching: what actually breaks it
OpenAI caching is automatic above the minimum — there is no marker to forget. That removes one failure mode and hides the rest, because a cache miss looks identical to a cold start.
The numbers
| Value | |
|---|---|
| Base input price | $5.00 / MTok |
| Cache read | 0.1× base → $0.50 / MTok |
| Cache write | 1.25× base → $6.25 / MTok |
| Minimum cacheable prefix | 1,024 tokens |
| Breakeven | 0.28 reads per cached write |
Breakeven is not one read. At 1.25× write and
0.1× read, caching only pays for itself after 0.28 reads of the same
prefix. Derived as (writeMult − 1) ÷ (1 − readMult). Below that, caching costs you money.
No marker to set, no error to read
Because caching is automatic, the only lever you control is prefix stability. A
rendered timestamp or a per-request ID inside the first block changes the bytes and the cache never
engages — and there is no cache_control mistake to find, because you never wrote one.
OpenAI does expose miss reasons through prompt_cache_diagnostics:
model_changed, prompt_cache_key_changed, service_tier_changed,
tools_changed, input_changed and others. Log them; they are the only signal you get.
What breaks the cache
Caching is an exact-prefix match. The first changed byte invalidates everything after it, and no provider raises an error. In 236 production system prompts:
- 76 (32.2%) rendered a date or time inside the cached prefix
- 68 (28.8%) were below the minimum cacheable length
- 36 (15.3%) carried a prefix far larger than it needed to be
- 27 (11.4%) kept mutable working memory inside the prefix
The fix
Order the request so the cached span is byte-identical on every call:
static system instructions ← cached tool definitions ← cached, breakpoint here --- cache breakpoint --- volatile context, dates, IDs ← after the breakpoint user message ← never cached
Content after the breakpoint costs only its own write. The same content before it invalidates everything downstream. Same bytes, different order, completely different bill.
Check your own prompt
Paste a system prompt or a raw API request. Get a score, the offending lines, and a modelled monthly cost. Runs in your browser — no API keys, no account, nothing leaves the page.
Run the audit →Or gate it in CI: npx prefix-audit prompts/ --provider openai