The cached prefix is far larger than it needs to be
PrefixAudit rule prefix-bloat · v0.1.0
Why it breaks your cache
Every cached token is re-read on every request at the read rate, and the whole span is rewritten on every miss. Reference material that the model only occasionally needs is being paid for on every single call, and it also pushes up latency. Duplicated blocks pay for themselves twice.
How to fix it
Move reference material, long examples and rarely-used policy behind a tool or a retrieval step so it is fetched on demand. Delete duplicated sections. Keep the cached span to the instructions that apply to every request.
How common is it?
Detected in 36 of 236 production system prompts (15.3%) across 16 vendors.
Estimated impact
Modelled as invalidating 0% of the cached prefix.
Is your prompt affected?
Paste it and find out. Static analysis, in-browser, nothing leaves the page.
Run the audit →