Negative Caching
Negative caching stores not-found and error responses for a short TTL, absorbing repeated lookups for missing keys and shielding the origin from miss storms.
Definition
Negative caching caches non-successful responses — 404s, empty results, known-absent keys — for a bounded duration. It prevents caches from passing every miss through to the origin, which matters when absent keys are requested at high frequency, as in user lookup by name or vanity URL probing. TTLs must stay short relative to positive entries.
Context and scope
Applied on CDN 404 handling, API gateways caching empty results, and application caches storing tombstones for deleted entities.
Common misconceptions
- Caching misses is pointless — repeated absent-key lookups hit the origin every time without it; the miss traffic is the problem
- Negative entries poison caches for real data — short TTLs bound the staleness window after creation; choose TTLs by expected creation rate
- 404 and 500 deserve the same treatment — client errors mean absent; server errors mean try again soon, and conflating them wastes both semantics
Anti-patterns
- Caching 5xx responses as long as 404s and serving outages as if they were facts
- Tombstoning deleted records without expiry, blocking legitimate recreation
- Caching authorization denials as negative entries and locking users out after one failure
Related concepts
Related examples
Authoritative source
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ConfiguringCaching.html
What it is
A cache usually stores only successes. Queries for keys that do not exist bypass it entirely, so a sweep of random usernames or a probing crawler hammers the origin. Negative caching closes the hole: absence becomes a cacheable, expiring fact.
How it works
The edge or application stores 404 or empty-result responses with a short TTL, distinct from the positive TTL. Creation flips the entry: the next lookup after TTL misses, recaches the success. Tombstones are the write-side equivalent — deleted entities cache their absence so repeated reads stop hitting the store.
When to use it
Username and slug lookups, vanity URL and short-link resolution, permission checks on absent resources, and any endpoint that bots sweep with random identifiers. Apply negative caching wherever miss traffic, not stale data, is the dominant load on the origin.
Pitfalls
TTL sizing versus creation rate: a 60-second negative TTL on names created within seconds frustrates the creator. Distinguish 404 from 410 — permanent absence deserves longer caching. Never negative-cache 5xx or auth denials: one failure must not become a minute of lockout.