Cache Negative Results Safely

Cache not-found results for a short period to protect against repeated lookups, but keep the TTL short and bypass it on user-visible writes.

~30 mincachingperformancereliability

Task

Add negative caching to stop repeated expensive misses without caching a result that may soon exist.

When to use this

An endpoint looks up a key that does not exist, so every request hits the database with a doomed query, and caching misses forever would hide a newly created key.

Prerequisites

A shared cache store
A bounded maximum TTL
A way to invalidate on create

Steps

1. Store a sentinel value for confirmed not-found results with a short TTL.

Expected output: Repeated misses do not hit the database.

2. On a create or update of the same key, delete the negative entry immediately.

Expected output: A newly created key is visible at once.

3. Keep the negative TTL much shorter than the positive TTL.

Expected output: New data is not hidden for long even if invalidation is missed.

4. Distinguish cache errors from real misses so a cache outage does not become an outage of the endpoint.

redis-cli SET "miss:user:404" "__absent__" EX 30

Expected output: The database remains the fallback.

Troubleshooting

Problem

New key hidden after creation

Solution

Invalidate the negative entry in the same write path and keep the negative TTL short.

Problem

Negative entries fill the cache

Solution

Bound the key space and the TTL; negative caching is for hot misses, not arbitrary keys.

Limitations

Negative caching temporarily hides data that is about to exist, so it is unsafe for read-after-write flows without invalidation.
It can be exploited to poison lookups if keys are attacker-chosen without rate limiting.

Context

Add negative caching to stop repeated expensive misses without caching a result that may soon exist. An endpoint looks up a key that does not exist, so every request hits the database with a doomed query, and caching misses forever would hide a newly created key.

Steps

Store a sentinel value for confirmed not-found results with a short TTL. Expected: Repeated misses do not hit the database.
On a create or update of the same key, delete the negative entry immediately. Expected: A newly created key is visible at once.
Keep the negative TTL much shorter than the positive TTL. Expected: New data is not hidden for long even if invalidation is missed.
Distinguish cache errors from real misses so a cache outage does not become an outage of the endpoint. Expected: The database remains the fallback.

const cached = await cache.get(key);
if (cached === '__absent__') return null;
if (cached) return cached;
const row = await db.find(id);
await cache.set(key, row ?? '__absent__', { ttl: row ? 600 : 30 });
return row;

Verify

Repeated misses do not hit the database.
A newly created key is visible at once.
New data is not hidden for long even if invalidation is missed.
The database remains the fallback.

Limits

Negative caching temporarily hides data that is about to exist, so it is unsafe for read-after-write flows without invalidation.
It can be exploited to poison lookups if keys are attacker-chosen without rate limiting.