阅读中文版本

Build an Idempotency Key Middleware

A middleware that caches responses by an idempotency key header, returning the cached response for duplicate requests and storing in-flight results to prevent double execution.

idempotency-key-middleware/
├── src/middleware.ts
├── src/cache.ts
├── src/inflight.ts
├── tests/idempotency.test.ts
└── package.json

const cached = await store.get(key); if (cached) return res.json(cached);

if (inflight.has(key)) return inflight.get(key).then(r => res.json(r));

inflight.set(key, promise); const r = await handler(); await store.set(key, r, TTL);

if (cached && cached.bodyHash !== hash(body)) return res.status(409).json({error:"key_reuse"});

Project overview

A POST that creates a resource is not idempotent: two calls create two resources. A client that retries a failed request, because of a timeout or a network blip, may create a duplicate, because the first request succeeded but the response was lost. An idempotency key makes the retry safe: the client sends a key, the server caches the response, and a retry with the same key returns the cached response without re-executing.

The middleware sits in front of the handler and checks the idempotency-key header. If the key is in the cache, the cached response is returned. If the key is in-flight, the middleware waits for the in-flight request and returns its result, so two concurrent requests with the same key run the handler once. Otherwise, the middleware runs the handler, caches the response, and returns it.

The in-flight dedup is what prevents double execution under concurrency. Two requests that arrive at the same time with the same key would both miss the cache and both run the handler, if the middleware only checked the cache. The in-flight map holds the promise of the running handler, so the second request awaits the first and returns its result.

The conflict detection is what prevents key reuse. A key that is reused with a different request body is a client bug, because a key must map to one request. The middleware detects the reuse by hashing the body and comparing to the cached hash, and returns a 409, so the client knows the key was reused.

The value of this project is the in-flight dedup, not the cache. Caching a response is trivial; preventing two concurrent requests from both running the handler is the part worth learning, because that is what makes the middleware correct under concurrency, not just under retry.

What you will build

The project delivers five capabilities, each driven by a test:

• Response cache by key
• In-flight dedup
• TTL expiry
• Conflict detection on key reuse
• Store-backed persistence

The middleware checks the idempotency-key header. If the key is in the cache, return the cached response. If the key is in-flight, wait for the in-flight request and return its result. Otherwise, mark the key in-flight, run the handler, cache the response, and clear in-flight. A store backs the cache so it survives a restart.

Architecture and data model

The cache is a store-backed key-value map with a TTL. The store is Redis or a database, so the cache survives a restart and is shared across instances. The key is the idempotency-key header value, and the value is the response body, the status code, and the body hash for conflict detection.

The in-flight map is a process-local map from key to a promise. It is not store-backed, because it is for concurrent requests within a process, not across processes. A cross-process in-flight dedup would use a store-backed lock, which is a distributed lock and out of scope. The process-local map is correct for a single instance and a shared cache is correct for multiple instances with the caveat that two instances may both run the handler if the cache miss is racy.

The middleware runs on every request with an idempotency-key header. It checks the cache, then the in-flight map, then runs the handler. The handler is wrapped in a promise that is stored in the in-flight map, so concurrent requests await it. On completion, the response is cached and the in-flight entry is cleared.

The conflict detection hashes the request body and stores the hash with the cached response. On a cache hit, the middleware recomputes the hash and compares. A mismatch means the key was reused with a different body, and the middleware returns a 409. The hash is a SHA-256 of the body, which is collision-resistant, so a hash match is a body match.

The TTL is set per key, typically 24 hours, which is long enough for a client to retry and short enough to keep the cache bounded. A key that expires and is reused re-executes the handler, which is correct but may surprise a client that expected the old response. The TTL is a trade-off between cache size and retry window.

Prerequisites and setup

You need the following before you begin:

• Node.js 20 LTS or later
• A store for the cache (Redis or a database)

The dependencies cover the core needs:

• ioredis (^5.4.0) — Store-backed idempotency cache
• express (^4.21.0) — HTTP middleware

pnpm init
pnpm add ioredis
pnpm add express
pnpm add -D typescript tsx vitest

Keep secrets and configuration in the environment, and validate them at boot so a misconfigured service fails loudly instead of silently dropping work.

Directory structure

idempotency-key-middleware/ is organized so each concern is testable in isolation:

idempotency-key-middleware/
├── src/middleware.ts
├── src/cache.ts
├── src/inflight.ts
├── tests/idempotency.test.ts
└── package.json

Phase 1: Check cache

On a request with a key, check the store for a cached response and return it if found.

const cached = await store.get(key); if (cached) return res.json(cached);

A cached response is returned without calling the handler.

The phase is driven by a test that asserts the expected behavior, so the implementation is correct before the next phase builds on it.

Phase 2: In-flight dedup

If the key is in-flight, wait for the in-flight request and return its result.

if (inflight.has(key)) return inflight.get(key).then(r => res.json(r));

Two concurrent requests with the same key run the handler once.

The phase is driven by a test that asserts the expected behavior, so the implementation is correct before the next phase builds on it.

Phase 3: Run and cache

Mark the key in-flight, run the handler, cache the response, and clear in-flight.

inflight.set(key, promise); const r = await handler(); await store.set(key, r, TTL);

The response is cached before clearing in-flight, so a late retry hits the cache.

The phase is driven by a test that asserts the expected behavior, so the implementation is correct before the next phase builds on it.

Phase 4: Conflict detection

If a key is reused with a different request body, return a conflict, because a key must map to one request.

if (cached && cached.bodyHash !== hash(body)) return res.status(409).json({error:"key_reuse"});

A key reused with a different body is a client bug, and a 409 tells the client.

The phase is driven by a test that asserts the expected behavior, so the implementation is correct before the next phase builds on it.

Testing

Send a request with a key, then a retry with the same key, and assert the handler runs once and the response is the same; send two concurrent requests with the same key and assert the handler runs once; reuse a key with a different body and assert a 409.

The retry test sends a request with a key, then a retry with the same key and body, and asserts the handler ran once and the response was the same. The concurrency test sends two requests with the same key at the same time and asserts the handler ran once, proving the in-flight dedup.

The conflict test sends a request with a key, then a request with the same key but a different body, and asserts a 409. The no-key test sends a request without the header and asserts the middleware passes through, because idempotency is optional.

The TTL test sets a short TTL, sends a request, waits past the TTL, and asserts a retry re-executes the handler, because the cache expired. The restart test sends a request, restarts the process, and asserts a retry returns the cached response, proving the store backing.

The streaming test sends a request whose handler streams the response and asserts the middleware buffers it to cache, which is a limitation, not a feature. The error test runs a handler that throws and asserts the error is not cached, so a retry re-executes, because a failed request should be retried, not returned from cache.

The cross-instance test runs two instances with a shared store, sends a request to one, and asserts a retry to the other returns the cached response, proving the store backing works across instances.

pnpm vitest run

Observability

Track the response cache by key rate, the idempotency resource usage, and the error count as the three primary signals for the idempotency-key-middleware system. A sudden change in any of these is the signal that something has shifted, not necessarily broken but worth a look before it becomes an incident. Chart the in-flight dedup alongside the response cache by key so a correlation is visible at a glance, because a rising in-flight dedup with a stable response cache by key means the system is doing more work per unit, not that it is failing.

Log every check cache transition with the context that triggered it, because an operator who sees the symptom but not the cause will guess wrong. The log entry should carry the idempotency identifier, the input that caused the transition, and the outcome, so the operator can reconstruct what happened without re-running it. A log without context is a line of text; a log with context is a diagnosis.

Set alerts on the derived metrics, not the raw counters. A rate of change is more actionable than a total, because a total grows monotonically and never tells you when the growth accelerated. Alert on the response cache by key error rate crossing a threshold, the idempotency utilization exceeding a percentage, and the in-flight dedup latency exceeding the SLO. Each alert should have a runbook link, so the operator who receives it at three in the morning knows what to do, not just what is wrong.

The idempotency-key-middleware system should expose its metrics in a format the existing dashboard can scrape, so the observability is additive, not a new tool. A metric that nobody reads is noise, so before adding one, confirm it has a consumer. The dashboard has three panels: throughput, error rate, and resource utilization, each with a threshold line that is the SLO, so a glance tells an operator whether the system is healthy without reading numbers.

Deployment and operations

Run the middleware in front of any non-idempotent endpoint, such as a POST that creates a resource or a PUT that charges a card. The middleware makes the endpoint safe to retry, which is the reliability guarantee a client needs to handle a network blip without creating a duplicate.

Back the cache with a shared store, such as Redis, so multiple instances dedup across processes. A process-local cache only dedups within a process, so two instances may both run the handler for the same key if the requests land on different instances. A shared cache closes this gap, because the second instance checks the shared cache and finds the first response.

Set the TTL from the retry window. A client that retries within 24 hours should find the cached response, so the TTL should be at least 24 hours. A TTL that is shorter than the retry window lets a retry re-execute, which defeats the purpose.

Monitor the cache hit rate and the conflict rate. A high hit rate means clients are retrying, which is expected; a high conflict rate means clients are reusing keys with different bodies, which is a client bug. Both are operator signals.

Do not use the middleware for streaming responses. The middleware buffers the response to cache it, so a streaming response is held in memory and returned as a single body. A streaming idempotency middleware is a different design, and the right answer depends on the streaming semantics.

Security considerations

Treat every input to the idempotency-key-middleware system as untrusted, because the response cache by key path is the one place an external caller controls what the system processes. Validate the input shape before any idempotency operation, and reject early with a clear error rather than letting a malformed input propagate into the check cache phase where it causes a confusing failure that is hard to trace back to its source.

Keep secrets and configuration in the environment, never in the repository. The ioredis credentials, the signing keys, and the service endpoints are all environment variables, read once at boot and validated, so a misconfigured idempotency-key-middleware fails loudly at startup instead of silently misbehaving at runtime. A system that starts with a bad configuration and runs anyway is a system that will produce wrong results for hours before anyone notices.

Rate-limit the response cache by key endpoint per caller, because an unbounded idempotency operation is a resource leak as much as a security problem. A caller that can trigger the in-flight dedup path without limit can exhaust the system, whether intentionally or not. The rate limit is a per-caller ceiling that costs legitimate callers nothing and stops the abuse before it reaches the check cache logic.

Audit-log every state-changing operation with the caller identity, the input, and the timestamp. The audit log is not for debugging, it is for accountability: if something went wrong, the log says who did what and when. The idempotency-key-middleware system writes the audit log to an append-only store that the system itself cannot modify, so a compromised system cannot rewrite its history.

Do not expose internal errors to the caller. A stack trace is a map of the system internals, and an attacker who collects them learns the idempotency structure. Return a generic error with a correlation id, and log the full error internally with the same id, so an operator can trace it without leaking it.

Limitations and next steps

• The cache is store-backed; a store outage means the middleware cannot dedup and must pass through.
• No async handler streaming; the response is buffered to cache it.
• The TTL is per key; a key that expires and is reused re-executes.

The cache is store-backed. A store outage means the middleware cannot dedup, because it cannot check the cache. The middleware should pass through on a store outage, not fail, because the handler is still correct without idempotency, just not safe to retry. A store outage that fails closed would block all requests, which is worse than a duplicate.

There is no async handler streaming. The response is buffered to cache it, so a handler that streams a large response is held in memory and returned as a single body. A streaming idempotency middleware would need to cache the stream, which is a different structure.

The TTL is per key. A key that expires and is reused re-executes the handler, which is correct but may surprise a client that expected the old response. The client should treat a key as unique for the TTL window, and the server should document the TTL so the client knows the window.