阅读中文版本

Build a Configurable API Gateway

A complete API gateway that routes by path prefix, injects auth headers, applies per-route rate limits and timeouts, and logs structured access events.

api-gateway/
├── src/routes.ts
├── src/proxy.ts
├── src/auth.ts
├── src/limits.ts
├── src/errors.ts
├── tests/gateway.test.ts
└── package.json

const routes = [
  { method: 'GET', prefix: '/users', upstream: 'http://users:8080', limit: 20 },
  { method: 'POST', prefix: '/orders', upstream: 'http://orders:8080', auth: true },
];

const { payload } = await jwtVerify(token, jwks, { issuer, audience });
req.subject = payload.sub;

if (!bucket.take(subject, route.limit)) return res.status(429).json({ error: 'rate_limited' });

const ac = new AbortController();
const t = setTimeout(() => ac.abort(), route.timeoutMs);
const upstream = await fetch(route.upstream + req.url, { signal: ac.signal });

res.status(code).json({ error, requestId: req.id });

Project overview

An API gateway sits between clients and a fleet of services, and almost every incident it causes traces back to a missing bound: no deadline, no rate limit, no consistent error shape. This build treats those bounds as the product rather than as polish added later.

You will route by method and path prefix to upstreams declared in a table, verify bearer tokens against a JWKS, meter each route with its own token bucket, forward with an AbortController deadline, and collapse every upstream failure into one error envelope that carries a request id.

The gateway holds no business state of its own; it is stateless so it can be scaled horizontally and replaced without draining domain data. That is deliberate, because the more a gateway remembers, the harder it becomes to reason about under partial failure.

The tutorial is written for a developer who has proxied requests before but has never had to make the proxy honest. It favors explicit configuration over convention, and it names the failure each bound prevents instead of presenting limits as abstract best practice.

Each phase ends with something observable — a route that resolves, a token that is rejected, a 429, a 504 — so you can stop at any point and still have a gateway that forwards traffic correctly.

What you will build

The finished gateway exposes five capabilities, each with a test that exercises it in isolation:

• Route table
• Authenticate and authorize
• Per-route rate limit
• Forward with deadline
• Normalize errors

Routing is configuration rather than code; identity is checked once at the edge; limits are per route; every upstream call has a ceiling; and errors are a single typed envelope. Together they turn the gateway into a policy choke point instead of a proxy that merely forwards bytes.

The build avoids pulling in a framework for the datapath. express supplies the HTTP surface and middleware chain, undici supplies timeouts and connection reuse, and jose verifies tokens, while routing, bucketing, and error mapping remain ordinary code you can read in one sitting.

Architecture and data model

A route table maps method plus path prefix to an upstream. The gateway authenticates the caller, checks a per-route token bucket, forwards the request with a deadline, and normalizes upstream responses into a single error envelope.

Routing, identity, limiting, and forwarding are four modules with one responsibility each, wired by a request pipeline. A request resolves a route, resolves a subject, checks the bucket, then dispatches, and a failure at any step short-circuits with the shared error envelope.

The route table is plain data — method, prefix, upstream, limit, timeout — validated at boot so a malformed entry fails startup instead of producing a surprising 404 for one path. Keeping it data means adding a route is a config change and reviewing routing is a diff of a list.

The error envelope is the contract clients depend on: an error code plus a request id, with the same id emitted to structured logs. Because every failure path funnels through one formatter, a client can branch on a stable field instead of parsing upstream HTML or guessing from status codes.

Configuration and code are kept apart on purpose: the table says what, the modules say how. A reviewer can change a limit or add an upstream without reading the forwarding code, which keeps the risky changes small and lets the diff stand as the review.

Prerequisites and dependencies

Have these ready before you install anything, since the gateway needs at least one real upstream to forward to.

• Node.js 20 LTS or later
• One or more reachable upstream services
• A shared secret or JWKS for authentication

The runtime dependencies are few on purpose, which keeps the datapath legible:

• express (^4.21.0) — HTTP surface and middleware
• undici (^6.19.0) — Fast HTTP forwarding with timeouts
• jose (^5.9.6) — Verify caller tokens

pnpm init
pnpm add express undici jose
pnpm add -D typescript tsx vitest

Versions are pinned to a major range so an install is reproducible; pin the lockfile in CI. The gateway reads the JWKS URL and upstream base addresses from the environment and validates them at startup, so a missing value fails the boot rather than the first request.

Directory structure

Keep routing, identity, limiting, error shaping, and the server entry in separate files so each can be tested without starting the others:

api-gateway/
├── src/routes.ts
├── src/proxy.ts
├── src/auth.ts
├── src/limits.ts
├── src/errors.ts
├── tests/gateway.test.ts
└── package.json

Phase 1: Route table

Declare routes as data so routing changes need no code.

const routes = [
  { method: 'GET', prefix: '/users', upstream: 'http://users:8080', limit: 20 },
  { method: 'POST', prefix: '/orders', upstream: 'http://orders:8080', auth: true },
];

Configuration-driven routing keeps behavior auditable.

Treat the route table as configuration with defaults: a method, a prefix, an upstream, and optional per-route limit and timeout. Validate that prefixes start with a slash and upstreams parse as URLs, and reject duplicate method-plus-prefix pairs loudly at boot.

Phase 2: Authenticate and authorize

Verify the bearer token and attach the subject to the request.

const { payload } = await jwtVerify(token, jwks, { issuer, audience });
req.subject = payload.sub;

A gateway is a single choke point for identity enforcement.

Verification happens once, eagerly, before any upstream is touched, so an unauthenticated request never consumes a backend slot. Attach the resolved subject to the request and let downstream modules read it rather than re-parsing the header.

Phase 3: Per-route rate limit

Keep a token bucket per route and per subject.

if (!bucket.take(subject, route.limit)) return res.status(429).json({ error: 'rate_limited' });

One noisy client must not exhaust another route.

A token bucket keyed by route and subject means one noisy tenant cannot drain capacity meant for another route. Keep buckets in process memory for a single instance, and note that a real multi-instance deployment must move them to Redis to keep limits honest.

Phase 4: Forward with deadline

Proxy the request with an AbortController timeout.

const ac = new AbortController();
const t = setTimeout(() => ac.abort(), route.timeoutMs);
const upstream = await fetch(route.upstream + req.url, { signal: ac.signal });

Without a deadline a slow upstream consumes gateway slots indefinitely.

Wrap each upstream call in an AbortController tied to the route timeout, and clear the timer in a finally block so a completed request leaves no dangling abort. A slow upstream then fails as a 504 instead of holding a gateway slot open.

Phase 5: Normalize errors

Map upstream failures to a stable error envelope with a request id.

res.status(code).json({ error, requestId: req.id });

Clients depend on a predictable error shape, not upstream HTML.

Normalize both upstream errors and gateway-generated failures through one formatter so clients always receive an error code and a request id. Log the upstream status and latency beside the id, but never echo upstream bodies, which often leak internal hostnames and stack traces.

Testing

Spin up a stub upstream that can delay and fail on demand. Tests assert routing, auth rejection, a 429 after the limit, a 504 on timeout, and that the error envelope contains a request id.

Start a stub upstream that can delay or fail on demand, then drive real HTTP requests through the gateway. Assert that a routed request reaches the stub, that an unauthenticated request is refused, and that the stub receives the injected subject.

Pin the timing behavior with assertions that a call past the route timeout returns 504 and that a burst past the bucket returns 429 with a retry hint. Because the clock and the stub are under test control, these cases stay deterministic rather than flaky.

A proxy suite must also prove the negative: an unknown path is refused before authentication, and an upstream 500 becomes the typed envelope rather than a passthrough of the upstream payload.

The entire suite runs with no external service; the stub and the in-process clock make every case reproducible in CI. That is what lets timeout and limit assertions live as tests rather than as a manual checklist that drifts out of date.

pnpm vitest run

Deployment and operations

Run at least two gateway instances behind a load balancer and share rate-limit state in Redis. Set upstream timeouts below client timeouts and above the upstream p99.

Run at least two stateless instances behind a load balancer and move rate-limit buckets into a shared store so the limit is global rather than per instance. Size the gateway deadline below the client timeout and above the upstream p99, so the gateway, not the client, is first to give up.

Drain connections on deploy: stop accepting new work, let in-flight requests finish within the deadline, then exit. Because the gateway holds no state, a rollout is a matter of cycling instances one at a time while watching the per-route error rate.

Keep the JWKS cached with a TTL and refresh on an unknown key id so identity-provider key rotation does not become a hard outage. Alert when the refresh rate spikes, which usually means the cache is too small or the TTL too short.

When the shared limiter store is unavailable, decide explicitly whether the gateway fails closed or falls back to a conservative local limit. Leaving that behavior to chance is how a cache outage becomes a flood, so make the choice visible in code and in a test.

Security considerations

Strip hop-by-hop headers, never forward the caller token verbatim unless required, and validate the path to block traversal. Reject requests with an unknown route before authenticating to save work.

The gateway is a choke point, so it is the right place to strip hop-by-hop headers, reject oversized bodies before buffering them, and refuse unknown routes before spending work on authentication. Validate the outbound path against traversal so a crafted URL cannot reach an unexpected upstream path.

Never forward the caller credentials blindly to upstreams; exchange or scope them deliberately. Treat the JWKS URL and the accepted algorithms as configuration checked at boot, since silently accepting an unexpected algorithm reopens algorithm-confusion attacks.

Keep upstream addresses and secrets in the environment or a secret manager, not in a route table that ships in the repository. Log the subject as a hash rather than the raw identifier, so an access log stays useful for debugging without becoming a record of who did what.

Limit what the gateway reveals in rate-limit headers, and keep upstream host addresses out of error responses. The gateway is a boundary, so every field it emits to a client should be a deliberate choice rather than whatever the upstream happened to say.

Observability

Log method, route, subject hash, status, upstream latency, and request id for every request. Track per-route error rates and alert on a jump that correlates with an upstream deploy.

Emit one structured line per request with method, route, subject hash, status, upstream latency, and request id. The request id is the thread that ties a client complaint to the gateway decision and to the upstream log entry for the same call.

Watch per-route error rate and p99 latency, and alert on a step change rather than a single spike. A gateway that suddenly returns 429 for one route usually means a downstream deploy slowed a dependency, not that the limit is wrong.

Track upstream pool saturation and connection reuse; a growing pool or a climbing connect time is the early sign that an upstream is degrading, and it shows up before client-visible errors do.

Chart the share of requests rejected by auth, by limit, and by route-not-found separately. A change in the mix points at the cause faster than an aggregate error rate can, because each rejection class has a different owner.

Troubleshooting

Client timeouts before the gateway responds. The gateway upstream timeout exceeded the client timeout; keep the gateway deadline strictly smaller.

Rate limit is global instead of per route. The bucket key omitted the route, so unrelated routes shared one quota.

Keep the gateway dashboards separate from upstream dashboards. When a request fails, the request id lets you decide in seconds whether the fault is the route table, the limiter, or the upstream, instead of guessing from a client-side timeout.

Limitations and next steps

• Request and response bodies are streamed without transformation.
• The route table is loaded at boot and not hot-reloaded.
• WebSocket proxying is not implemented.

Bodies are streamed without inspection or transformation, so response filtering and request rewriting are out of scope, and WebSocket and HTTP/2 proxying are not implemented because the forwarding path assumes plain HTTP requests.

The route table is loaded once at boot and is not hot-reloaded, so a routing change is a deploy. The token bucket lives in process memory, which is correct for one instance but must move to a shared store before scaling out.

There is no service discovery or health-based ejection yet; an upstream address is static until you change config. Adding a discovery client and passive health checks is the natural next step once the fleet grows beyond a handful of services.

There is no response caching or request collapsing, so a chatty client still produces one upstream call per request. A short-TTL cache for idempotent reads is the cheapest next improvement once the datapath is trusted.