Rate Limiting

Rate limiting controls how many requests a client may make in a time window, protecting shared capacity and enforcing fair use with explicit rejection signals.

security

Definition

Rate limiting is an admission-control policy that caps request volume per client, key, or resource over a defined interval. Common algorithms include fixed windows, sliding windows, token buckets, and leaky buckets. A correct implementation defines the key, the limit, the window, the burst allowance, and the response when the limit is exceeded, typically HTTP 429 with a Retry-After header.

Context and scope

Rate limits protect APIs from abuse, accidental retry storms, and noisy-neighbor effects. They also make capacity planning explicit. Limits can apply at the edge, per service, per user, or per API key, and should be enforced close enough to the entry point that rejected requests do not consume downstream resources.

Common misconceptions

Rate limiting is the same as throttling — throttling may delay or shape traffic; rate limiting rejects traffic beyond a limit
A fixed per-second counter is always fair — fixed windows allow bursts at boundaries; sliding windows or token buckets smooth them
A limit protects the whole system — a per-IP limit can be bypassed by distributed clients; choose keys that match the abuse model

Anti-patterns

Returning 500 or silently dropping requests when the limit is exceeded
Counting rejected requests against the limit without documenting it
Applying one global limit that lets a single client starve all others

Related concepts

Related examples

Authoritative source

https://datatracker.ietf.org/doc/draft-ietf-httpapi-ratelimit-headers/

Algorithms and their trade-offs

A fixed window is simple but allows a burst at the end of one window and the start of the next. A sliding window logs timestamps and counts the last interval, which is more accurate at the cost of memory. A token bucket stores tokens that refill at a fixed rate and allows a configured burst. A leaky bucket drains at a constant rate, smoothing output rather than admitting bursts.

class TokenBucket {
  constructor(capacity, refillPerSecond) {
    this.capacity = capacity;
    this.tokens = capacity;
    this.refillPerSecond = refillPerSecond;
    this.updatedAt = Date.now();
  }

  take(count = 1) {
    const now = Date.now();
    const elapsed = (now - this.updatedAt) / 1000;
    this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillPerSecond);
    this.updatedAt = now;
    if (this.tokens < count) return false;
    this.tokens -= count;
    return true;
  }
}

Defining the contract

Document the key (IP, API key, user), the limit, the window, the burst size, and the failure response. Return 429 Too Many Requests with Retry-After and rate-limit headers when available. Rejected requests should be cheap: enforce at the edge before authentication-heavy or database-heavy work.

When to use it

Use rate limiting for public APIs, login endpoints, password reset, search, and expensive computations. Compared to debounce and throttle, which shape client-side event volume, rate limiting is a server-side admission policy. Compared to a retry budget, it limits all traffic, not only retries.

Pitfalls

Distributed enforcement needs shared counters or a gossip/approximate scheme; per-instance limits multiply by the number of instances. A limit keyed by IP breaks for NAT and shared offices. A limit that is too strict blocks legitimate bursts, while one that is too loose provides no protection. Measure rejection rates and latency, and review limits as traffic patterns change.

Token bucket rate limiter