阅读中文版本

Timeout Budget

A timeout budget is a deadline shared across a request chain, so each call gets only the time remaining and retries cannot exceed the caller's patience.

One deadline instead of many timeouts

A fixed timeout at each layer looks safe in isolation. In a chain of five calls, five 10-second timeouts can keep a request alive for 50 seconds, long after the user has closed the tab. The upper layers time out first, but the lower layers keep working, holding connections and consuming capacity for work nobody will read.

A timeout budget reverses the calculation. The edge sets a deadline, for example 2 seconds. Before calling a dependency, the handler subtracts the time already spent and passes the remainder. If 1.6 seconds are left, the next call gets 1.6 seconds minus a small safety margin. When the budget is exhausted, the handler stops and returns a timeout response without starting new work.

export function remainingMs(deadline: number): number {
  return Math.max(0, deadline - Date.now());
}

export async function callWithinBudget<T>(
  deadline: number,
  operation: (signal: AbortSignal) => Promise<T>,
): Promise<T> {
  const budget = remainingMs(deadline);
  if (budget < 50) throw new Error('request budget exhausted');

  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), budget);
  try {
    return await operation(controller.signal);
  } finally {
    clearTimeout(timer);
  }
}

The deadline is a number carried in the request context. In Node.js, accept an AbortSignal from the caller and combine it with the local timer; in Go, use context.WithDeadline derived from the inbound context. Do not create a fresh background context at an intermediate layer, or the deadline is lost.

Spend the budget on retries too

Retries are not free. If a retry policy allows three attempts with two-second timeouts, the worst case is six seconds plus backoff. That policy is only valid when the remaining budget can accommodate it. A retry that starts with 80 milliseconds left cannot succeed within a reasonable time and should not run.

Budget-aware retries check the deadline before each attempt and cap the attempt timeout to the remaining time. They also distinguish retryable failures from caller cancellations. If the caller aborted because it no longer needs the result, retrying is wasted work and may hide the cancellation.

Propagate cancellation

A timeout that only stops the caller from waiting is not a cancellation. The remote service may continue processing, and the client may still hold a connection. Propagate the cancellation signal through HTTP requests, database drivers, and message consumers. Record why work stopped: deadline exceeded, caller cancelled, or shutdown. Those reasons lead to different metrics and different responses.

Trade-offs

Shorter budgets protect latency but increase the chance of partial results and retries. Very long budgets hide dependency problems and hold resources. Measure the end-to-end latency distribution and set the edge budget from the product’s tolerance, then allocate a fraction to each layer with margin for serialization and network jitter. Revisit the budget when the call graph changes; a new hop without budget propagation silently breaks the invariant.

When to use it

Use a timeout budget when a request crosses more than one service or retries are enabled. The main advantage is predictable end-to-end latency: the caller’s deadline is the only limit that matters. The drawback is that every layer must accept and forward a deadline, which requires shared context plumbing. Compared to independent per-hop timeouts, a budget is more work to implement but avoids hidden additive latency.