OAuth — Concurrent Refreshes Race and Return invalid_grant

Fix 401 storms caused by concurrent refresh token requests invalidating each other, serialize refresh per session, and handle rotation gracefully.

advanceddebuggingoauthauthenticationconcurrency

Symptoms

A burst of API calls triggers multiple simultaneous refreshes; the first succeeds and the rest fail with invalid_grant
Users get logged out randomly under concurrency (multiple tabs, parallel requests) despite valid sessions
Refresh token rotation invalidates the old token, so late-arriving parallel refreshers use the just-revoked value

Possible causes

Multiple tabs or concurrent requests detect expiry at the same moment and each send a refresh request
The provider rotates refresh tokens on use, revoking the presented one — parallel requests race the rotation
Clients retry failed refreshes with the already-consumed token, deepening the failure
A singleton HTTP layer has no per-session refresh coordination, so every caller refreshes independently

Diagnosis steps

Correlate refresh attempts in time

grep "grant_type=refresh_token" access.log | awk '{print $1, $NF}' | head -20

Interpretation

Two or more refreshes within the same second for one session is the race, visible before any code reading.

Read the error details from the provider

grep -A3 "invalid_grant" auth-client.log | head -12

Interpretation

Providers document rotation semantics in the error: token already used, token revoked — each implies the coordination fix rather than a credential problem.

Audit the refresh call sites

grep -Rn "refreshToken\|refresh(" src/http/ src/auth/ | head

Interpretation

Every independent caller that can refresh is a racer; the fix centralizes refresh into one coordinator per session.

Root cause

With refresh token rotation, the provider issues a new token pair and revokes the presented refresh token on each use. Concurrent refreshes from the same session race: the winner rotates the token, and every other in-flight request now presents a revoked token and fails with invalid_grant, cascading into user-visible 401s.

Fix

Serialize refresh per session with a single-flight promise (or distributed lock across instances), queue all concurrent 401-recoveries behind one refresh, propagate the new token to waiting callers, and implement a recovery path with re-authentication when the rotated token is genuinely lost.

Risk note

Retrying the failed refresh with the same old token repeatedly can trip provider lockouts. Retry with the coordinator current token only.

Verify the fix

A concurrency test firing N simultaneous expiring requests performs exactly one refresh, all callers succeed with the new token, and no invalid_grant appears in logs.

Symptom walkthrough

The timeline shows the race before any code does:

09:14:03.112  POST /token grant_type=refresh_token  -> 200 (rotated)
09:14:03.118  POST /token grant_type=refresh_token  -> 400 invalid_grant
09:14:03.121  POST /token grant_type=refresh_token  -> 400 invalid_grant

Three tabs hit expiry together. The first refresh succeeded and rotated the token; the two that followed presented the now-revoked value. The user experience is random logout under load; the provider is behaving exactly as rotation requires — the client lacks coordination.

Fix in practice

Single-flight refresh per session — one in-flight promise, everyone waits on it:

class TokenCoordinator {
  constructor() { this.pending = null; }

  async getAccessToken(session) {
    if (this.pending) return this.pending;      // join the in-flight refresh
    this.pending = this.doRefresh(session)
      .finally(() => { this.pending = null; });
    return this.pending;
  }

  async doRefresh(session) {
    const tokens = await store.load(session);
    const res = await tokenEndpoint({
      grant_type: 'refresh_token',
      refresh_token: tokens.refreshToken,   // current value, once
    });
    await store.save(session, res.data);   // rotated pair persisted
    return res.data.access_token;
  }
}

Across multiple instances, the same coordination needs a shared store or a distributed lock keyed by session. And plan for the genuine loss: if the rotated token is gone (process crash between issue and persist), the recovery path is silent re-authentication, not retry-with-revoked-token, which only deepens lockout risk.