Node.js — ERR_HTTP2_GOAWAY_SESSION When the Server Sends GOAWAY

Diagnose ERR_HTTP2_GOAWAY_SESSION in Node.js http2 clients, correlate it with server GOAWAY frames and idle timeouts, and drain in-flight streams before reconnecting.

advanceddebuggingnodejshttp2network

Symptoms

Requests fail with Error [ERR_HTTP2_GOAWAY_SESSION]: The HTTP/2 session has been shut down (GOAWAY)
Failures arrive in bursts: hundreds of requests fail within the same second, then everything recovers
A long-lived HTTP/2 connection to an upstream (nginx, Envoy, or a CDN) works for minutes before the burst

Possible causes

The upstream closes idle HTTP/2 sessions and sends GOAWAY, while the client keeps issuing streams on that session
The client ignores the lastProcessedStreamID in the GOAWAY frame and treats all in-flight streams as failed
A proxy in front of the upstream lowers http2 idle timeout below the client keep-alive usage pattern
The client opens a new session per burst and the server closes old sessions at nearly the same moment

Diagnosis steps

Confirm the session teardown with nghttp

nghttp -v https://api.example.com/health 2>&1 | grep -i goaway

Interpretation

A GOAWAY line with last stream id and error code proves the server, not the client, is closing the session.

Raise client visibility with http2 session events

client.on('session', (s) => { s.on('close', () => logSessionClose(s.socket.remoteAddress)); s.on('goaway', (code, lastStreamID) => logGoaway(code, lastStreamID)); });

Interpretation

A goaway event before the close event, with code 0 or 1, points to graceful or protocol-level shutdown from the server side.

Compare idle timeouts on both sides

grep -R "http2_idle_timeout" /etc/nginx/ && grep -R "keepAliveTimeout" src/

Interpretation

If the server idle timeout is lower than the interval between client requests, every quiet period ends with a GOAWAY.

Root cause

The upstream closes the HTTP/2 session with a GOAWAY frame while the Node client still multiplexes new streams onto it. The window between the GOAWAY frame and the client observing the close lets requests fail instead of being retried on a fresh session.

Fix

Treat GOAWAY as a routine lifecycle event: listen for the goaway event, finish or retry streams below lastProcessedStreamID, pin an overall per-request timeout, and set server keepalive to a few seconds longer than the client idle window so the client reconnects first.

Risk note

Blindly retrying every failed stream can duplicate non-idempotent writes. Gate automatic retries on the method and an Idempotency-Key header.

Verify the fix

Load-test with an idle period longer than the upstream timeout and confirm zero ERR_HTTP2_GOAWAY_SESSION in logs while sessions rotate cleanly.

Symptom walkthrough

The error lands on requests that were issued moments before the session teardown. Node wraps the stream error as ERR_HTTP2_GOAWAY_SESSION, which tells you the session, not the individual request, died:

Error [ERR_HTTP2_GOAWAY_SESSION]: The HTTP/2 session has been
shut down (GOAWAY), code: 0, lastProcessedStreamID: 2147483647

The key detail is lastProcessedStreamID. Streams with IDs below or equal to it were processed by the server and their results may already be committed upstream. Anything above it was never seen by the server, which makes those requests safe to retry.

Fix in practice

Instrument the session first, then align timeouts. A resilient client looks like this:

const client = http2.connect(origin);
client.on('goaway', (code, lastStreamID) => {
  metrics.increment('http2.goaway', { code });
  // Streams above lastStreamID were never processed: safe to replay.
});
client.setTimeout(30_000, () => client.close());

On the server side, make the idle timeout longer than the client interval, or ping from the client within the window. Never disable keep-alive to hide the error; that trades a rare error for a full handshake on every request.

Verification

Reproduce the teardown by setting the upstream http2_idle_timeout to 5 seconds and sending one request every 10 seconds for two minutes. Before the fix you see the burst of ERR_HTTP2_GOAWAY_SESSION; after the fix, the goaway counter increments while request failures stay at zero. Watch the retry budget as well: retries should track goaway events, not exceed them, otherwise you are also masking another fault.