HTTP — Chunked Responses Truncated by a Proxy

Debug truncated chunked responses where intermediaries mis-handle Transfer-Encoding, compare Content-Length expectations, and fix buffering or streaming layers.

intermediatedebugginghttpstreamingproxy

Symptoms

Clients receive partial bodies ending mid-chunk with no error, or a protocol error when the terminator never arrives
Streaming endpoints work directly against the app but truncate behind nginx or an API gateway
Downloaded files have wrong sizes; the tail is missing but the connection closes cleanly

Possible causes

A proxy buffers the chunked stream and closes early on its own timeout, truncating before the final zero chunk
An intermediary rewrites Transfer-Encoding into Content-Length incorrectly, declaring a shorter body
Response body filters (compression modules, WAF inspection) abort mid-stream on unexpected bytes
The origin writes the terminating zero-length chunk after a delay longer than proxy idle timeouts

Diagnosis steps

Compare byte counts end to end

curl -s https://direct.internal/stream | wc -c; curl -s https://app.example.com/stream | wc -c

Interpretation

The direct count is ground truth; the proxied count missing the tail localizes the truncation to the intermediate path.

Watch the framing on the wire

curl -v --raw https://app.example.com/stream 2>&1 | grep -E "Transfer-Encoding|Content-Length|< [0-9a-f]" | tail -10

Interpretation

Missing final zero-size chunk or a rewritten Content-Length smaller than the body shows the framing violation directly.

Check proxy buffering and timeouts

grep -Rn "proxy_buffering\|proxy_read_timeout\|chunked" nginx.conf gateway.yaml

Interpretation

Buffering on with a short read timeout truncates slow streams exactly at the timeout boundary; the truncation time equals the configured value.

Root cause

Chunked transfer relies on strict framing: chunks with sizes, a terminating zero-length chunk, and trailers. An intermediary that buffers, re-frames, or times out mid-stream breaks the contract — the client receives a body that ends early, sometimes with a clean TCP close that hides the truncation entirely.

Fix

Disable proxy buffering for streaming routes (proxy_buffering off with X-Accel-Buffering), raise read timeouts beyond the slowest legitimate chunk interval, keep the origin sending heartbeats or progress chunks during long gaps, and fix or bypass response-body rewriters that abort on stream content.

Risk note

Disabling buffering on high-traffic routes shifts concurrency cost to upstream workers. Scope it to the streaming endpoints that need it.

Verify the fix

Streamed bodies byte-match the origin output through the full path; a slow-chunk test (gaps near the timeout) completes without truncation.

Symptom walkthrough

The truncation is quantified by two commands:

$ curl -s https://direct.internal/stream | wc -c
10485760                                  <- ground truth
$ curl -s https://app.example.com/stream | wc -c
9911478                                   <- missing the tail

Then watch the framing itself with curl -v –raw: a healthy chunked response ends with a zero-size chunk (the lone 0 before the final CRLF). A stream that simply stops — connection closed, no terminator, no error — was cut by an intermediary, and when the cut aligns exactly with a configured proxy_read_timeout, the config line is the confession. The rewritten-Content-Length variant is sneakier: the proxy declares a smaller body than it delivers or vice versa, and strict clients fail while lenient ones show silently wrong file sizes.

Fix in practice

Route-scoped streaming configuration:

location /stream/ {
    proxy_buffering off;              # stream through, do not accumulate
    proxy_read_timeout 3600s;         # survive slow producers
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    chunked_transfer_encoding on;     # pass framing through honestly
}

Keep the origin heart-healthy during long gaps — a comment or progress chunk every few seconds keeps idle timeouts from firing on legitimately slow generations. And audit any response-body rewriter (compression modules, WAF inspection) on the path: one that aborts on unexpected bytes mid-stream produces exactly this truncation with zero log lines at the origin.