HTTP — Double Compression Breaks Responses or Wastes Them

Diagnose Content-Encoding applied twice: mojibake or decompression failures downstream, and wasted CPU compressing already-compressed bytes. Fix the pipeline order.

intermediatedebugginghttpcompressionproxy

Symptoms

Clients receive Content-Encoding: gzip twice or br over gzip and fail to decode, or bodies arrive as garbage
Reverse proxy CPU spikes serving image or video traffic because it compresses already-compressed bytes
Some clients work (they tolerate one layer) while strict ones fail with decode errors

Possible causes

The app compresses the body and a proxy compresses again, stacking Content-Encoding values
A proxy compresses responses that are already compressed (images, videos, pre-compressed assets) with no size benefit
Accept-Encoding is transformed or stripped between layers, so an intermediate re-applies encoding the origin already applied
Manual decompression in the app double-unwraps a single encoding layer

Diagnosis steps

Read the served headers end to end

curl -sI -H "Accept-Encoding: gzip, br" https://app.example.com/api/data | grep -iE "content-encoding|content-length|vary"

Interpretation

A single Content-Encoding listing two algorithms (gzip, gzip or br, gzip) is the double-compression receipt; absent Vary: Accept-Encoding warns of cache corruption next.

Verify decodability with strict tools

curl -s --compressed https://app.example.com/api/data | head -c 200

Interpretation

Strict clients failing where lenient ones succeed marks an encoding-stack bug rather than data corruption.

Inspect proxy compression configuration

grep -Rn "gzip\|brotli\|compress" nginx.conf gateway.yaml | head -20

Interpretation

Compression on at multiple layers (app framework, proxy, CDN) with overlapping MIME types is the structural cause.

Root cause

Content-Encoding is meant to be applied once by the layer closest to the wire. When two layers each compress, the headers stack and compliant clients try to decode in the wrong order or fail; when a layer compresses already-compressed media, the CPU cost is paid with zero size benefit.

Fix

Compress at exactly one layer (prefer the outermost proxy for uniform policy), disable compression for already-compressed content types, preserve Accept-Encoding and Vary: Accept-Encoding between layers, and decompress exactly once in middleware when inspection is required.

Risk note

Stripping Accept-Encoding to "disable" compression breaks cache variant selection and disables legitimate client-side decoding. Configure types instead of removing the header.

Verify the fix

Responses carry exactly one Content-Encoding (or none for media), strict clients decode cleanly, and proxy CPU serving media traffic returns to baseline.

Symptom walkthrough

The header tells on the pipeline directly:

$ curl -sI -H "Accept-Encoding: gzip, br" https://app.example.com/api/data
HTTP/1.1 200
Content-Encoding: gzip, gzip     <- applied twice
Vary: Accept-Encoding

Two values stacked means two layers compressed. The client symptoms split by strictness: lenient decoders unwrap one layer and show mojibake; strict decoders refuse outright. The opposite waste shows in CPU graphs: the proxy compressing JPEG and MP4 traffic burns cycles for byte-negative returns, because entropy-coded media does not compress further.

Fix in practice

One layer, chosen deliberately — the outermost proxy is the usual owner:

# nginx (the only compressor)
gzip on;
gzip_comp_level 5;
gzip_types application/json application/javascript text/css;
# NOT image/jpeg video/mp4 application/zip — already compressed

Then turn compression off at the inner layers (the app framework middleware) so the same body is never compressed twice. When middleware must inspect bodies, decompress exactly once and re-set the header honestly:

if (res.getHeader('content-encoding') === 'gzip') {
  const body = await gunzip(raw);
  inspect(body);
  res.removeHeader('content-encoding');  // pass along uncompressed
}

And keep Vary: Accept-Encoding at every caching layer — dropping it lets a compressed variant be served to a client that asked for identity, which is corruption of a different kind.