Python — urllib3 Connection Pool Is Full Warning and Slowdown

Address urllib3 connection pool is full discarding connection warnings, size the pool to real concurrency, and stop discarding keep-alive connections on every request.

intermediatedebuggingpythonhttpperformance

Symptoms

Connection pool is full, discarding connection: api.example.com warnings flood the logs under load
Request latency rises with concurrency even though the server reports idle capacity
TIME_WAIT sockets accumulate on the client host proportional to request volume

Possible causes

The pool size (pool_connections, pool_maxsize) is smaller than actual concurrent requests, so extra connections are created then discarded after each use
A requests.Session is created per request instead of shared, defeating pooling entirely
Each task creates its own adapter configuration, multiplying pools that are individually too small
pool_block=False default means overflow connections are silently created and torn down instead of queuing

Diagnosis steps

Count actual concurrency

grep -c "Connection pool is full" app.log; ss -tan state time-wait | wc -l

Interpretation

Warning volume proportional to request count and a large TIME_WAIT population confirm churn: connections created, used once, discarded.

Audit session usage

grep -Rn "requests.Session()\|requests.get(" src/ | head -20

Interpretation

A Session per request or module-level requests.get calls bypass pooling; shared sessions concentrate the pool where sizing matters.

Check adapter configuration

grep -Rn "pool_maxsize\|pool_connections\|HTTPAdapter" src/ | head

Interpretation

Missing or small pool_maxsize versus worker count predicts the discard rate precisely.

Root cause

urllib3 pools up to pool_maxsize persistent connections per host. When concurrent requests exceed that, extra connections are created to satisfy demand and then discarded instead of pooled, because the pool is full. The warning is the discard notice; the cost is a full TCP and TLS handshake per overflowing request plus growing TIME_WAIT on the client.

Fix

Share one Session per process, mount an HTTPAdapter sized to real concurrency (pool_maxsize at or above the thread or task count), enable retries on the adapter, and turn on pool_block=True when you want overflow to queue rather than churn.

Risk note

pool_block=True with an undersized pool converts churn into queuing latency. Size first, then decide the overflow policy.

Verify the fix

Under the same load, assert zero pool-full warnings, TIME_WAIT count bounded, and p99 latency reduced from the handshake elimination.

Symptom walkthrough

The warning is the discard receipt:

2026-10-22 10:02:11 WARNING urllib3.connectionpool
Connection pool is full, discarding connection: api.example.com.
Pool size: 10

Pool size 10 against 32 concurrent workers means 22 connections per burst are built, used once, and thrown away — every one paying TCP plus TLS handshake. The client-side evidence stacks up in the socket table:

ss -tan state time-wait | wc -l
# 4821   <- proportional to discarded connections

Latency rises with concurrency while the server sits idle, because the bottleneck is handshakes on the client host, not service capacity.

Fix in practice

One session per process, one adapter sized to the truth:

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

session = requests.Session()
retry = Retry(total=3, backoff_factor=0.2, status_forcelist=[502, 503, 504])
adapter = HTTPAdapter(
    pool_connections=4,        # distinct hosts you call
    pool_maxsize=64,           # >= your real concurrency
    max_retries=retry,
)
session.mount("https://", adapter)
session.mount("http://", adapter)

Deliver the session to workers by reference — a single module-level instance or dependency injection — never a factory call per request. When you prefer queueing over churn, add pool_block=True, but only after sizing: a blocking undersized pool converts one failure mode into another.

Verification

　　Replay the exact load profile that produced the warnings. Success is threefold: zero pool-is-full warnings in the log, TIME_WAIT count bounded to a small steady state, and p99 latency measurably lower because overflow requests stopped paying handshakes. Then stress the boundary deliberately: drop pool_maxsize below concurrency in a staging copy and confirm the warning returns — proving the knob you turned is the one that governs the behavior.