FastAPI — One Slow sync def Endpoint Blocks All Requests

Diagnose FastAPI endpoints declared with sync def that block the event loop, read the threadpool exhaustion pattern, and move CPU or IO-heavy work correctly.

intermediatedebuggingpythonfastapiasyncio

Symptoms

All endpoints, including trivial async ones, stall whenever one heavy sync def endpoint runs
Requests queue behind a slow endpoint with latencies clustering at multiples of its duration
run_in_executor / AnyIO threadpool saturated warnings appear, or starlette threadpool metrics max out

Possible causes

A sync def route runs on the AnyIO threadpool (default 40 threads), and heavy blocking calls occupy them all
A sync route performs long CPU work, and 40+ concurrent calls exhaust the pool, queueing everything else
An async def route calls a blocking library directly, blocking the loop itself — worse than threadpool exhaustion
A shared dependency does blocking IO (requests, heavy DB drivers) inside sync code paths

Diagnosis steps

Classify each route by declaration

grep -Rn "def \|async def" src/routes/ | head -30

Interpretation

Routes declared sync def get the threadpool; async def routes that call blocking code block the loop directly. The declaration decides which failure mode you have.

Measure the stall scope

ab -n 100 -c 50 http://localhost:8000/health & ab -n 10 -c 5 http://localhost:8000/heavy-sync

Interpretation

If /health latency spikes only while the heavy sync endpoint runs, the shared threadpool (not the endpoint itself) is the contention point.

Inspect the threadpool limit

python -c "import anyio; print(anyio.to_thread.current_default_thread_limiter().limit)"

Interpretation

The default 40 is a per-process cap across all sync routes and sync dependencies; heavy fan-out exceeds it quickly.

Root cause

FastAPI runs sync def routes in a threadpool shared by every sync route and dependency, sized 40 by default. When a blocking endpoint saturates the pool, every other sync route queues behind it; when an async def route calls blocking code directly, the single event loop itself stalls and even async endpoints freeze.

Fix

Declare routes async def when the body awaits, keep genuinely blocking calls in sync def routes but bound their concurrency, raise the AnyIO limiter for CPU-light blocking work, and offload CPU-heavy work to process pools or task queues. Never call blocking libraries inside async def.

Risk note

Raising the threadpool limit multiplies threads and their memory; for CPU-bound work it also fights the GIL. Fix the placement first.

Verify the fix

Run the mixed-load test after the fix: trivial endpoints keep sub-millisecond latency while the heavy endpoint saturates its own bounded pool.

Symptom walkthrough

The decisive experiment separates loop stalls from pool stalls. Fire a trivial async endpoint concurrently with the heavy sync one:

GET /health      p99 during idle:      2 ms
GET /health      p99 while /heavy-sync: 4200 ms

If even async endpoints freeze, blocking code lives inside an async def — the loop itself is stuck. If only sync endpoints queue while async stays fast, the shared threadpool is exhausted. Confirm the pool size:

import anyio
anyio.to_thread.current_default_thread_limiter().limit   # 40

Forty tokens serve every sync route and sync dependency in the process; 50 concurrent heavy calls will queue the last ten.

Fix in practice

Route by work type. Blocking IO in moderation stays sync def and rides the pool:

@app.get("/orders")          # sync def: threadpool, fine for moderate blocking IO
def list_orders():
    return legacy_client.fetch_orders()

Raise the limiter only when the work is thread-friendly (IO, releases GIL):

import anyio.to_thread

anyio.to_thread.current_default_thread_limiter().limit = 100

CPU-heavy work leaves threads entirely:

from concurrent.futures import ProcessPoolExecutor

pool = ProcessPoolExecutor(max_workers=4)

@app.get("/report")
async def report():
    result = await loop.run_in_executor(pool, cpu_bound_build, payload)
    return result

And the cardinal rule: async def bodies must await, never call blocking libraries directly — that variant stalls the loop and freezes every endpoint including async ones.