Python: threading deadlock with locks

Diagnose Python threading deadlocks from lock ordering, non-reentrant locks, and forgetting to release, and fix them with timeouts and lock ordering.

advanceddebuggingpythonconcurrencythreading

Symptoms

The program hangs with threads blocked and no output
A non-reentrant Lock is acquired twice in the same thread
Two threads each hold one lock and wait for the other

Possible causes

Two threads acquire the same two locks in opposite order
A Lock is acquired again before release in the same thread (needs RLock)
An exception skips the release, leaving the lock held
A condition wait is signalled without holding the lock correctly

Diagnosis steps

Dump all thread stacks while hung

faulthandler.dump_traceback_later(10) or send SIGABRT

Interpretation

Stacks show which locks each thread holds and waits on, exposing the cycle.

Check lock acquisition order across threads

Interpretation

A consistent ordering of locks removes the circular wait that causes deadlock.

Guard every acquire with a timeout

lock.acquire(timeout=5)

Interpretation

A false return with a logged message turns a silent hang into a detectable failure.

Root cause

A deadlock needs a circular wait: thread A holds lock 1 and waits for lock 2 while thread B holds lock 2 and waits for lock 1. Python locks are also non-reentrant by default, so a thread re-acquiring its own lock blocks forever. Missing release on an exception path holds a lock indefinitely.

Fix

Always acquire multiple locks in a global, documented order so no cycle can form, and use context managers to guarantee release even on exceptions. Use RLock only where genuine reentrancy is needed, add timeouts to acquire, and prefer higher-level primitives such as a single lock or a queue to avoid multiple-lock protocols entirely.

Verify the fix

Run the previously deadlocking workload under load; it completes with no blocked threads, and a timeout-enabled acquire test asserts lock acquisition succeeds within the limit.

Symptoms

The process hangs with several threads blocked and no further output. CPU is idle because every thread is waiting, which is the signature of a deadlock rather than a busy loop.

Diagnosis

Dump thread stacks while hung:

import faulthandler
faulthandler.dump_traceback_later(10, repeat=False)

Stacks reveal a thread holding lock A waiting for B and another holding B waiting for A.

Fix

Order locks and guarantee release:

import threading

_lo = threading.Lock()   # always acquire in this order
_hi = threading.Lock()

def transfer():
    with _lo, _hi:       # consistent order, released on exception
        do_work()

lock = threading.Lock()
if not lock.acquire(timeout=5):
    raise TimeoutError('lock contention')

Use RLock only for genuine reentrancy.

Verification

Run the deadlocking workload; it finishes with no blocked threads, and a test asserting acquire(timeout=…) succeeds confirms the ordering removes the cycle.

Additional Notes

A deadlock needs a cycle, so eliminating cycles through a documented global lock order is the structural fix rather than a per-call workaround. Acquiring with a timeout converts a silent hang into a detectable error that can be logged with the contended lock name, which makes future occurrences diagnosable instead of invisible. Context managers guarantee release even when an exception unwinds the stack, and reserving reentrant locks only for genuine reentrancy keeps the non-reentrant guarantees that make lock misuse fail fast during testing rather than in production.