RabbitMQ queue backlog — Find the Bottleneck Behind Growing Depth

Diagnose RabbitMQ queue backlog by comparing ready, unacknowledged, and consumer counts, then fix slow consumers, prefetch, requeue loops, and memory alarms.

intermediatedebuggingrabbitmqqueues

Symptoms

Queue depth grows without bound while the queue still has consumers
messages_ready is high and messages_unacknowledged keeps climbing
Publishers block or connections stall when the memory or disk alarm fires
Message age in the queue exceeds the business latency target

Possible causes

Consumer throughput is lower than the publish rate because of slow downstream calls
prefetch_count is too high, so one consumer hoards messages it cannot process in time
A poison message is rejected with requeue=true and loops forever
Consumers disconnect repeatedly and unacknowledged messages are returned to the queue

Diagnosis steps

Measure ready, unacked, and consumer counts per queue

rabbitmqctl list_queues name type messages messages_ready messages_unacknowledged consumers consumer_utilisation

Interpretation

High messages_ready with healthy consumers means consumers are too slow. High unacknowledged with low consumers means work is checked out but not acknowledged. Zero consumers means nobody is bound to the queue.

Check broker alarms that block publishers

rabbitmq-diagnostics check_alarms && rabbitmqctl status | grep -A4 -E 'memory|disk_free'

Interpretation

An active memory or disk alarm makes publishers block and connections stall. The status output shows the watermark and the current value that triggered it.

Inspect consumer delivery settings

rabbitmqctl list_consumers queue_name channel_pid prefetch_count active ack_required

Interpretation

A prefetch_count of 0 means unlimited delivery, which lets one consumer take a large share. Inactive consumers still hold unacked messages.

Look for redelivery and connection churn

rabbitmqctl list_queues name message_stats.redeliver message_stats.publish message_stats.ack

Interpretation

Redeliveries close to publishes indicate a requeue loop or consumers dying before ack. Compare publish and ack rates over the same interval.

Root cause

RabbitMQ keeps a message ready until a consumer receives it, then moves it to unacknowledged until ack or reject. If consumption plus ack is slower than publishing, the backlog grows even though every component looks alive. Unlimited prefetch, poison-message requeues, or repeated consumer disconnects remove capacity without reducing the incoming rate, and a memory alarm eventually stops all publishers.

Fix

Add consumers or increase per-message throughput, set a bounded prefetch_count such as 20 to 50, and acknowledge only after the work is durable. Route repeatedly failing messages to a dead-letter exchange instead of requeueing them, use quorum queues for durability, and add publisher confirms so producers can apply backpressure. Raise the memory watermark only after fixing the consumer bottleneck.

Risk note

Purging a queue destroys messages. Lowering prefetch can reduce throughput if processing is fast, and requeue=false without a DLX discards failures. Quorum queues cost more disk and memory than classic mirrored queues.

Verify the fix

Watch messages_ready and message age over a full traffic cycle; both should stabilize or fall while ack rate tracks publish rate. Confirm redeliveries stop rising and no memory or disk alarm is active.

Depth alone does not say why

A RabbitMQ queue holds messages in two places that matter: messages_ready waiting for delivery, and messages_unacknowledged already delivered but not acknowledged. The ratio tells you whether consumers are too slow to take work or too slow to finish it.

rabbitmqctl list_queues name messages messages_ready \
  messages_unacknowledged consumers consumer_utilisation

If messages_ready grows and consumer utilisation is near 1, the consumers are saturated. If unacknowledged is high with few consumers, work is checked out but stuck.

Rule out broker alarms

rabbitmq-diagnostics check_alarms
rabbitmqctl status | grep -A4 -E 'memory|disk_free'

An active memory or disk alarm blocks publishers, so producers appear hung while the queue itself is fine. Fixing the alarm without fixing the backlog only postpones the next stall.

Fix the pipeline

Bound prefetch_count so no consumer hoards messages.
Ack only after the downstream write succeeds; use basic_nack with requeue=false plus a dead-letter exchange for poison messages.
Add consumers or batch downstream calls to raise throughput.
Use publisher confirms so the producer notices backpressure instead of buffering without limit.
Prefer quorum queues when durability matters, and monitor message age rather than only depth.

publish rate 800/s, ack rate 620/s -> depth grows 180/s
add 2 consumers -> ack rate 850/s -> depth drains

Verify

After the change, confirm messages_ready plateaus and message age returns below the SLA. Check message_stats.redeliver against message_stats.publish: redeliveries should stop tracking publishes. Finally, confirm no alarm is active and that a forced consumer restart does not create a new spike, which would mean unacked work is still too large.