Redis — SLOWLOG Filled with Big-Key Operations

Diagnose Redis latency spikes from big keys and blocking commands via SLOWLOG and LATENCY HISTORY, find the keys with memory usage analysis, and break them up.

intermediatedebuggingredislatencybigkeys

Symptoms

SLOWLOG entries show KEYS, SMEMBERS on huge sets, or HGETALL on million-field hashes taking hundreds of milliseconds
Other clients see periodic latency spikes synchronized with those commands
LATENCY HISTORY shows a sawtooth aligned with report or batch jobs

Possible causes

A set, hash, or list grew to millions of members, and whole-structure commands process it atomically
A reporting job runs KEYS * or a wide SMEMBERS during business hours
A single key stores what should be many keys (user-bucketed data in one hash)
Deletion of a big key with DEL blocks the main thread for the whole free()

Diagnosis steps

Read the slow log entries

redis-cli slowlog get 10 --no-raw

Interpretation

The command name and duration identify the operation class; the client address maps to the calling service.

Find the offending keys

redis-cli --bigkeys -i 0.01; redis-cli memory usage report:monthly

Interpretation

Sampling bigkeys plus memory usage per suspect quantifies which keys are structurally oversized.

Confirm the latency correlation

redis-cli latency history command; redis-cli latency latest

Interpretation

Latency events aligned with the slowlog timestamps tie client-observed spikes to the specific commands.

Root cause

Redis executes commands on a single thread, so one command processing millions of elements delays every other client for its full duration. Big keys make that O(n) cost large; blocking whole-structure operations (KEYS, SMEMBERS, HGETALL, DEL of a big key) turn structural bloat into periodic latency spikes.

Fix

Replace whole-structure reads with cursor iteration (SSCAN, HSCAN), bucket big structures into many small keys with predictable prefixes, use UNLINK instead of DEL for large keys, and replace KEYS with SCAN in application code permanently.

Risk note

SSCAN with a changing key can yield duplicates or misses; make scan-based jobs idempotent and tolerant of mild drift.

Verify the fix

After the restructure, SLOWLOG stays empty during the same workload, LATENCY HISTORY shows no command events, and client p99 returns to baseline.

Symptom walkthrough

The slow log names the operation and its cost:

redis-cli slowlog get 3
1) 1) (integer) 14
   2) (integer) 1729584000
   3) (integer) 384112          <- 384 ms
   4) 1) "HGETALL"
      2) "report:2026-10"

One command, 384 milliseconds of single-threaded time — every other client waited behind it. Find the structural culprit:

redis-cli --bigkeys -i 0.01
# Sampled 100000 keys
# Biggest hash found 'report:2026-10' has 3120000 fields

Three million fields in one hash is the whole disease. The client address in the entry maps to the service that can stop asking for it wholesale.

Fix in practice

Cursor-based reads replace whole-structure operations:

cursor = 0
while True:
    cursor, batch = r.hscan("report:2026-10", cursor=cursor, count=500)
    process(batch)
    if cursor == 0:
        break

Bucket the structure at write time so no key grows unbounded:

# one hash per day instead of one hash per month:
key = f"report:{day:%Y-%m-%d}"
r.hset(key, field, value)

And the two permanent swaps: UNLINK instead of DEL for large keys (background free), and SCAN instead of KEYS anywhere in application code — KEYS in a cron is a scheduled outage.