ZooKeeper — connection loss during operation

ZooKeeper client throws ConnectionLossException because the session expired or the server is unreachable, leaving ephemeral nodes in an indeterminate state.

advanceddebuggingzookeeperdistributed

Symptoms

error: KeeperErrorCode = ConnectionLoss
Ephemeral nodes disappear after reconnection
Watches fire with expired events after reconnect

Possible causes

The ZooKeeper server is down or network partitioned
The session timeout is too short for the network latency
The client did not handle the expired session by re-establishing

Diagnosis steps

Check server status

echo ruok | nc zookeeper 2181

Interpretation

Returns imok if the server is healthy; otherwise the server is down.

Review session timeout

grep -n "sessionTimeout" src/zk-client.ts

Interpretation

Compare against network round-trip time; too short causes spurious expirations.

Root cause

The ZooKeeper session expired or the connection was lost during an operation, so the server discards ephemeral nodes and watches for that session.

Fix

Reconnect with a new session, recreate ephemeral nodes and watches, and retry the operation. Increase session timeout for high-latency networks.

Verify the fix

Reconnect and confirm ephemeral nodes are recreated; verify watches fire correctly after reconnection.

Symptom walkthrough

ZooKeeper client throws ConnectionLossException because the session expired or the server is unreachable, leaving ephemeral nodes in an indeterminate state. The error appears in production but may not surface in local development because the conditions that trigger it depend on the deployment environment or runtime configuration. Understanding the full symptom set helps distinguish this issue from similar errors.

Diagnosis in practice

Follow the diagnosis steps in the frontmatter to isolate the cause. Each step includes a command and an interpretation of the output. The goal is to narrow down whether the issue is in the configuration, the runtime, or the application code. Reproduce the error in a minimal setup before applying a fix.

Root cause

The ZooKeeper session expired or the connection was lost during an operation, so the server discards ephemeral nodes and watches for that session.

Fix

Reconnect with a new session, recreate ephemeral nodes and watches, and retry the operation. Increase session timeout for high-latency networks. After applying the fix, monitor the system to confirm the error does not recur under load or in different environments.

Verification

Reconnect and confirm ephemeral nodes are recreated; verify watches fire correctly after reconnection. If the verification passes, the fix is confirmed; if not, re-examine the diagnosis steps for missed contributing factors.

Diagnostic commands

Run these commands to reproduce and diagnose the issue:

echo ruok | nc zookeeper 2181
grep -n "sessionTimeout" src/zk-client.ts