Kafka — transaction timed out before commit

Kafka aborts a transaction because the producer did not commit or abort within the transaction timeout interval, leaving messages in an indeterminate state.

advanceddebuggingkafkatransactions

Symptoms

error: ProducerFencedException or InvalidPidMappingException
Transaction state is aborted after timeout
Consumers see messages that were never committed

Possible causes

The transaction timeout is too short for the processing logic
The producer is slow to commit due to downstream latency
A long-running operation between begin and commit exceeds the timeout

Diagnosis steps

Check the transaction timeout config

grep transaction.timeout.ms config/server.properties

Interpretation

The default is 60 seconds; compare against the processing time.

Monitor transaction metrics

kafka-producer-perf-test --transactional-id tx1

Interpretation

Shows transaction commit rate and abort count.

Root cause

The producer did not commit or abort the transaction within the configured timeout, so the coordinator aborts it and fences the producer.

Fix

Increase transaction.timeout.ms, reduce the processing time between begin and commit, or split the transaction into smaller batches.

Verify the fix

Run a transaction and confirm it commits within the timeout; verify aborted transactions do not produce visible messages.

Symptom walkthrough

Kafka aborts a transaction because the producer did not commit or abort within the transaction timeout interval, leaving messages in an indeterminate state. The error appears in production but may not surface in local development because the conditions that trigger it depend on the deployment environment or runtime configuration. Understanding the full symptom set helps distinguish this issue from similar errors.

Diagnosis in practice

Follow the diagnosis steps in the frontmatter to isolate the cause. Each step includes a command and an interpretation of the output. The goal is to narrow down whether the issue is in the configuration, the runtime, or the application code. Reproduce the error in a minimal setup before applying a fix.

Root cause

The producer did not commit or abort the transaction within the configured timeout, so the coordinator aborts it and fences the producer.

Fix

Increase transaction.timeout.ms, reduce the processing time between begin and commit, or split the transaction into smaller batches. After applying the fix, monitor the system to confirm the error does not recur under load or in different environments.

Verification

Run a transaction and confirm it commits within the timeout; verify aborted transactions do not produce visible messages. If the verification passes, the fix is confirmed; if not, re-examine the diagnosis steps for missed contributing factors.

Diagnostic commands

Run these commands to reproduce and diagnose the issue:

grep transaction.timeout.ms config/server.properties
kafka-producer-perf-test --transactional-id tx1