Kubernetes — CrashLoopBackOff

Diagnose CrashLoopBackOff from pod events, previous logs, and exit codes, then fix the application, probe, configuration, or resource limit that restarts the container.

intermediatedebuggingkubernetescontainers

Symptoms

kubectl get pods shows STATUS CrashLoopBackOff with an increasing restart count
The container starts, exits, and restarts with exponential backoff
Logs from the current container are empty because it already restarted
The pod worked locally but fails in the cluster

Possible causes

The application exits non-zero because of a missing environment variable, secret, or config file
The process is killed with exit code 137 because it exceeds the memory limit
A liveness probe fails and Kubernetes restarts a healthy but slow-starting process
The command or entrypoint is wrong, or a required file is not in the image
A dependency is unavailable at startup and the app chooses to exit

Diagnosis steps

Read the pod events and last state

kubectl describe pod POD -n NAMESPACE

Interpretation

Events show image pulls, probe failures, OOMKilled, and the last termination reason. The exit code distinguishes an application error from a signal kill.

Read logs from the previous container instance

kubectl logs POD -n NAMESPACE --previous

Interpretation

The current container may have no logs yet. Previous logs show why the last instance exited.

Inspect the effective command, environment, and mounts

kubectl get pod POD -n NAMESPACE -o jsonpath='{.spec.containers[0].command} {.spec.containers[0].args}'

Interpretation

A missing config mount, wrong command, or absent secret appears here even when the image is correct.

Root cause

The container exits or is killed repeatedly. Kubernetes restarts it with exponential backoff, but the restart is a symptom of an application error, a resource limit, a probe misconfiguration, or a missing dependency.

Fix

Fix the underlying exit cause: provide the missing configuration, raise or right-size memory limits, add a startup probe for slow initialization, correct the command, or make startup resilient to a temporarily unavailable dependency. Do not disable probes to hide a failing app.

Risk note

Removing liveness probes or raising limits without diagnosis can turn a crash loop into a silent hang or a node-level OOM.

Verify the fix

Confirm the pod reaches Running and Ready, restart count stops increasing, and logs show a successful startup. Delete the pod and let the controller recreate it to verify the fix survives a fresh schedule. Check that resource usage stays below the limit.

Exit code first

CrashLoopBackOff means the kubelet is restarting a container that keeps exiting. The restart policy is working; the application or its configuration is not. Start with kubectl describe pod and find the last termination reason and exit code. 137 usually means SIGKILL, often an OOM kill; 1 is a generic application error; 143 is SIGTERM, often a failed liveness probe or a shutdown.

kubectl describe pod api-7d9f -n prod | sed -n '/Events:/,$p'
kubectl logs api-7d9f -n prod --previous

Distinguish the failure classes

A missing secret or config map produces a clear error in the previous logs. An OOM kill shows OOMKilled and a memory limit in the pod spec. A liveness probe failure shows probe events and often no application error, because the process was healthy but slow. A wrong command exits immediately with a shell error. Each class has a different fix; raising the restart count or disabling probes does not address any of them.

Probes and startup

A slow-starting app with an aggressive liveness probe can be killed before it finishes initializing. Add a startup probe or increase initialDelaySeconds for liveness, and keep readiness separate. A liveness probe should test the process, not a dependency that may be temporarily down; otherwise a database outage restarts every pod.

Fix and verify

Fix the configuration, command, memory limit, or probe, then confirm the pod becomes Ready and the restart count stops rising. Delete the pod and let the controller recreate it to prove the fix survives a fresh schedule. Watch memory and CPU against the limits; a pod that runs just under the limit will crash again under a traffic spike.