Kubernetes — node stuck in NotReady state

Diagnose NotReady nodes by inspecting kubelet health, resource pressure conditions and network reachability.

intermediatedebuggingkubernetesnodes

Symptoms

kubectl get nodes shows a node NotReady
Pods on the node are evicted or rescheduled
The node condition mentions MemoryPressure or DiskPressure
kubelet is not updating node status

Possible causes

kubelet is down or cannot reach the API server
The node has memory or disk pressure
Container runtime is unhealthy
Network partition between node and control plane

Diagnosis steps

Read the node conditions

kubectl describe node <name> | grep -A10 Conditions

Interpretation

Conditions such as MemoryPressure, DiskPressure and Ready show whether the cause is resources or kubelet connectivity.

Check kubelet status on the node

systemctl status kubelet

Interpretation

A stopped or failing kubelet explains a node that stops reporting and eventually goes NotReady.

Inspect resource pressure

df -h ; free -m

Interpretation

Full disks or exhausted memory trigger pressure conditions that make the node unschedulable and evict pods.

Root cause

The kubelet on the node stopped reporting a healthy status, because it was down, blocked by resource pressure, or unable to reach the control plane, so the node was marked NotReady.

Fix

Restart or repair kubelet, free disk and memory space, ensure container runtime health, and restore network connectivity between the node and the API server.

Verify the fix

Confirm the node returns to Ready, and confirm pods can be scheduled and remain running on it.

Symptom walkthrough

A node shows NotReady and its pods may be evicted or rescheduled elsewhere. Node conditions and kubelet health explain why: the kubelet may be stopped, unable to reach the API server, or reporting memory or disk pressure. Because the control plane treats a NotReady node as unsuitable, workloads move away, and the fault often starts with a resource problem on the node rather than with the cluster.

Fix in practice

Restart or repair kubelet, clear resource pressure by freeing disk and memory, verify the container runtime is healthy, and restore connectivity between the node and the control plane. Fixing the node condition lets the scheduler place pods there again.

Verification

Confirm the node returns to Ready and that pods can be scheduled and stay running on it. A Ready node with stable pods confirms the underlying cause is resolved.

Diagnostic commands

Check the behavior with commands like these:

kubectl describe node <name> | grep -A10 Conditions
systemctl status kubelet
df -h ; free -m