Spread Replicas Across Zones with Topology Constraints

Use topologySpreadConstraints with DoNotSchedule so one zone failure cannot take every replica, then verify the spread on real pods.

~30 minkubernetesschedulingavailability

Task

Keep replicas balanced across availability zones instead of landing on whichever node the scheduler prefers.

When to use this

Three replicas schedule onto one node because of resource requests. A node reboot takes the whole service down even though the cluster spans three zones.

Prerequisites

Nodes labeled with topology.kubernetes.io/zone
At least three nodes across zones
A Deployment with three or more replicas

Steps

1. Add a hard zone constraint and a soft node preference.

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule
    labelSelector:
      matchLabels:
        app: payments
  - maxSkew: 1
    topologyKey: kubernetes.io/hostname
    whenUnsatisfiable: ScheduleAnyway
    labelSelector:
      matchLabels:
        app: payments

Expected output: Pods are spread across zones, and a second pod on the same node is allowed but deprioritized.

2. Confirm every schedulable node reports a zone before rolling out.

kubectl get nodes -L topology.kubernetes.io/zone

Expected output: Each node shows a zone value; an empty zone breaks the spread calculation for that node.

3. Roll out and check where the pods actually landed.

kubectl get pods -n prod -l app=payments -o wide && kubectl get nodes -L topology.kubernetes.io/zone

Expected output: Pods are on different nodes, and those nodes are in different zones.

4. Scale down and back up to prove the constraint still balances the fleet.

kubectl scale deployment/payments --replicas=1 -n prod && kubectl scale deployment/payments --replicas=3 -n prod

Expected output: Scaling back refills the empty zones rather than stacking pods on the first node.

Troubleshooting

Problem

Pods stay Pending with DoNotSchedule

Solution

The cluster lacks zones or capacity to satisfy the skew. Set minDomains, add nodes, or use ScheduleAnyway when availability matters more than strict balance.

Problem

Spread ignores existing pods

Solution

The label selector only matches pods with the same labels. Use a stable app label rather than version-specific labels.

Limitations

Hard constraints can leave pods Pending when a zone is full or missing.
Spread controls placement, not capacity; each zone still needs enough CPU and memory.

Replicas are not availability

Three replicas on one node have the availability of one node. The scheduler places pods by resource fit and simple scoring, so an empty node with enough CPU can receive every replica of a new deployment. Topology spread constraints add a distribution rule: pods sharing a label are balanced across a topology domain such as a zone or a host. A hard constraint (DoNotSchedule) refuses to place a pod that would violate the skew; a soft constraint (ScheduleAnyway) prefers balance but still schedules when the cluster cannot satisfy it.

Choose hard and soft deliberately

Use a hard zone constraint for the property you actually need, and a soft host constraint for the nice-to-have. That combination keeps zones balanced even when a zone has fewer nodes, and still spreads pods within a zone when capacity allows. Keep maxSkew at one for small replica counts; larger values tolerate imbalance that may be exactly what you are trying to prevent. Set minDomains when the cluster can legitimately have fewer zones than expected, and remember that a Pending pod is a visible failure while an unbalanced fleet is a silent one.

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule
    labelSelector:
      matchLabels:
        app: payments

Verify the actual placement

Scheduling decisions are only visible after the fact. List pods with their nodes, then list nodes with their zone label; the spread is real only when both lineups agree. Roll the deployment and check again, because the constraint applies to new pods and a rolling update is when placement changes. If pods remain Pending, check zone capacity before relaxing the constraint, since a hard constraint on a two-zone cluster with three replicas will always leave one pod waiting.