Tune a HorizontalPodAutoscaler for Stable Scaling

Set behavior policies, a sensible utilization target, and a floor that survives bursty traffic instead of flapping between replica counts.

~30 minkubernetesautoscalingcapacity

Task

Scale on real load with predictable behavior, avoiding rapid oscillation and surprise scale-to-zero events.

When to use this

The autoscaler scales from 3 to 12 pods and back every few minutes because the default window reacts to a two-minute spike; cold starts make latency worse each cycle.

Prerequisites

metrics-server installed
A Deployment with CPU requests set on every container
kubectl access to the autoscaling/v2 API

Steps

1. Define the autoscaler with utilization, replica bounds, and scale-down behavior.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: payments
  namespace: prod
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: payments
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 25
          periodSeconds: 60

Expected output: Scale-down waits five minutes and removes at most a quarter of the pods per minute.

2. Confirm the autoscaler sees metrics and a stable desired replica count.

kubectl get hpa payments -n prod && kubectl top pods -n prod -l app=payments

Expected output: The HPA reports a current utilization percentage and a desired replica count that matches the deployment.

3. Verify CPU requests exist, because utilization is meaningless without them.

kubectl get deployment payments -n prod -o jsonpath='{.spec.template.spec.containers[*].resources.requests.cpu}'

Expected output: A CPU request value is printed; an empty output explains an unknown metrics warning.

4. Load test and watch that replicas rise and fall without oscillating.

kubectl get hpa payments -n prod -w

Expected output: Replicas climb during the test and decrease slowly after it ends, with no repeated spikes.

Troubleshooting

Problem

The HPA reports unknown current utilization

Solution

metrics-server is missing or containers lack CPU requests. Install metrics-server and set requests before tuning thresholds.

Problem

Adding replicas does not reduce latency

Solution

The bottleneck is elsewhere, or pods start too slowly. Raise minReplicas or improve startup before trusting CPU-based scaling.

Limitations

CPU utilization is only a proxy for load; queue depth or request rate is better for I/O-bound services.
The autoscaler cannot scale faster than pods become ready, so slow startups still cause latency spikes.

Scaling is a control loop, not a switch

The autoscaler compares observed utilization with a target and adjusts replicas every fifteen seconds by default. That loop is intentionally simple, which means it will follow noise if nothing constrains it. A two-minute traffic spike raises utilization, the HPA adds pods, the spike ends, utilization collapses, and the HPA removes them; the next spike repeats the cycle. Each cycle pays for cold starts and scheduler churn while users see latency that the extra pods never had time to fix.

Slow the loop down

behavior.scaleDown.stabilizationWindowSeconds keeps the highest recommendation from the recent window, so a short dip does not immediately remove capacity. A 300 second window combined with a percentage policy gives a gradual decline that is easy to observe and cancel. On the way up, a short window is usually fine, but a policy that limits surge prevents a resource-constrained cluster from failing to schedule a large batch of new pods at once.

behavior:
  scaleDown:
    stabilizationWindowSeconds: 300
    policies:
      - type: Percent
        value: 25
        periodSeconds: 60

Pick a signal that leads

CPU is convenient but often a lagging signal: a service waiting on a database shows low CPU while latency climbs. If the workload is I/O-bound, scale on request rate or queue depth with a custom or external metric. Set minReplicas from the traffic the service must absorb with zero warm-up, not from the average, and verify after a real load test that the fleet settles instead of oscillating. Finally, alert when the HPA sits at maxReplicas, because that is a capacity incident, not a scaling success.