Scale on a Custom Metric with HPA
Configure a Horizontal Pod Autoscaler on a custom queue depth metric so the deployment scales with backlog not just CPU.
Task
Configure kubectl autoscale deploy/worker --min=2 --max=20 --cpu-percent=80 so the cluster enforces the policy at admission time instead of relying on manual review.
When to use this
A developer deploys a pod without kubectl autoscale deploy/worker --min=2 --max=20 --cpu-percent=80 and it disrupts a node, causing evictions on a production cluster.
Prerequisites
- A Kubernetes cluster with kubectl access
- Cluster admin or namespace edit RBAC
Steps
1. Configure kubectl autoscale deploy/worker --min=2 --max=20 --cpu-percent=80 in the project configuration file with the recommended defaults.
kubectl autoscale deploy/worker --min=2 --max=20 --cpu-percent=80
Expected output: The configuration is accepted and kubectl autoscale deploy/worker --min=2 --max=20 --cpu-percent=80 runs without errors on the first invocation.
2. Run the check locally to verify kubectl autoscale deploy/worker --min=2 --max=20 --cpu-percent=80 produces the expected result before pushing.
pnpm test
Expected output: Local run matches CI behavior; no surprises after push.
3. Add the gate to the CI pipeline so kubectl autoscale deploy/worker --min=2 --max=20 --cpu-percent=80 blocks merge on failure.
- name: Run add hpa custom metric scaling
run: pnpm test
Expected output: Pull requests with failing kubectl autoscale deploy/worker --min=2 --max=20 --cpu-percent=80 are blocked from merging.
Troubleshooting
Problem
The check passes locally but fails in CI
Solution
Pin dependency versions and the runtime to match CI exactly; reproduce with the same Docker image or act.
Problem
Configuration is rejected with a schema error
Solution
Check the tool version matches the config schema; older versions may not support newer fields.
Limitations
- The gate only runs on events that trigger the workflow; locally pushed commits bypass it.
- Configuration drift is possible if someone edits settings outside version control.
Enforce at admission
Start by measuring the problem. Configure a Horizontal Pod Autoscaler on a custom queue depth metric so the deployment scales with backlog not just CPU. The most common mistake is adding the gate without first confirming it catches the issue you care about. Run kubectl autoscale deploy/worker –min=2 –max=20 –cpu-percent=80 against the current codebase and inspect the output before wiring it into CI.
kubectl autoscale deploy/worker --min=2 --max=20 --cpu-percent=80Verify the policy
Once the gate is in place, make failures actionable. A failing kubectl autoscale deploy/worker –min=2 –max=20 –cpu-percent=80 should point to the exact file and line, not a generic error. Pin the tool version in the lockfile so a upstream release does not silently change behavior. Review the gate output in every pull request just like a test result.