Horizontal Pod Autoscaling

replicas: 3 in a Deployment spec is a static number a person chose, presumably based on some expected load. Real traffic rarely stays constant — a spike during a product launch needs more replicas than the number set weeks earlier, and quiet overnight hours are paying for capacity nobody's using. Manually editing replicas in response to traffic is possible but doesn't scale as an operational practice, and reacts far slower than an automated system watching load in real time.

A HorizontalPodAutoscaler (HPA) adjusts a Deployment's replica count automatically, based on observed metrics — most commonly CPU or memory utilization:

This targets 70% average CPU utilization across all Pods in the Deployment — if actual usage rises above that, the HPA controller increases replicas, and the Deployment controller (from lesson 1) does the actual work of creating new Pods to reach that new target, the same reconciliation loop already covered, just with the desired replica count now set by the HPA instead of a person.

HPA calculates utilization as a percentage of each Pod's resources.requests.cpu (from Docker & Containers' Pod lesson) — a Pod with no CPU request set has nothing for the HPA to calculate a percentage against, and autoscaling based on CPU simply won't function correctly. This is a genuine, common gotcha: teams enable an HPA, see it never scaling as expected, and the actual cause is a missing or inaccurate resource request on the underlying Pod spec, not anything wrong with the HPA configuration itself.

CPU and memory are convenient defaults, but they're not always what actually indicates load — a queue-processing service might scale better based on queue depth (from the Cloud Computing domain's queues content) than CPU, since a slow, CPU-light task can still mean a growing backlog that needs more workers. HPA supports scaling on custom metrics for exactly this reason, at the cost of needing a metrics pipeline (like Prometheus) actually exposing that custom value to the cluster.

Scaling up quickly matters for handling a spike; scaling down too aggressively risks removing Pods that are still finishing in-flight requests. HPA includes stabilization windows specifically to avoid "flapping" — rapidly scaling up and down in response to noisy, short-lived metric fluctuations — the same underlying concern as the failure-detector flapping problem covered in the Distributed Systems Fundamentals domain, here applied to scaling decisions instead of failure detection.

HPA changes how many Pods exist. A separate object, the VerticalPodAutoscaler, instead adjusts how much CPU/memory each individual Pod requests — useful when a workload's actual resource needs were simply set wrong at first, rather than when the number of Pods needed varies with load. The two solve different problems and, with care, can be combined, though using both on the same metric for the same workload needs deliberate coordination to avoid them fighting each other.