Scale & autoscale workloads

Once your app is running (see Deploy your first app), the next question is capacity: how do you handle more traffic without over-provisioning? Kubernetes scales workloads in two ways, manually and automatically.

Scale by hand

The simplest setting is the replica count:

kubectl -n demo scale deployment/web --replicas=4
kubectl -n demo get pods

Kubernetes starts new Pods and the Service load-balances across all of them immediately. Scaling back down is the same command with a smaller number.

Spread replicas across nodes

More replicas only help availability if they do not all land on the same node. Topology spread constraints keep them distributed:

spec:
  template:
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: ScheduleAnyway
          labelSelector:
            matchLabels: { app: web }

Now a single node going away takes out at most a fraction of your Pods, and the Deployment reschedules them elsewhere.

Autoscale with an HPA

A HorizontalPodAutoscaler adjusts the replica count for you based on observed load. This one targets 60% average CPU, between 2 and 10 replicas:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web
  namespace: demo
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60

kubectl apply -f hpa.yaml
kubectl -n demo get hpa web -w

The HPA needs the metrics-server to read Pod CPU and memory. It is included on every Gardener cluster, whether on your own landscape or one I run for you, so the autoscaler works without further setup. HPA scaling relies on the resource requests you set on the container, which is another reason to always declare them.

Pods scale in seconds; nodes scale in minutes. When the HPA wants more Pods than your nodes can fit, the cluster autoscaler adds a node from your node pool. On a cluster with adopted servers, the pooled servers are used first, so you fill the servers you already pay for before paying for new ones.

Generate some load to see it work

kubectl -n demo run load --rm -it --image=busybox --restart=Never -- \
  /bin/sh -c "while true; do wget -q -O- http://web.demo; done"

Watch kubectl -n demo get hpa web -w and you will see the target replica count rise with CPU, then fall back after you stop the load (Ctrl-C).

Next steps

• Set sensible requests and limits so the autoscaler and scheduler have accurate signals to work with.
• Add a PodDisruptionBudget so voluntary disruptions (upgrades, drains) never take too many replicas down at once.