Scheduling and autoscaling

Scaling is several feedback loops, not one magic switch.

Pod placement

The scheduler compares requests with available capacity. Node selectors and affinity attract Pods. Taints repel Pods unless they tolerate them. Topology spread distributes replicas across failure domains. Hard rules can protect isolation but also make Pods impossible to schedule.

Pod scaling

The Horizontal Pod Autoscaler changes replica count from CPU, memory, or custom metrics. Choose a metric that follows demand. Set minimums for availability and maximums from downstream capacity. The Vertical Pod Autoscaler helps right-size requests but may restart Pods in automatic modes.

Node scaling

Cluster Autoscaler changes managed node-group size. Karpenter selects and launches EC2 instances from Pod requirements. Auto Mode manages a Karpenter-based capacity layer through NodePools and NodeClasses. Pod scaling and node scaling are separate: new Pods can remain Pending while capacity arrives.

Capacity choices

Use On-Demand for reliable baseline capacity. Use Spot for replicated, checkpointed, queue-driven, or retryable work. Diversify instance types and zones. Graviton can improve price-performance when images support arm64.

Feynman check

Requests are the shopping list the scheduler reads. If the list is wrong, no autoscaler can buy the right shelves.