Kubernetes Autoscaling: HPA, Node Scaling, and Cost

Key Takeaways

Treat HPA and node scaling as one control loop. Use realistic requests, explicit min/max bounds, stable scale-down behavior, and a safe minimum worker fleet.

Serdar Tekin
Serdar Tekin
Co-Founder & CTO
Updated 17 min read
Written for: Developers and platform teams tuning Kubernetes HPA, node autoscaling, capacity, and worker cost boundaries.
KubernetesAutoscalingHPAHorizontal Pod AutoscalerNode AutoscalingCost
Last reviewed:
Guide
Kubernetes Autoscaling: HPA, Node Scaling, and Cost
rafftechnologies.com/learn
Free control plane

Scale node pools as your pods grow, workers from $9.99 a month.

CNCF-certified Kubernetes with a free control plane. Add nodes when you need them, with no egress fees.

Kubernetes autoscaling adjusts workload replicas or worker-node capacity as demand changes. The two layers solve different problems:

HPA changes the number of Pods. Node autoscaling changes the amount of worker capacity available to schedule those Pods.

For small teams, the safe operating model is:

measure → set realistic requests → scale Pods → observe Pending Pods → scale workers → consolidate carefully

Raff Technologies managed Kubernetes supports worker-pool autoscaling between the minimum and maximum values configured per pool. The application team still owns HPA signals, replica bounds, workload requests, readiness behavior, and the cost ceiling.

This guide owns Kubernetes autoscaling strategy: Horizontal Pod Autoscaler, node autoscaling, scale-to-zero, scheduling interaction, scale-down stability, and cost boundaries.

What is Kubernetes autoscaling?

Kubernetes autoscaling changes resources automatically in response to workload demand.

The three common layers are:

Autoscaling layerWhat changesTypical signal
Horizontal Pod Autoscaler (HPA)Number of Pod replicasCPU, memory, custom, object, or external metrics
Vertical Pod Autoscaler (VPA)Pod resource requests/recommendationsHistorical and current resource use
Node autoscalingNumber of worker nodesUnschedulable Pods and removable excess capacity

For most small teams, the key production combination is HPA + node autoscaling.

HPA answers:

How many replicas should this workload run?

Node autoscaling answers:

Does the cluster have enough compatible worker capacity to place them?

These decisions are connected, but they are not interchangeable.

How HPA works

The HorizontalPodAutoscaler periodically reads metrics and adjusts the desired replica count of a scalable workload such as a Deployment or StatefulSet.

With CPU utilization, the important relationship is approximately:

Text
current utilization = measured CPU usage / requested CPU

That means resource requests directly affect the signal.

Example:

Text
Measured CPU = 200m CPU request = 100m Utilization ≈ 200%

But:

Text
Measured CPU = 200m CPU request = 500m Utilization ≈ 40%

The application is doing the same amount of CPU work. Only the request changed.

This is why Kubernetes Requests vs Limits must be correct before aggressive HPA tuning.

Bad requests create bad autoscaling signals.

HPA replica calculation is a ratio problem

At a high level, HPA calculates desired replicas from the relationship between current and target metrics.

A simplified form is:

Text
desired replicas ≈ current replicas × current metric / target metric

Example:

Text
current replicas = 4 current CPU utilization = 80% target CPU utilization = 50% desired replicas ≈ 4 × 80 / 50 ≈ 6.4

The controller then applies Kubernetes autoscaling behavior, tolerance, missing-data handling, stabilization, and configured min/max bounds.

Do not use this simplified formula as a billing prediction. It is useful for understanding why the controller wants more or fewer replicas.

Requests must exist for resource-utilization HPA

For CPU or memory utilization targets, HPA needs meaningful resource requests.

If a container does not have the relevant request, Kubernetes may be unable to calculate utilization for that Pod correctly.

Before enabling CPU-based HPA, verify:

Bash
kubectl get deployment <name> -o yaml kubectl get hpa kubectl describe hpa <name>

Check:

  • requests are present;
  • metrics are available;
  • current and target values make sense;
  • minReplicas and maxReplicas are intentional;
  • HPA conditions do not show metric errors.

If HPA does not scale, inspect the metric path before increasing replica limits.

Use the metric that represents demand

CPU is convenient, but it is not always the right signal.

Different workloads may scale better from:

  • CPU utilization;
  • memory;
  • request rate;
  • queue depth;
  • concurrent jobs;
  • custom application metrics;
  • external metrics.

Examples:

WorkloadUseful signal to evaluate
CPU-bound APICPU
queue workerqueue depth
request-driven web tierrequest rate or CPU
background processorbacklog / external metric
memory-bound servicememory, with caution
event consumerlag or queue depth

A metric should represent work that more replicas can actually help process.

If a database bottleneck causes API latency, adding more API Pods may make the database problem worse.

Autoscale the layer that owns the constraint.

HPA scale-up and scale-down should not behave identically

Scale-up often needs to react faster than scale-down.

Removing replicas too quickly can create oscillation:

Text
demand falls → replicas removed → traffic rises slightly → replicas recreated → demand falls → replicas removed again

Kubernetes HPA includes behavior controls and a default downscale stabilization window. In current Kubernetes, the default downscale stabilization window is five minutes unless configured otherwise.

The autoscaling/v2 API allows teams to configure scale-up and scale-down behavior, including rate policies and stabilization.

For a small team, a good starting principle is:

scale up fast enough to protect service quality; scale down slowly enough to prove the capacity is really excess.

Tune from production behavior rather than copying one universal policy.

Kubernetes 1.37 can scale suitable HPA workloads to zero

Kubernetes 1.37 added Beta support for HPA scale-to-zero and enables it by default.

This is useful for workloads such as:

  • queue consumers;
  • event-driven workers;
  • batch-style processors;
  • workloads whose demand is represented by an object or external metric.

A suitable HPA can reduce a workload to zero replicas and later scale it back when the metric changes.

This can remove the last idle Pod for workloads that do not need permanent warm capacity.

But there is an important trade-off:

scale-to-zero introduces cold-start latency.

Kubernetes Services do not buffer requests while no Pods are ready. An ordinary HTTP application generally needs another component to queue or buffer demand if zero replicas are allowed.

Use scale-to-zero where work can wait safely, not simply because zero idle cost sounds attractive.

HPA tolerance reduces small scale oscillations

Kubernetes 1.37 also stabilizes configurable HPA tolerance.

Tolerance defines a band around the target metric where small variations do not trigger scaling.

Conceptually:

Text
target CPU = 60% small movement around 60% → no scale action meaningful movement beyond tolerance → autoscaler considers a replica change

This helps prevent tiny metric fluctuations from constantly changing replica counts.

The correct tolerance depends on workload variability and how costly replica churn is.

Node autoscaling starts with unschedulable Pods

Node autoscaling operates below HPA.

A common scale-up sequence is:

Text
traffic rises → HPA requests more replicas → scheduler tries to place new Pods → existing workers cannot fit them → Pods become Pending → node autoscaler evaluates compatible capacity → new worker joins → Pending Pods schedule

The important word is compatible.

A Pending Pod can be unschedulable because of:

  • insufficient CPU;
  • insufficient memory;
  • node affinity;
  • nodeSelector;
  • taints and tolerations;
  • topology rules;
  • storage constraints;
  • node-pool maximums;
  • incompatible worker shapes;
  • unavailable provider capacity.

Adding any random node does not necessarily solve the problem.

Use:

Bash
kubectl describe pod <pending-pod>

Read the scheduler events before assuming a Pending Pod means "add more nodes."

HPA and node autoscaling form one feedback loop

Consider:

  • 3 normal replicas;
  • HPA min = 3;
  • HPA max = 12;
  • one worker pool with autoscaling;
  • enough existing headroom for 5 replicas.

Traffic increases.

HPA may request 6 replicas.

Five fit immediately.

The sixth stays Pending.

Node autoscaling then decides whether compatible capacity should be added.

This distinction matters for cost:

more Pods do not automatically mean more nodes.

If HPA growth fits inside existing headroom, paid worker capacity may not change.

Node cost increases when new worker capacity is actually required.

The reverse path is:

Text
demand falls → HPA reduces replicas → scheduler has fewer Pods to place → nodes become underused → node autoscaler evaluates consolidation → removable workers leave

Scale-down is deliberately more conservative because workloads may need to move and disruption rules must be respected.

Node-pool architecture constrains autoscaling

Node autoscaling can only add useful capacity if the new node is eligible for the Pending workload.

For example:

Memory workload → required node affinity: pool=memory → memory pool at max nodes → general pool has spare capacity

The workload can still remain Pending.

Total cluster capacity is irrelevant if scheduling rules prevent the Pod from using it.

This is why Kubernetes Node Pools and autoscaling should be designed together.

For every pool, define:

  • worker shape;
  • minimum nodes;
  • maximum nodes;
  • workload eligibility;
  • dedicated taints if required;
  • expected failure reserve;
  • scaling reason.

Too many pools can make autoscaling less efficient by fragmenting capacity.

The safe minimum fleet comes before autoscaling

Autoscaling does not create capacity instantly.

Scale-up requires:

  1. unschedulable demand to exist;
  2. the autoscaler to evaluate it;
  3. infrastructure provisioning;
  4. the node to join;
  5. system Pods to start;
  6. the node to become Ready;
  7. application Pods to schedule.

Your minimum worker count should therefore support the workload conditions that must survive immediately.

That may include:

  • baseline traffic;
  • one-worker failure;
  • normal deployment surge;
  • maintenance drain;
  • always-on replicas.

Autoscaling should handle additional demand above the safe baseline.

Use Kubernetes Cluster Sizing to calculate that baseline.

Cost control comes from explicit bounds

Autoscaling does not inherently reduce cost.

It automates capacity inside the rules you provide.

Important boundaries include:

ControlWhat it limits
HPA minReplicasMinimum running application capacity
HPA maxReplicasMaximum workload expansion
Worker minimumPaid infrastructure floor
Worker maximumMaximum node expansion
RequestsScheduler reservation and utilization math
Node shapeCPU/memory added per scale event
Scale-down behaviorHow quickly capacity is removed

Every minimum and maximum should have a reason.

Examples:

HPA minimum = 3

Reason: application must preserve three warm replicas.

Worker minimum = 2

Reason: baseline workloads and maintenance requirements need two workers.

Worker maximum = 8

Reason: expected peak demand fits inside eight nodes and this creates an explicit infrastructure-cost ceiling.

If nobody can explain why a maximum is 50 rather than 10, the autoscaling policy is incomplete.

Cost increases often begin with inaccurate requests

Suppose a workload uses 300m CPU per Pod but requests 1 CPU.

The scheduler reserves capacity as if each Pod needs 1 CPU.

HPA adds more replicas.

The existing workers fill earlier.

Pods become Pending earlier.

Node autoscaling adds another worker earlier.

The autoscaler is behaving correctly.

The bad input is the request.

The reverse problem also matters. Requests that are far too low can pack too many Pods onto workers and create contention.

Use production observations to compare:

  • actual usage;
  • requests;
  • limits;
  • replicas;
  • worker utilization;
  • Pending Pod events;
  • node scale events.

Rightsizing and autoscaling should be tuned together.

How Raff Kubernetes worker autoscaling works

Raff Kubernetes supports worker-pool autoscaling between the minimum and maximum values configured per pool.

The public product behavior is:

  • scale up in response to Pending Pods that need capacity;
  • scale down after sustained low load;
  • set min and max independently per worker pool;
  • scale manually when desired.

The current platform also supports multiple node pools, each with its own plan and node count.

Current shared-CPU workers include:

WorkervCPURAMMonthly price
Starter12 GB$9.99
Standard24 GB$17.99
Performance48 GB$33.99
High Memory816 GB$71.99
Large832 GB$119.99
Scale1664 GB$229.99

Raff also offers CPU-Optimized Kubernetes workers for latency-sensitive workloads.

The standard control plane is currently $0/month, while three-master HA is $30/month.

For current worker pricing and product behavior, use the Raff Kubernetes product page.

The practical relationship is:

Text
HPA / workload replicas ↓ resource requests ↓ scheduler fit ↓ Pending Pods ↓ worker-pool autoscaling ↓ paid worker capacity

That makes requests, replica limits, and worker-pool maximums direct cost controls.

Autoscaling troubleshooting matrix

When autoscaling does not behave as expected, identify the layer first.

SymptomCheck first
CPU rises but HPA stays unchangedHPA metrics, requests, conditions, maxReplicas
HPA adds replicas but Pods stay Pendingscheduler events
Pending Pod says insufficient memoryworker capacity / requests
Pending Pod says affinity mismatchnode-pool placement
Pool is at maximummax-node boundary
Nodes scale but Pods still cannot runtaints, affinity, storage, topology
Workers do not scale downminimum nodes, utilization, disruption, rescheduling feasibility
Replicas oscillateHPA tolerance, stabilization, startup behavior
Cost rises without traffic growthrequests, replica floor, node minimums, fragmentation

Do not debug HPA, scheduling, and infrastructure scaling as one black box.

Autoscaling checklist

Before enabling production autoscaling:

HPA

  • CPU/memory requests are meaningful.
  • The selected metric represents useful demand.
  • minReplicas protects required warm capacity.
  • maxReplicas is intentional.
  • readiness/startup behavior is correct.
  • scale-down behavior is conservative enough.

Worker autoscaling

  • The safe minimum fleet is defined.
  • Each pool has explicit min/max node counts.
  • Worker shape matches the workload.
  • Scheduling constraints are understood.
  • Pool maximums fit expected peak demand.
  • Failure capacity is not dependent on instant scale-up.

Cost

  • Requests are reviewed against actual usage.
  • Node-pool fragmentation is limited.
  • Maximum worker count creates a known cost ceiling.
  • Scale events can be correlated with workload demand.

Final recommendation

Treat Kubernetes autoscaling as one coordinated control loop.

Use HPA to change replica demand. Use node autoscaling to add compatible worker capacity when those replicas cannot be scheduled. Keep realistic resource requests between the two layers, define explicit minimums and maximums, and make scale-down more conservative than scale-up.

For workloads that can tolerate cold starts and use suitable external or object metrics, Kubernetes 1.37 HPA scale-to-zero can eliminate the last idle replica. For ordinary request-driven services, preserve warm capacity unless another layer safely buffers demand.

Continue with Kubernetes Requests vs Limits, Kubernetes Cluster Sizing, Kubernetes Node Pools, and Kubernetes Cost Optimization to tune the inputs and boundaries around autoscaling.

Run Raff Kubernetes with worker-pool autoscaling bounded by the minimum and maximum capacity your workloads actually need.

Free control plane · 14-day money-back guarantee

Deploy a Cluster Now
Was this article helpful?
Serdar Tekin
Serdar Tekin
Co-Founder & CTO

Co-Founder & CTO at Raff, focused on cloud infrastructure, systems architecture, security, reliability, and production operations.

Free control plane

Scale node pools as your pods grow, workers from $9.99 a month.

CNCF-certified Kubernetes with a free control plane. Add nodes when you need them, with no egress fees.

14-day money-back guarantee

Frequently asked questions

What is Kubernetes autoscaling?
Kubernetes autoscaling changes workload replicas or worker-node capacity automatically as demand changes. HPA scales Pods, while node autoscaling changes worker capacity.
What is HPA in Kubernetes?
HorizontalPodAutoscaler adjusts the replica count of scalable workloads from metrics such as CPU, memory, custom, object, or external metrics.
What is the difference between HPA and node autoscaling?
HPA changes application replica demand. Node autoscaling adds or removes worker capacity when Pods cannot be scheduled or capacity can be consolidated.
Do resource requests affect HPA?
Yes. For CPU or memory utilization targets, HPA compares measured usage with the configured request. Missing or inaccurate requests can make autoscaling signals misleading.
When does node autoscaling add capacity?
A common trigger is Pending Pods that cannot fit on existing eligible nodes. The new capacity must also satisfy affinity, taints, topology, storage, and pool constraints.
Can Kubernetes HPA scale to zero?
Kubernetes 1.37 added Beta scale-to-zero support for suitable HPA workloads using object or external metrics. It is best for workloads that can tolerate cold starts or buffered demand.
How does autoscaling work on Raff Kubernetes?
Raff worker pools can scale between configured minimum and maximum node counts. Current shared-CPU workers start at $9.99/month, with Standard 2 vCPU / 4 GB workers at $17.99/month.