Cloud autoscaling is the process of increasing or decreasing compute capacity according to measured workload demand. For virtual machines, that usually means scaling out by adding healthy VM instances when demand rises and scaling in by safely removing capacity when demand falls.
Autoscaling should not be the first response to an undersized or poorly understood workload. First right-size the baseline VM, identify the actual bottleneck, and confirm that additional nodes can become healthy and replaceable without losing sessions, files, jobs, or durable data.
Raff Technologies supports cloud VM architectures that can grow from one right-sized server to multiple application or worker nodes. The planning principles in this guide apply whether scaling actions are manual, scheduled, or implemented through team-operated automation. Do not assume every cloud platform exposes the same native managed autoscaling feature.
Use Horizontal vs Vertical Scaling if you are still deciding between scale up and scale out. Use Right-Sizing Cloud Servers if the current VM itself is over- or under-provisioned.
What is cloud autoscaling?
Cloud autoscaling is a control loop that observes workload demand, evaluates a scaling policy, changes capacity, and verifies that the resulting infrastructure is healthy.
A complete VM autoscaling loop looks like this:
Observe demand ↓ Evaluate autoscaling policy ↓ Add or remove capacity ↓ Configure and register nodes ↓ Wait for readiness and warm-up ↓ Verify workload health
The important point is that creating or deleting a VM is only one step. A production autoscaling system also needs readiness checks, capacity limits, warm-up, dependency guardrails, safe draining, failure handling, and a manual override.
VM autoscaling is different from simply resizing a server
Vertical resizing changes the resources of one VM. Autoscaling normally manages the number of replaceable instances serving a workload.
| Decision | What changes | Typical use |
|---|---|---|
| Right-size | CPU, RAM, storage, or VM class | Correct an existing VM profile |
| Scale up | Resources on one node | Remove a node-level CPU or memory constraint |
| Scale out | Number of nodes | Add parallel application or worker capacity |
| Scale in | Number of nodes decreases | Remove excess capacity safely |
| Autoscaling | Capacity changes according to policy | Repeated variable demand |
A workload that cannot safely scale out and scale in is not ready for full autoscaling.
Scale out vs scale in: both directions need different safeguards
Scale out adds capacity. Scale in removes it. The two operations should not use identical rules.
Scale-out can usually respond faster when sustained user or queue pressure is clear. Scale-in should be more conservative because removing the wrong node can terminate active requests, interrupt jobs, reduce failure headroom, or cause repeated capacity oscillation.
| Scale-out decision | Scale-in decision |
|---|---|
| Is demand above the tested capacity range? | Has demand stayed low for long enough? |
| Can a new node become useful before the peak passes? | Can one node drain without losing active work? |
| Are dependencies still healthy? | Will remaining nodes retain enough headroom? |
| Is maximum capacity still safe? | Is a deployment, incident, or batch job active? |
| Does added capacity improve the workload outcome? | Does removal preserve latency and reliability targets? |
A practical policy often lets one approved demand signal justify adding capacity, while requiring all removal-safety conditions to agree before scaling in.
An autoscaling policy needs more than a CPU threshold
An autoscaling policy defines when capacity changes and the boundaries that prevent unsafe behavior.
Document at least:
- the workload and owner;
- the scaling objective;
- the primary demand signal;
- how long the signal must persist;
- minimum, desired, and maximum capacity;
- provisioning and warm-up time;
- readiness and health checks;
- scale-out step size;
- scale-in observation window;
- stabilization or cooldown behavior;
- dependency limits;
- drain and shutdown behavior;
- manual override and rollback.
Avoid universal rules such as “add a VM at 70% CPU.” The correct threshold depends on how CPU pressure relates to latency, throughput, queue age, job duration, and the time required for new capacity to become useful.
Cloud capacity planning comes before reactive autoscaling
Cloud capacity planning establishes the safe operating range that the autoscaling policy will manage.
Measure representative periods for:
- request or transaction rate;
- concurrent requests or connections;
- p50, p95, and p99 latency;
- CPU pressure;
- available memory, swap, and OOM events;
- queue depth and oldest-job age;
- worker completion rate;
- database latency and connections;
- storage latency and throughput;
- application startup time;
- node warm-up time;
- healthy node count;
- cost at minimum, normal, and maximum capacity.
Include ordinary traffic, launches, deployments, backups, batch work, recurring business peaks, and known seasonal demand.
The baseline should answer two questions: how much useful work one healthy node can handle, and how long it takes additional capacity to become useful.
Choose scaling signals that improve when capacity is added
A good autoscaling signal represents demand or unfinished work and responds predictably to added capacity.
| Workload | Better primary signals | Useful guardrails |
|---|---|---|
| Web/API tier | Requests per healthy node, concurrency, sustained p95 latency | Error rate, DB latency, healthy-node count |
| Queue workers | Oldest-job age, queue depth per worker, completion rate | Retry rate, DB pressure, external API limits |
| Long-lived connections | Active connections per node, connection delay | Memory, network, drain time |
| Batch processing | Remaining work vs deadline | CPU, storage, dependency health |
| CPU-bound jobs | Work queue plus sustained CPU pressure | Job duration, errors, storage waits |
CPU can be a useful signal, but rarely the only one
CPU works best when the workload is genuinely compute-bound and adding another node reduces both CPU pressure and the user-visible or queue-visible problem.
CPU is weaker when the system is waiting on:
- database locks or slow queries;
- storage latency;
- memory pressure;
- external APIs;
- network dependencies;
- serial application code.
Use Cloud Server Bottlenecks before automating the wrong response.
Memory is usually a guardrail before it becomes a scaling trigger
Memory pressure can represent real demand, a leak, duplicated cache state, or excessive concurrency. Adding nodes may help a stateless workload, but it can also replicate memory-heavy state on every node.
Diagnose the working set before using memory as the only scale-out metric. See How Much RAM Do I Need for a VPS?.
Minimum, desired, and maximum capacity define the safe range
An autoscaling policy should have explicit bounds.
Minimum capacity
Minimum capacity must cover normal demand plus the failure and warm-up headroom the workload requires.
Two nodes are not automatically highly available if losing one leaves the other unable to carry normal peak traffic.
Desired capacity
Desired capacity is the current healthy fleet target. It can be changed manually, by schedule, or by an automated policy.
Maximum capacity
Maximum capacity protects more than budget. It can also protect:
- database connection limits;
- database write capacity;
- queue and cache throughput;
- storage limits;
- external API quotas;
- observability systems;
- the team's ability to operate the fleet safely.
A runaway application tier should not be allowed to overwhelm a slower dependency simply because more VMs can be created.
Warm-up determines whether reactive autoscaling can work
A newly created VM is not useful capacity immediately.
Time to useful capacity may include:
- VM provisioning and boot;
- operating-system configuration;
- application deployment or startup;
- configuration and secret retrieval;
- database and cache connections;
- application warm-up;
- readiness checks;
- traffic or job registration;
- gradual intake of production work.
Measure this duration in real conditions.
If a traffic burst usually ends before a node can complete this sequence, reactive autoscaling is too slow for that demand shape. Use scheduled capacity, buffering, caching, rate control, or a larger minimum instead.
Scheduled scaling fits predictable demand
Scheduled scaling works well when the demand window is known in advance.
Examples include:
- campaign launches;
- recurring business hours;
- end-of-month reporting;
- billing or payroll jobs;
- planned migrations;
- scheduled batch processing;
- known customer events.
Add capacity early enough for provisioning and warm-up to finish before demand begins. Do not remove it immediately at the expected end time; use the same drain and safety checks required by normal scale-in.
Scheduled capacity is often simpler and safer than reactive automation for predictable workloads.
Reactive autoscaling fits sustained variable demand
Reactive scaling is more defensible when demand is unpredictable but persists long enough for new capacity to become useful.
Typical candidates include:
- stateless application tiers;
- APIs with variable sustained traffic;
- queue workers processing independent jobs;
- event processors with measurable backlog;
- services where one healthy node has a known capacity range.
The workload should already support multiple replaceable instances. Sessions, files, queues, and durable state must not depend on one application VM.
Use Stateful vs Stateless Applications before adding dynamic app capacity.
Dependencies define the real scaling limit
Adding application or worker nodes increases pressure elsewhere.
| Added capacity | Dependency that may become constrained |
|---|---|
| More app VMs | Database connections and query concurrency |
| More workers | Queue throughput, writes, and API quotas |
| More upload processors | Object storage, network, temporary disk |
| More report workers | Database reads, memory, output storage |
| More connection nodes | Shared state, pub/sub, routing |
Before allowing a policy to reach maximum capacity, estimate how the full fleet affects the database, storage, network, queues, APIs, and monitoring systems.
This is why a maximum node count is partly a dependency-protection control.
Safe scale-in requires draining and workload handoff
Scale-in is the lifecycle step most likely to damage active work if it is rushed.
A safe removal sequence is:
- Stop sending new traffic or jobs to the selected node.
- Wait for routing changes to take effect.
- Allow active requests or jobs to finish.
- Release leases, locks, and queue ownership safely.
- Confirm local temporary data is disposable.
- Confirm the remaining fleet has enough capacity.
- Remove the node from routing and monitoring.
- Remove the VM.
- Verify latency, errors, queues, and dependencies.
Long-running workers need special handling because a node can look lightly loaded while still owning a job that cannot be interrupted safely.
Stabilization prevents autoscaling oscillation
Metrics arrive with delay, workloads fluctuate, and a capacity change needs time to affect the system.
Use:
- different scale-out and scale-in thresholds;
- longer scale-in observation windows;
- a stabilization period after changes;
- limits on how many nodes can be removed at once;
- protection for newly created instances;
- minimum time between scale-in events.
This creates hysteresis: the condition that adds capacity is intentionally different from the condition that removes it.
Without that separation, a fleet can repeatedly add and remove instances around one noisy threshold.
Test the complete autoscaling control loop
A production test should cover more than scale-out.
Test scale-out
Measure signal detection, provisioning, configuration, readiness, time to useful capacity, and whether workload pressure improves.
Test failure behavior
Test a VM that never becomes ready, an unhealthy release, dependency slowdown, maximum-capacity behavior, and automation failure.
Test scale-in
Verify draining, active requests, long-running jobs, state continuity, and remaining capacity.
Test manual override
Confirm operators can freeze scale-in, increase a maximum temporarily, disable automation, and return to the last known-safe state.
A successful test proves that capacity can be added before service targets are missed and later removed without interrupting useful work.
Cloud autoscaling has a cost beyond VM hours
Autoscaling cost includes:
- minimum always-on capacity;
- failure headroom;
- overlapping nodes during warm-up and draining;
- databases, queues, caches, and shared storage;
- monitoring and logging;
- automation maintenance;
- false or noisy scaling events.
| Capacity model | Cost posture | Main trade-off |
|---|---|---|
| One larger VM | Predictable | Single-node capacity and failure boundary |
| Fixed multi-node fleet | Constant headroom | Simpler redundancy and operations |
| Autoscaled fleet | Capacity follows sustained demand | Highest policy and lifecycle complexity |
For a small team, a modest fixed fleet can be cheaper operationally than fragile automation even if the raw VM-hours are slightly higher.
How cloud autoscaling applies on Raff
Raff can provide the infrastructure building blocks around a team-operated scaling design without assuming a native managed VM autoscaling service.
Relevant current components include:
- Raff Cloud Servers for application and worker VMs;
- VPC for private service-to-service networking;
- Object Storage for shared uploads and durable assets;
- Volumes for persistent block storage where appropriate;
- Data Protection for recovery planning.
Raff VMs currently use a 3 Gbps public connection with unmetered VM traffic and no VM egress fee. Hourly pay-as-you-go VM usage is available alongside monthly, yearly, and 24-month billing terms, which can be useful when capacity changes over time.
A practical growth path is:
Right-sized VM ↓ sustained measured constraint Separated database or workers ↓ replaceable application nodes Fixed multi-node fleet ↓ tested readiness, drain, and policy logic Team-operated autoscaling workflow
Verify the current API, CLI, Terraform, traffic-distribution, and provisioning capabilities before building automation around a specific resource lifecycle. Use Infrastructure Automation on Raff for the automation boundary.