VM autoscaling is a capacity-control process that adds or removes compute resources according to measured workload demand.
Do not automate scaling until the current VM is correctly sized, the bottleneck is understood, and additional nodes can become healthy and replaceable without losing sessions, files, jobs, or durable data. Use scheduled capacity for predictable peaks. Use reactive horizontal scaling only when demand lasts longer than provisioning and warm-up time and when capacity can be removed safely.
This guide explains the policy and architecture decisions behind autoscaling. It does not assume that every cloud platform provides the same managed autoscaling feature.
Autoscaling is a control loop
A complete scaling loop has six stages:
Observe demand ↓ Evaluate policy and safety limits ↓ Change capacity ↓ Configure and register the node ↓ Wait for readiness and warm-up ↓ Verify service health
A useful policy defines:
- the demand signal;
- how long the condition must persist;
- minimum, desired, and maximum capacity;
- provisioning and warm-up time;
- readiness and health checks;
- stabilization after a capacity change;
- dependency limits;
- safe removal behavior;
- manual override and rollback.
Without these controls, the fleet can oscillate, add capacity too late, remove active work, or overload the database.
Demand shape determines the scaling model
Autoscaling is only one option.
| Demand pattern | Better starting model | Reason |
|---|---|---|
| New or poorly understood workload | Manual scaling and measurement | The baseline is not reliable yet |
| Gradual growth | Periodic right-sizing | One larger VM may remain simpler |
| Known business or campaign peaks | Scheduled scaling | Capacity can arrive before demand |
| Bursts shorter than node warm-up | Queue, cache, rate control, or fixed headroom | Reactive capacity arrives too late |
| Sustained variable web traffic | Reactive horizontal scaling | Replaceable nodes can share demand |
| Variable worker backlog | Queue-based worker scaling | Queue age represents unfinished work |
| Stateful database pressure | Tuning or vertical scaling first | More app nodes may increase DB load |
| Availability requirement | Fixed minimum redundant capacity | Failure protection should already exist |
Use Horizontal vs Vertical Scaling for the scale-up versus scale-out decision.
Autoscaling readiness has seven gates
The bottleneck is known
Identify whether the limiting factor is CPU, memory, disk, database, queue processing, network, an external service, or application behavior.
Use Cloud Server Performance Bottlenecks before automating the wrong response.
Work can be divided safely
Requests or jobs must be distributable. A stateful or tightly coupled workload may still be better served by a larger VM.
Nodes are replaceable
A node should not own durable sessions, uploads, queue state, or business data that disappears when it is removed.
Use Stateful vs Stateless Applications to decide where state should live.
Provisioning is repeatable
Every new node needs the same operating-system baseline, application version, configuration, monitoring, and network policy.
Readiness is measurable
A running process or open port is not always ready for production traffic. Readiness should prove that the application can serve its intended workload.
Dependencies have headroom
More application nodes may create more database connections, cache traffic, queue consumers, storage operations, or third-party API calls.
Capacity can be removed safely
The system must stop sending new work, allow active work to finish, and confirm that the remaining fleet can carry the load.
A workload that can scale out but cannot scale in safely is not ready for full autoscaling.
Build a capacity baseline before selecting thresholds
Measure normal and peak behavior across:
- request or transaction rate;
- concurrent requests or connections;
- p50, p95, and p99 latency;
- error and timeout rate;
- CPU and memory pressure;
- queue depth and oldest-job age;
- worker completion rate;
- database latency, connections, and locks;
- storage latency and throughput;
- application startup and warm-up time;
- node count and cost.
Include ordinary traffic, deployments, backups, batch jobs, campaigns, and known seasonal peaks.
A threshold selected from one busy hour can turn normal variation into constant scaling activity. The baseline should show the relationship between demand, user impact, and useful capacity.
Scaling signals should respond to added capacity
A strong metric changes predictably when capacity changes.
| Workload | Stronger signals | Guardrails |
|---|---|---|
| Web or API tier | Requests per healthy node, concurrency, sustained p95 latency | Error rate, database latency, healthy-node count |
| Queue workers | Oldest-job age, queue depth per worker, completion rate | Retry rate, database pressure, external limits |
| Long connections | Active connections per node, connection delay | Memory, network, draining time |
| Batch processing | Remaining work and completion deadline | CPU, storage, dependency capacity |
| CPU-bound jobs | Work queue plus sustained CPU pressure | Error rate and job duration |
CPU alone is rarely sufficient
CPU is useful when the workload is genuinely compute-bound, work distributes across nodes, and added capacity reduces both CPU pressure and user or queue impact.
It is weaker when the system waits on storage, database locks, network calls, or external rate limits.
Memory is often a diagnostic guardrail
Memory pressure can represent legitimate demand, a leak, duplicated cache state, or excessive concurrency. Diagnose the cause before using memory as the only scale-out trigger.
Errors can indicate a broken system rather than insufficient capacity
A bad deployment, expired credential, or unavailable dependency will not improve when more identical nodes are added. Use error and dependency signals to stop or escalate unsafe scaling decisions.
Minimum, desired, and maximum capacity define safety
Minimum capacity
Minimum capacity should support normal demand, maintenance, expected node failure, and the time required to add replacement capacity.
Two nodes are not automatically resilient if losing one leaves the other unable to handle peak traffic.
Desired capacity
Desired capacity is the current healthy fleet target. It may be changed manually, by schedule, or by policy.
Maximum capacity
Maximum capacity protects:
- the approved cost boundary;
- database connections and throughput;
- cache and queue limits;
- network and storage limits;
- external API quotas;
- the team’s operational ability to observe the fleet.
Maximum capacity is not only a budget setting. It prevents the application tier from overwhelming a slower dependency.
Keep a documented manual override for launches and incidents.
Warm-up determines whether reactive scaling can help
A new VM is not useful capacity immediately after creation.
Warm-up may include:
- VM boot;
- host configuration;
- application deployment or startup;
- configuration retrieval;
- database and cache connections;
- readiness checks;
- load-balancer registration;
- gradual traffic or job intake.
Measure time to useful capacity.
When a traffic burst ends before this sequence completes, use scheduled headroom, buffering, caching, or a higher minimum instead of relying on reactive scale-out.
New nodes should not be treated as full capacity before they are ready and stable.
Stabilization prevents repeated scaling
Metrics arrive with delay and normal workloads fluctuate. Use different behavior for adding and removing capacity.
Scale-out can respond faster when sustained user impact is clear. Scale-in should wait longer and confirm that demand has remained low.
Use:
- separate scale-out and scale-in thresholds;
- longer scale-in observation windows;
- a stabilization period after changes;
- limits on how much capacity can be removed at once;
- minimum time between removals;
- protection for newly added nodes.
This creates hysteresis: the condition that adds capacity is intentionally different from the condition that removes it.
Scale-out and scale-in policies should be asymmetric
Example planning logic:
Scale out when: * p95 latency is above target for a sustained period * requests per healthy node exceed the tested range * database latency and error rate remain acceptable Scale in when: * p95 latency remains healthy for a longer period * demand per healthy node stays below the removal threshold * no deployment or incident is active * the remaining fleet has sufficient headroom * one node can drain successfully
The exact thresholds must come from testing and production evidence.
A practical multi-metric rule is:
- any approved demand signal may justify adding capacity;
- all removal safety conditions should agree before reducing it.
Scale-in is the riskier operation
A safe removal sequence is:
- Stop sending new traffic or jobs to the selected node.
- Wait for routing changes to take effect.
- Allow active requests or jobs to finish.
- Release leases, locks, and queue ownership safely.
- Confirm local temporary data is disposable.
- Confirm the remaining fleet has enough capacity.
- Remove the node from routing and monitoring.
- Remove the compute resource.
- Verify latency, errors, queues, and dependencies.
Worker fleets require special care because a worker may appear quiet while holding a long-running job.
Use Load Balancing Explained for readiness and draining decisions.
Stateless nodes are easier to add and remove
Externalize or coordinate:
- user sessions;
- uploaded files;
- durable generated assets;
- background jobs;
- scheduled-task ownership;
- application configuration and secrets;
- logs and metrics;
- database state.
Sticky sessions can preserve compatibility temporarily but keep users tied to one node. Local uploads can prevent safe removal. A scheduled task running on every node may execute several times after scale-out.
The operational test is:
Can one node be removed without losing files, duplicating jobs, or destroying important state?
Load balancing controls the node lifecycle
A horizontally scaled application tier normally needs health-aware routing.
Provision ↓ Configure and deploy ↓ Pass readiness ↓ Register ↓ Serve traffic ↓ Drain ↓ Remove
Plan:
- readiness behavior;
- registration delay;
- connection draining;
- long-lived requests and WebSockets;
- health thresholds;
- behavior when all backends are unhealthy;
- spare capacity after one backend fails.
Autoscaling cannot compensate for weak health checks or insufficient failure capacity.
Dependencies define the real scale unit
| Added capacity | Dependency that may become constrained |
|---|---|
| More app VMs | Database connections and query concurrency |
| More workers | Queue throughput, writes, and external API quotas |
| More upload processors | Object storage, network, and temporary disk |
| More report workers | Database reads, memory, and output storage |
| More WebSocket nodes | Shared presence, pub/sub, and routing |
A scale unit is the group of resources and limits required to support additional work.
Before allowing the application fleet to reach maximum capacity, estimate database connections per node, requests to external systems, storage and network demand, queue throughput, and observability volume.
Scheduled scaling fits predictable events
Use a schedule for:
- campaign launches;
- billing or payroll processing;
- end-of-month reporting;
- planned migrations;
- recurring business hours;
- known customer events;
- batch-processing windows.
Schedule capacity early enough to complete warm-up before demand begins. After the event, reduce capacity through the same safe removal process instead of removing it immediately at a fixed clock time.
Review schedules periodically so yesterday’s peak plan does not become permanent idle cost.
Cost planning includes headroom and lifecycle overlap
Autoscaling cost includes:
- minimum always-on capacity;
- failure headroom;
- overlapping nodes during warm-up and draining;
- load balancing and networking;
- databases, caches, queues, and shared storage;
- monitoring and logging;
- automation maintenance;
- false or noisy scaling events.
| Model | Cost posture | Main trade-off |
|---|---|---|
| One larger VM | Predictable | One-node capacity and failure boundary |
| Fixed multi-node fleet | Constant headroom | Simpler availability and deployment behavior |
| Autoscaled fleet | Capacity follows sustained demand | Highest policy and lifecycle complexity |
For small teams, a modest fixed fleet can be operationally cheaper than unreliable automation.
Use the live Raff pricing page rather than embedding temporary plan prices.
Test the complete control loop
A production-readiness test should include:
Scale-out
Measure threshold detection, provisioning, application readiness, registration, time to useful capacity, and whether user or queue pressure improves.
Failure
Test a node that fails readiness, an unhealthy application release, dependency slowdown, maximum-capacity behavior, and automation failure.
Scale-in
Confirm draining, long-running request and worker behavior, session and file continuity, and remaining capacity.
Override
Confirm the team can freeze removal, raise the maximum temporarily, disable the policy, and return to the previous safe state.
A successful test proves that capacity arrives before service targets are missed and can later be removed without disrupting work.
A scaling contract makes the policy reviewable
Maintain one short record for each workload:
| Field | Required decision |
|---|---|
| Workload and owner | What is scaling and who owns it? |
| Scaling goal | Which user or job outcome is protected? |
| Primary signal | Which metric represents demand? |
| Scale-out condition | How large and persistent is the breach? |
| Scale-in condition | How long must the system remain healthy? |
| Minimum and maximum | What protects availability, cost, and dependencies? |
| Warm-up | How long until capacity is useful? |
| Readiness and draining | How are nodes added and removed safely? |
| Dependency guardrails | Which database, queue, storage, or API limits apply? |
| Manual override | How is the policy frozen or disabled? |
| Validation | Which metrics prove the change worked? |
Review the contract after launches, incidents, architecture changes, and major traffic shifts.
How this applies on Raff
A Raff scaling architecture can use:
- Raff Cloud Servers for application and worker nodes;
- Load Balancers for health-aware traffic distribution;
- Private Cloud Networks for backend communication;
- Object Storage for shared uploads and assets;
- Volumes for persistent block storage where appropriate;
- Data Protection for recovery planning.
A practical growth path is:
Properly sized Raff VM ↓ sustained measured constraint Separated database or workers ↓ replaceable application nodes Load balancer + fixed multi-node fleet ↓ tested lifecycle and demand policy Team-operated scaling workflow
Raff documents programmatic infrastructure interfaces for supported operations. Those interfaces can support team-operated automation, but this guide does not assume a managed native autoscaling service or identical support across every interface.
Verify current documentation for supported resources, provisioning behavior, load-balancer capabilities, limits, and safe removal workflows. Use Infrastructure Automation on Raff for automation boundaries.
VM autoscaling checklist
Readiness
- The bottleneck is measured.
- Work divides safely.
- State is externalized or coordinated.
- New nodes are configured repeatably.
Policy
- Signals represent user or queue pressure.
- Scale-out and scale-in use different conditions.
- Minimum, desired, and maximum capacity are documented.
- Dependency and cost ceilings are included.
Lifecycle
- Time to useful capacity is measured.
- Readiness is meaningful.
- Stabilization is defined.
- Traffic and jobs drain safely.
Validation
- Scale-out, failure, scale-in, and override tests are complete.
- Maximum-capacity behavior is known.
- The policy has an owner and review date.