Horizontal and vertical scaling increase capacity in different ways. Vertical scaling adds CPU, memory, or storage to one server. Horizontal scaling adds more servers or workers and distributes traffic or jobs across them. Scale up when one node can still meet the workload with acceptable downtime and simplicity matters. Scale out when one node has become the capacity or availability boundary and the application can distribute work safely.
Neither strategy fixes inefficient code, slow queries, storage latency, or an external dependency. Identify the bottleneck first.
Horizontal vs vertical scaling at a glance
| Decision factor | Vertical scaling | Horizontal scaling |
|---|---|---|
| Method | Increase resources on one node | Add more nodes |
| Common term | Scale up or down | Scale out or in |
| Application changes | Usually limited | Often requires statelessness and shared services |
| Operational complexity | Lower | Higher |
| Capacity ceiling | Limited by the largest practical node | Higher when work distributes efficiently |
| Availability | One node remains a failure boundary | Can tolerate node loss when designed correctly |
| Deployment | Simple single-node process | Requires coordinated or rolling releases |
| Best early use | MVPs, internal tools, stateful software, primary databases | Web/API fleets, workers, independently scalable services |
A useful rule is:
Scale vertically to remove a resource constraint. Scale horizontally to remove one node as the capacity or availability boundary.
Use a five-part scaling decision
Before changing the architecture, answer five questions:
- What is limiting the workload? CPU, memory, disk, network, database, queue, or dependency?
- Can one larger node meet the expected demand? Include growth and failure headroom.
- Can the work be divided safely? Requests, jobs, and data must tolerate distribution.
- Where does state live? Sessions, files, queues, and durable data cannot depend on one disposable app node.
- Can the team operate multiple nodes? Health checks, deployments, logs, networking, and recovery must be repeatable.
Use Cloud Server Performance Bottlenecks before selecting a scaling method.
Vertical scaling is the simplest first move
Vertical scaling increases resources on the current VM or moves the workload to a larger profile.
It is often the right choice when:
- CPU or memory is the measured constraint
- one server still fits the application model
- the workload is difficult to partition
- the team wants the lowest operating complexity
- a maintenance window or restart is acceptable
- the larger node provides enough growth headroom
Common candidates include:
- early SaaS applications
- internal business systems
- legacy or stateful applications
- primary databases
- single-node APIs
- workloads with modest and predictable growth
A larger VM can create valuable engineering time. The team can stabilize deployment, monitoring, backups, and recovery before introducing distributed-system responsibilities.
Vertical scaling has hard and practical limits
Scaling up eventually reaches a ceiling:
- the largest available VM size
- poor cost efficiency at very large sizes
- maintenance or migration risk
- one-node downtime and failure exposure
- continued contention between application, database, workers, and monitoring
- software limits that do not improve with more cores
A resize is not a high-availability design. Backups and fast replacement reduce recovery time, but the service still depends on one active node.
Before resizing, verify the platform procedure, expected interruption, storage behavior, rollback path, and whether the application can use the additional resources.
Horizontal scaling requires distributable work
Horizontal scaling adds application instances, workers, or service replicas.
Users ↓ Load balancer ↓ App VM 1 App VM 2 App VM 3 ↓ private network Database, cache, queue, and object storage
More nodes increase useful capacity only when work can be distributed without conflicting state, duplicate side effects, or a downstream bottleneck.
Horizontal scaling usually requires:
- health-aware traffic or job distribution
- repeatable provisioning and configuration
- externalized sessions and durable files
- centralized logs and metrics
- private networking and firewall rules
- database connection planning
- graceful shutdown and connection draining
- safe scale-in behavior
Microsoft’s Azure Architecture Center recommends designing horizontally scaled services without instance affinity and avoiding state that forces repeated requests to one specific node.
Stateless application nodes are easier to scale out
An application node is operationally stateless when another healthy node can handle the next request without depending on files or session data stored only on the first server.
Move shared state to the right layer:
- durable records → database
- sessions → shared session store or suitable database
- uploads and assets → object storage
- background work → queue
- cached shared data → distributed cache
Read Stateful vs Stateless Applications before adding application instances.
Sticky sessions can reduce short-term migration work, but they preserve node affinity and make failure, balancing, and deployment behavior less predictable.
Capacity and availability are different scale-out goals
Horizontal scaling can solve two distinct problems.
Capacity
Several nodes process more requests or jobs in parallel.
This works when:
- the work is parallelizable
- traffic distribution is effective
- the database and dependencies have headroom
- synchronization does not consume the expected gain
Scalability should be measured as throughput gained per resource added. Adding twice the application capacity does not guarantee twice the throughput when a database, queue, lock, or shared service becomes the new bottleneck.
Availability
Traffic continues when one node is unhealthy.
This requires:
- at least two healthy nodes
- meaningful readiness checks
- automatic removal of unhealthy nodes
- enough remaining capacity after one node fails
- state outside the failed node
- resilient database, storage, DNS, and routing dependencies
Two nodes that each require 70% of total traffic capacity cannot safely absorb one-node failure. Design for failure capacity, not only normal traffic.
Use scale units to protect dependencies
A scale unit is the set of resources that must grow together to support additional demand.
For example, adding application nodes may also require:
- more database connections
- more worker capacity
- greater queue throughput
- more cache memory
- more load-balancer capacity
- more storage or outbound network capacity
Scale the complete path rather than one visible component. Microsoft’s Well-Architected guidance notes that scaling one resource can move pressure to stateful dependencies, especially databases.
A practical test is:
When one app node is added, which other limits move closer to exhaustion?
Workers and queues are natural scale-out candidates
Background workers often distribute more easily than stateful web applications.
Job queue ↓ Worker 1 Worker 2 Worker 3
Before adding workers, confirm:
- jobs can run concurrently
- duplicate execution is safe or prevented
- retries are bounded
- work is idempotent where practical
- database writes can handle added concurrency
- external API rate limits are respected
- queue depth and oldest-job age are monitored
More workers can overload the database or external provider. Scale consumers only when the entire processing path can support them.
Databases usually scale differently
The application tier and database tier rarely follow the same scaling sequence.
A practical order for a primary database is:
- measure slow queries, locks, storage, and connection usage
- fix inefficient queries and indexes
- introduce connection pooling
- increase RAM, CPU, or storage performance
- separate analytics, backups, or heavy jobs
- add read replicas when read demand justifies them
- partition or shard only when simpler options are insufficient
Vertical scaling is often effective for a primary database because more memory and faster storage can improve many queries without changing consistency architecture.
Read replicas distribute selected reads but do not automatically increase write capacity. Sharding can increase write capacity, but it adds data placement, routing, migration, and operational complexity.
Load balancing needs readiness, not only reachability
A load balancer distributes traffic, but it needs a meaningful health signal.
A readiness check should determine whether the node can safely receive new work. Depending on the application, it may verify:
- startup completion
- required configuration
- critical dependency access
- a representative lightweight operation
- migration or maintenance state
Avoid checks that are so shallow they mark a broken process healthy, or so deep they overload dependencies.
Use Load Balancers when multiple application nodes need health-aware traffic distribution.
Deployment changes after scaling out
A single-node deployment may restart one service. Multi-node deployment requires version coexistence and traffic control.
A rolling release usually follows this sequence:
- Drain one node from traffic.
- Deploy the new version.
- Wait for readiness.
- Return the node to service.
- Continue with the next node.
During the rollout, old and new versions may operate simultaneously. Database schemas, message formats, sessions, and cache structures must remain compatible.
Read Blue-Green vs Rolling Deployments for the release decision.
Autoscaling is a control system
Autoscaling is not only a rule that creates another VM. It requires:
- instrumentation
- decision logic
- provisioning and configuration
- traffic registration
- warm-up time
- cooldown periods
- safe scale-in
- graceful shutdown
- state and connection handling
Useful signals may include:
- request latency and error rate
- queue age or depth
- concurrency
- CPU or memory pressure
- scheduled demand
Scale on signals that represent workload demand and user impact. A short CPU spike may not justify adding capacity, while a growing job queue may require action even with moderate CPU.
Test scaling delay. Capacity that arrives after users are already timing out is not an effective response.
Scale-in is often riskier than scale-out
Adding capacity is usually safer than removing it.
Before terminating a node, confirm:
- it no longer receives new traffic
- existing requests and connections can finish
- queued or in-progress jobs are handed off safely
- local temporary data is disposable
- leases and locks expire correctly
- the remaining fleet has enough capacity
Aggressive scale-in can create oscillation, interrupted jobs, and repeated cold starts. Use cooldowns and conservative removal rules.
Test both performance and failure
A scaling test should represent real behavior:
- request mix
- authentication
- cache hit rate
- database queries
- uploads
- background jobs
- connection duration
- external dependencies
Measure the complete system while load increases. Then test failure:
- remove one application node
- stop a worker
- slow a dependency
- deploy a bad version to one node
- verify remaining capacity
Scaling succeeds when throughput and failure behavior improve predictably—not merely when more instances exist.
Choose the scaling strategy from evidence
| Evidence | Better decision |
|---|---|
| One measured resource is constrained | Scale vertically |
| The app cannot distribute requests safely | Scale vertically while removing state dependencies |
| One node still meets growth and recovery requirements | Keep the simpler design |
| Stateless app tier reaches repeatable capacity | Scale horizontally |
| One-node failure is unacceptable | Add redundant nodes and resilient dependencies |
| Workers can consume jobs independently | Scale workers horizontally |
| Database is the bottleneck | Tune or scale the database before adding app nodes |
| Traffic is short and bursty | Add buffering, caching, or scheduled headroom before complex autoscaling |
| Different components grow differently | Separate and scale them independently |
The best design may combine both methods: larger database nodes with horizontally scaled application and worker tiers.
How this applies on Raff
A practical Raff growth path may use:
- Raff Cloud Servers for application, worker, and database workloads
- Load Balancers for health-aware traffic distribution
- Private Cloud Networks for backend communication
- Object Storage for shared uploads and assets
- Volumes for persistent block storage where appropriate
- Data Protection for recovery planning
One Raff VM ↓ measured constraint Larger VM or separated database/worker ↓ throughput or availability requirement Load Balancer + multiple stateless app VMs ↓ private network Database, cache, queue, and storage
Verify current plan sizes, resize behavior, load-balancer capabilities, and service limits on live product pages before implementation.
Horizontal vs vertical scaling checklist
Scale up readiness
- The bottleneck is CPU, memory, or another node-level limit.
- One larger node can meet expected demand.
- Downtime or maintenance risk is acceptable.
- The application can use the additional resources.
Scale out readiness
- Requests or jobs can be distributed safely.
- Sessions, files, and durable state are externalized.
- Health checks and traffic draining are tested.
- Deployments and configuration are repeatable.
- Dependencies have enough capacity.
- Remaining nodes can handle one-node failure.
Autoscaling readiness
- Demand signals represent real workload pressure.
- Provisioning and warm-up time are known.
- Cooldowns prevent oscillation.
- Scale-in drains work safely.
- Minimum, maximum, and failure capacity are defined.