Serverless Cold Starts, Concurrency, and Scale-to-Zero

Key Takeaways

Cold starts occur when a serverless platform must prepare a new execution environment before code handles work. Scale-to-zero reduces idle capacity but can make first-request latency more variable. Concurrency and burst traffic can create new instances even while other instances are already warm. Raff Functions offers scale-to-zero, load-based autoscaling, sub-second cold starts, optional warm instances, and a default 0→10 scaling range.

Serdar Tekin
Serdar Tekin
Co-Founder & CTO
Updated 14 min read
Written for: Developers and platform teams tuning serverless latency, concurrency, autoscaling, and scale-to-zero behavior
Serverless FunctionsCold StartsConcurrencyAutoscalingScale-to-ZeroPerformance
Last reviewed:
Guide
Serverless Cold Starts, Concurrency, and Scale-to-Zero
rafftechnologies.com/learn

Scale to zero between requests and pay only while functions run.

Deploy code without a server, billed per second of use, with a free allowance every month.

A cold start in serverless computing is the extra startup work required when a new execution environment must be prepared before a function can handle work.

For Raff Technologies Functions, the same workload-fit principle applies: decide whether scale-to-zero or ready capacity better matches the endpoint's latency and usage pattern.

That startup cost is part of the same model that makes serverless attractive. When idle functions can scale to zero, teams avoid keeping capacity running only in case a request arrives. The trade-off is that the next request—or a burst that needs additional execution environments—may experience more startup latency than work handled by an already warm instance.

For small teams, this is not a reason to avoid serverless. It is a reason to decide which workloads can tolerate variable startup latency and which need ready capacity. Concurrency matters because cold starts are not limited to the first request after a long idle period: a sudden increase in simultaneous work can require the platform to create more instances even while existing instances are warm.

At Raff, we treat cold-start tuning as a workload-shape decision rather than a contest to eliminate every millisecond. A nightly cron task and a user-facing authentication endpoint should not receive the same latency policy.

Serverless cold starts are a scale-to-zero trade-off

Scale-to-zero means the platform can reduce an idle function to no active execution instances. That is valuable for webhooks, scheduled jobs, file events, internal tools, and other workloads that may sit unused for long periods.

The basic trade-off is simple:

Text
Scale to zero ↓ Less idle capacity ↓ Possible startup work on next invocation ↓ More variable first-request latency

Keeping capacity warm changes the other side of the trade-off:

Text
Warm capacity ↓ Ready execution environment ↓ More consistent startup path ↓ Some capacity remains allocated while idle

Neither model is universally better. The useful question is whether a workload benefits more from avoiding idle capacity or from reducing latency variability.

The existing Serverless Function Pricing guide owns the detailed cost model. Here, pricing matters only as a design constraint: warm capacity changes the economic advantage of scale-to-zero, so it should be reserved for workloads whose latency requirements justify it.

Scale-to-zero saves idle capacity by accepting the possibility of startup latency later.

Cold and warm execution follow different latency paths

A cold invocation has more work to do before application code can serve the request. Depending on the platform and runtime, that can include preparing an execution environment, loading the runtime and application code, importing dependencies, and running initialization logic.

A warm invocation can reuse an already prepared environment. That can remove much of the startup path, but it does not guarantee a fast application response. Database queries, external APIs, DNS, object storage, network calls, locks, or slow business logic can still dominate request latency.

This distinction matters when debugging serverless performance.

Latency sourceCold invocationWarm invocation
Environment preparationPossibleUsually avoided
Runtime/application initializationPossibleOften reused
Dependency loadingPossibleOften reused
Database/API call latencyYesYes
Application computationYesYes
Downstream rate limitingYesYes

Do not label every slow first request a cold-start problem without evidence. Measure the complete request path and separate initialization from the work the function performs after it starts.

The same principle applies in the other direction: a low average latency does not prove cold starts are irrelevant. If only a small percentage of invocations are cold, averages can hide the user-visible tail. For interactive functions, p95 or p99 behavior can be more informative than a single mean value.

Concurrency can create cold starts during traffic bursts

Concurrency describes how much work is executing at the same time. It is different from request rate.

A service can receive 100 requests per second with low concurrency if every request finishes almost instantly. A slower handler can have high concurrency at a lower request rate because many invocations remain in flight simultaneously.

For serverless platforms, that difference matters because one warm environment cannot necessarily absorb unlimited simultaneous work. As concurrency grows, the platform may need additional execution environments.

A simplified burst looks like this:

Text
1 warm instance ↓ traffic burst arrives ↓ more simultaneous work than current capacity can serve ↓ platform adds instances ↓ some new instances follow a cold-start path

This is why “we keep getting traffic, so the function must already be warm” is not always a safe assumption. A workload can have warm instances and still create cold starts when demand expands beyond current capacity.

Concurrency controls also protect downstream systems. A function that scales aggressively may create more simultaneous database connections, API calls, object-storage requests, or queue operations than the dependency can safely handle.

The production decision therefore has two sides:

  • enough concurrency to absorb legitimate bursts;
  • enough control to avoid overwhelming the systems behind the function.

Hard runtime and concurrency limits belong in the dedicated runtime-limits guide rather than here. The architectural point is that autoscaling capacity and downstream capacity must be planned together.

Scale-to-zero fits asynchronous workloads especially well

Cold-start sensitivity varies by workload.

A nightly cleanup job usually does not care whether initialization adds a short delay before processing begins. A storage event that generates a thumbnail can normally tolerate startup variation if the user sees an asynchronous processing state. A public API called inside a user-facing checkout path may have much less tolerance.

Use this framework:

WorkloadDefault latency postureWhy
Cron/maintenance taskScale to zeroStart time usually matters less than completion
Object-storage processingScale to zeroWork is asynchronous and event-driven
Webhook receiverUsually scale to zeroMost can tolerate small startup variation if acknowledgement remains acceptable
Internal admin toolScale to zeroLow traffic often makes idle capacity wasteful
User-facing APIMeasure firstTail latency may affect experience
Authentication or synchronous payment pathFavor predictable readiness if latency is strictStartup variation can sit directly in the user journey
Constant high trafficCompare serverless with always-on computeLittle idle time reduces the value of scaling to zero

This framework also prevents overengineering. Teams sometimes keep every function warm because one endpoint is latency-sensitive. That gives low-importance background jobs the same cost posture as a critical synchronous API.

Set latency policy per function or workload class instead.

For the broader deployment-model question, Serverless Functions vs VMs covers when sustained or environment-heavy workloads fit a VM more naturally.

Warm capacity should be reserved for latency-sensitive paths

Keeping an instance warm is a mitigation, not a universal default.

Warm capacity can reduce the chance that an incoming request needs a newly prepared environment, but it changes the resource model because some capacity remains ready even when there is no work.

Use warm capacity when three conditions are true:

  1. the path is latency-sensitive;
  2. measured cold-start behavior is material to that latency target;
  3. the value of more consistent readiness justifies the idle baseline.

Do not use it to hide unrelated latency. If a function spends most of its time waiting on a slow database query, a warm instance will not fix the query. If a third-party API takes two seconds, removing function initialization does not remove that dependency latency.

A practical operating pattern is:

Text
Background/event functions → scale to zero Latency-sensitive APIs → measure cold vs warm behavior Critical endpoints → add warm capacity only if needed Sustained workloads → compare against VM/container capacity

This keeps the latency decision tied to actual user impact.

Initialization design often matters more than adding capacity

Cold-start work includes application initialization, so code structure matters.

Common contributors include:

  • large dependency graphs;
  • expensive imports or module initialization;
  • loading configuration that is not needed for every invocation;
  • opening external connections during startup;
  • initializing SDK clients repeatedly inside the request path;
  • oversized custom images or runtime packages;
  • doing business work during global initialization.

The safest optimization is not “make everything lazy.” Some clients and configuration are useful to initialize once and reuse while the environment remains warm. The goal is to keep startup work intentional.

A useful split is:

Text
Initialization → configuration → reusable clients → lightweight validation Per invocation → request-specific validation → business logic → database/API/storage work

Do not depend on a warm environment for correctness. A platform may replace an instance, scale down, or create a fresh one. Reused connections and cached objects should be treated as performance opportunities, not durable state.

Durable records belong in a database, and durable files belong in object storage. This keeps a function safe whether the current invocation is cold, warm, retried, or moved to another instance.

Raff exposes the latency-cost trade-off directly

Raff Functions scales to zero and autoscales with load, with sub-second cold starts, optional warm instances, and a default scaling range of 0→10 instances.

Those are Raff's current published product characteristics, not an independent benchmark performed for this guide. Real application latency still depends on runtime choice, dependencies, initialization, function code, external services, and traffic shape.

Raff warm instances bill at 30% of the memory rate. That creates a direct decision between allowing a function to scale to zero and paying for some ready memory capacity on latency-sensitive paths. For the exact memory, active-CPU, free-tier, request, and egress model, use Serverless Function Pricing or the Raff pricing page.

The practical Raff model is:

RequirementStarting point
Rare webhook, cron, or file eventScale to zero
Bursty endpoint with tolerant latencyScale to zero + observe tail latency
Customer-facing endpoint with strict latencyMeasure and consider warm capacity
Sustained always-busy serviceCompare Functions with Raff VM or another always-on runtime

Raff also includes per-function logs and metrics with invocation count, p50, p95, and error rate. That is useful because the decision should be based on observed endpoint behavior, not a generic assumption that every cold start is unacceptable.

Compare the Raff Functions pricing model before choosing scale-to-zero or warm capacity for latency-sensitive workloads.

Production teams should measure traffic shape and tail latency

Cold-start tuning needs production-like traffic rather than one local timing.

Measure at least four things:

  • traffic shape: steady, bursty, scheduled, or mostly idle;
  • concurrency: how many invocations are in flight at once;
  • latency distribution: p50 plus tail measurements such as p95 or p99 where available;
  • downstream pressure: database connections, API rate limits, queue depth, and storage dependencies.

A useful review asks:

QuestionWhy it matters
Does latency spike mainly after idle periods?Cold-start sensitivity may be real
Does latency spike during bursts?New-instance scaling or downstream contention may be involved
Are warm requests also slow?The problem is probably not only cold start
Is the endpoint user-facing?Tail latency may directly affect experience
Is the function idle most of the day?Scale-to-zero may have strong economic value
Is it busy continuously?Always-on capacity may be simpler
Does scale-out overload the database/API?Concurrency needs a downstream guardrail

Do not optimize from one number. A function with occasional cold starts can still be the right architecture if the workload is asynchronous. A function with very fast startup can still be the wrong architecture if sustained traffic makes always-on capacity simpler.

Choose the latency policy by workload shape

Cold starts are not a serverless defect to eliminate at any cost. They are one side of the scale-to-zero model.

Use scale-to-zero for work that can tolerate startup variation and benefits from avoiding idle capacity. Watch concurrency during bursts because new instances can follow a cold-start path even when some instances are already warm. Keep initialization light and make correctness independent of instance reuse.

For user-facing endpoints with strict latency requirements, measure tail behavior and use warm capacity only when startup latency is a material part of the problem. If traffic becomes constant enough that there is little benefit in scaling down, compare the workload with an always-on VM or container model instead.

The goal is not zero cold starts. It is a latency and capacity policy that matches the workload your team is actually running.

Sources

Was this article helpful?
Serdar Tekin
Serdar Tekin
Co-Founder & CTO

Co-Founder & CTO at Raff, focused on cloud infrastructure, systems architecture, security, reliability, and production operations.

Scale to zero between requests and pay only while functions run.

Deploy code without a server, billed per second of use, with a free allowance every month.

14-day money-back guarantee

Frequently asked questions

What is a cold start in serverless computing?
A serverless cold start is the extra startup work required when a new execution environment must be prepared before a function can handle an invocation.
Does a cold start only happen after a function has been idle?
No. Idle scale-to-zero is one cause, but traffic bursts can also require additional execution environments, so new instances may cold start while other instances are already warm.
What is the difference between a cold start and a warm start?
A cold start includes environment and application initialization before work begins. A warm start reuses an already prepared environment, although database, API, network, and application latency can still make the request slow.
How does concurrency affect serverless cold starts?
Higher simultaneous demand can require the platform to create more execution environments. That scale-out can introduce cold starts and can also increase pressure on downstream databases or APIs.
Should every production serverless function use warm instances?
No. Warm capacity is most useful for latency-sensitive endpoints where measured startup latency materially affects the user experience. Cron, storage events, and many asynchronous workloads usually benefit from scale-to-zero.
Is scale-to-zero good for production workloads?
Yes when the workload can tolerate startup variation and has meaningful idle periods. For sustained or latency-critical workloads, measure tail latency and compare warm serverless capacity with an always-on runtime.
How does Raff handle serverless cold starts?
Raff Functions scales to zero, autoscales with load, starts in under a second from cold, offers optional warm instances, and uses 0→10 as the default scaling range. Application latency still depends on code, initialization, dependencies, and traffic shape.