A cold start in serverless computing is the extra startup work required when a new execution environment must be prepared before a function can handle work.
That startup cost is part of the same model that makes serverless attractive. When idle functions can scale to zero, teams avoid keeping capacity running only in case a request arrives. The trade-off is that the next request—or a burst that needs additional execution environments—may experience more startup latency than work handled by an already warm instance.
For small teams, this is not a reason to avoid serverless. It is a reason to decide which workloads can tolerate variable startup latency and which need ready capacity. Concurrency matters because cold starts are not limited to the first request after a long idle period: a sudden increase in simultaneous work can require the platform to create more instances even while existing instances are warm.
At Raff, we treat cold-start tuning as a workload-shape decision rather than a contest to eliminate every millisecond. A nightly cron task and a user-facing authentication endpoint should not receive the same latency policy.
Serverless cold starts are a scale-to-zero trade-off
Scale-to-zero means the platform can reduce an idle function to no active execution instances. That is valuable for webhooks, scheduled jobs, file events, internal tools, and other workloads that may sit unused for long periods.
The basic trade-off is simple:
Scale to zero ↓ Less idle capacity ↓ Possible startup work on next invocation ↓ More variable first-request latency
Keeping capacity warm changes the other side of the trade-off:
Warm capacity ↓ Ready execution environment ↓ More consistent startup path ↓ Some capacity remains allocated while idle
Neither model is universally better. The useful question is whether a workload benefits more from avoiding idle capacity or from reducing latency variability.
The existing Serverless Function Pricing guide owns the detailed cost model. Here, pricing matters only as a design constraint: warm capacity changes the economic advantage of scale-to-zero, so it should be reserved for workloads whose latency requirements justify it.
Scale-to-zero saves idle capacity by accepting the possibility of startup latency later.
Cold and warm execution follow different latency paths
A cold invocation has more work to do before application code can serve the request. Depending on the platform and runtime, that can include preparing an execution environment, loading the runtime and application code, importing dependencies, and running initialization logic.
A warm invocation can reuse an already prepared environment. That can remove much of the startup path, but it does not guarantee a fast application response. Database queries, external APIs, DNS, object storage, network calls, locks, or slow business logic can still dominate request latency.
This distinction matters when debugging serverless performance.
| Latency source | Cold invocation | Warm invocation |
|---|---|---|
| Environment preparation | Possible | Usually avoided |
| Runtime/application initialization | Possible | Often reused |
| Dependency loading | Possible | Often reused |
| Database/API call latency | Yes | Yes |
| Application computation | Yes | Yes |
| Downstream rate limiting | Yes | Yes |
Do not label every slow first request a cold-start problem without evidence. Measure the complete request path and separate initialization from the work the function performs after it starts.
The same principle applies in the other direction: a low average latency does not prove cold starts are irrelevant. If only a small percentage of invocations are cold, averages can hide the user-visible tail. For interactive functions, p95 or p99 behavior can be more informative than a single mean value.
Concurrency can create cold starts during traffic bursts
Concurrency describes how much work is executing at the same time. It is different from request rate.
A service can receive 100 requests per second with low concurrency if every request finishes almost instantly. A slower handler can have high concurrency at a lower request rate because many invocations remain in flight simultaneously.
For serverless platforms, that difference matters because one warm environment cannot necessarily absorb unlimited simultaneous work. As concurrency grows, the platform may need additional execution environments.
A simplified burst looks like this:
1 warm instance ↓ traffic burst arrives ↓ more simultaneous work than current capacity can serve ↓ platform adds instances ↓ some new instances follow a cold-start path
This is why “we keep getting traffic, so the function must already be warm” is not always a safe assumption. A workload can have warm instances and still create cold starts when demand expands beyond current capacity.
Concurrency controls also protect downstream systems. A function that scales aggressively may create more simultaneous database connections, API calls, object-storage requests, or queue operations than the dependency can safely handle.
The production decision therefore has two sides:
- enough concurrency to absorb legitimate bursts;
- enough control to avoid overwhelming the systems behind the function.
Hard runtime and concurrency limits belong in the dedicated runtime-limits guide rather than here. The architectural point is that autoscaling capacity and downstream capacity must be planned together.
Scale-to-zero fits asynchronous workloads especially well
Cold-start sensitivity varies by workload.
A nightly cleanup job usually does not care whether initialization adds a short delay before processing begins. A storage event that generates a thumbnail can normally tolerate startup variation if the user sees an asynchronous processing state. A public API called inside a user-facing checkout path may have much less tolerance.
Use this framework:
| Workload | Default latency posture | Why |
|---|---|---|
| Cron/maintenance task | Scale to zero | Start time usually matters less than completion |
| Object-storage processing | Scale to zero | Work is asynchronous and event-driven |
| Webhook receiver | Usually scale to zero | Most can tolerate small startup variation if acknowledgement remains acceptable |
| Internal admin tool | Scale to zero | Low traffic often makes idle capacity wasteful |
| User-facing API | Measure first | Tail latency may affect experience |
| Authentication or synchronous payment path | Favor predictable readiness if latency is strict | Startup variation can sit directly in the user journey |
| Constant high traffic | Compare serverless with always-on compute | Little idle time reduces the value of scaling to zero |
This framework also prevents overengineering. Teams sometimes keep every function warm because one endpoint is latency-sensitive. That gives low-importance background jobs the same cost posture as a critical synchronous API.
Set latency policy per function or workload class instead.
For the broader deployment-model question, Serverless Functions vs VMs covers when sustained or environment-heavy workloads fit a VM more naturally.
Warm capacity should be reserved for latency-sensitive paths
Keeping an instance warm is a mitigation, not a universal default.
Warm capacity can reduce the chance that an incoming request needs a newly prepared environment, but it changes the resource model because some capacity remains ready even when there is no work.
Use warm capacity when three conditions are true:
- the path is latency-sensitive;
- measured cold-start behavior is material to that latency target;
- the value of more consistent readiness justifies the idle baseline.
Do not use it to hide unrelated latency. If a function spends most of its time waiting on a slow database query, a warm instance will not fix the query. If a third-party API takes two seconds, removing function initialization does not remove that dependency latency.
A practical operating pattern is:
Background/event functions → scale to zero Latency-sensitive APIs → measure cold vs warm behavior Critical endpoints → add warm capacity only if needed Sustained workloads → compare against VM/container capacity
This keeps the latency decision tied to actual user impact.
Initialization design often matters more than adding capacity
Cold-start work includes application initialization, so code structure matters.
Common contributors include:
- large dependency graphs;
- expensive imports or module initialization;
- loading configuration that is not needed for every invocation;
- opening external connections during startup;
- initializing SDK clients repeatedly inside the request path;
- oversized custom images or runtime packages;
- doing business work during global initialization.
The safest optimization is not “make everything lazy.” Some clients and configuration are useful to initialize once and reuse while the environment remains warm. The goal is to keep startup work intentional.
A useful split is:
Initialization → configuration → reusable clients → lightweight validation Per invocation → request-specific validation → business logic → database/API/storage work
Do not depend on a warm environment for correctness. A platform may replace an instance, scale down, or create a fresh one. Reused connections and cached objects should be treated as performance opportunities, not durable state.
Durable records belong in a database, and durable files belong in object storage. This keeps a function safe whether the current invocation is cold, warm, retried, or moved to another instance.
Raff exposes the latency-cost trade-off directly
Raff Functions currently publishes scale-to-zero and load-based autoscaling as part of the product model. The current product page also advertises sub-second cold starts, optional warm instances, and a default scaling range shown as 0→10 instances.
Those are Raff's current published product characteristics, not an independent benchmark performed for this guide. Real application latency still depends on runtime choice, dependencies, initialization, function code, external services, and traffic shape.
Raff's current warm-instance pricing is published at 30% of the memory rate. That creates a direct decision between allowing a function to scale to zero and paying for some ready memory capacity on latency-sensitive paths. For the exact memory, active-CPU, free-tier, request, and egress model, use Serverless Function Pricing or the current Raff pricing page.
The practical Raff model is:
| Requirement | Starting point |
|---|---|
| Rare webhook, cron, or file event | Scale to zero |
| Bursty endpoint with tolerant latency | Scale to zero + observe tail latency |
| Customer-facing endpoint with strict latency | Measure and consider warm capacity |
| Sustained always-busy service | Compare Functions with Raff VM or another always-on runtime |
Raff also includes per-function logs and metrics with invocation count, p50, p95, and error rate. That is useful because the decision should be based on observed endpoint behavior, not a generic assumption that every cold start is unacceptable.