The question this answers
What am I actually giving up when I move a workload onto a function platform?
The checkout API must answer in under 300 ms at p99, run at a steady 400 requests per second all day, and hold a database connection while it works. Every one of those three clauses argues against a function.
A clear-eyed list of the constraints the model imposes, so the decision is made before the workload is on the platform rather than during the incident that reveals one.
The cold start is initialization you moved onto the request path
A cold start is not mysterious latency. It is the sum of work that a long-lived process does once at boot — start the runtime, load the deployment package, run module-level code, construct clients, read configuration, establish connections — happening while a user waits. Every scale-out event pays it again, which means cold starts cluster precisely when traffic is rising and you can least afford them.
The size of it is dominated by things you control. A deployment package with a large dependency tree loads slower. A framework that scans for routes at import time costs more than one that does not. Eagerly opening a database connection in module scope moves a TCP and TLS handshake into the cold path. Placing the function inside a virtual network historically added network-interface setup on top. None of that is inherent to serverless; it is initialization that a persistent process amortizes and a function does not.
The mitigations all have a price. Provisioned or pre-warmed concurrency keeps environments alive — and reintroduces exactly the fixed, always-on cost the model was supposed to remove. Trimming dependencies costs engineering time. Lazy initialization moves the cost to the first request that needs the client, which helps only if not every request needs it. There is no free version.
COLD INVOCATION WARM INVOCATION runtime start ~180 ms runtime start 0 ms (reused) package load ~120 ms package load 0 ms (reused) module init ~ 90 ms module init 0 ms (reused) db client + TLS ~110 ms db client 0 ms (reused, if it survived) ------------------------------ ------------------------------ init subtotal ~500 ms init subtotal 0 ms handler body ~ 40 ms handler body ~40 ms ============================== ============================== user-visible ~540 ms user-visible ~40 ms p50 is warm. p99 is cold. Reporting the mean hides the entire problem.
The constraint sheet
Each constraint below is a design input, not a complaint. Read the right-hand column as "this workload is disqualified unless" rather than as a warning label — the model is excellent for work that clears all of them and awkward for work that clears most.
The statelessness row is the one that bites hardest in practice, because it fails *intermittently*. A warm environment reuses whatever the last invocation left in module scope, so a cached tenant id or a half-configured client works perfectly until the environment turns over, and then produces a bug that reproduces on roughly one request in a hundred.
| Constraint | What it means in practice | Disqualifies |
|---|---|---|
| Cold start | Scale-out and post-idle requests pay initialization on the request path. | Latency-critical synchronous paths with a tight p99 budget and bursty traffic. |
| Wall-clock limit | The platform kills the execution at a hard ceiling, mid-work, with no unwind. | Long report generation, large migrations, video transcoding of full-length files. |
| Enforced statelessness | Environments are reclaimed without notice; module-scope state survives only by accident. | In-process caches, session affinity, anything that assumes the same process handles the next call. |
| Concurrency ceiling | Limits are per function and per account; exceeding them throttles rather than queues. | Workloads whose burst rate is unbounded, unless a queue is placed in front to absorb it. |
| Thin execution context | No shell, no sidecar, no persistent agent; you get the platform's logs and traces. | Deep profiling, custom instrumentation agents, anything that expects to inspect a running process. |
| Vendor shape | Triggers, packaging, identity injection and limits are the least portable layer in cloud. | Workloads under a hard portability requirement — though the handler body usually ports fine. |
| External connections | A pool per environment, multiplied by concurrency, against a fixed database limit. | Anything talking to a pooled relational database without a proxy. See Serverless and Database Connections. |
"Serverless is always cheaper" is a red flag
The claim is true in exactly one regime and false in the other, and the regimes are easy to tell apart. Serverless wins decisively when utilization is low: a workload that is busy 3% of the day pays for 3% of the day, while an instance sized for its peak pays for all of it. This is a large, real saving, and it is why the model exists.
At steady high traffic the arithmetic reverses. A function billed per invocation and per gigabyte-second, running continuously, is paying a premium for allocation flexibility it is no longer using. The same continuous load on a right-sized instance — especially a committed or reserved one — is usually materially cheaper, because a reservation is the exact opposite trade: give up flexibility, get a discount. An engineer who says "serverless is cheaper" without asking about the duty cycle has skipped the only question that determines the answer.
And the compute line is rarely the whole bill. The gateway in front, the queue between steps, the log ingestion from a function that logs on every invocation, and the data transfer out all have their own meters. Log ingestion in particular surprises people, because per-invocation logging that is trivial at 1000 invocations a day is a line item at 100 million.
Bars are relative weights, not currency. Real rates depend on provider, region, commitment and volume.
Key points
- A cold start is ordinary initialization moved onto the request path, and it clusters exactly when traffic is rising.
- Every mitigation for cold starts costs something; provisioned concurrency costs the always-on bill the model was meant to remove.
- Enforced statelessness fails intermittently, because a warm environment happily reuses whatever the previous invocation left behind.
- Concurrency limits throttle rather than queue — put a queue in front if the burst rate is unbounded.
- "Serverless is always cheaper" is a red flag: it is cheaper at low duty cycle and usually more expensive at steady high traffic than a reserved instance.
The loop, answered
Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.
- • Scale-out creates a new execution environment, which must run runtime start, package load and module initialization before the handler sees the event.
- • A frozen environment is reused if another event arrives soon enough, which is why warm and cold latencies differ by an order of magnitude for identical code.
- • The platform enforces a wall-clock deadline in the execution context and terminates the environment when it expires.
- • Concurrency is counted per function and per account; requests beyond the limit are rejected with a throttle response rather than buffered.
- • Billing meters invocation count and allocated-memory × duration, so memory size scales both cost per millisecond and available CPU.
- • Keep the deployment package small and initialize lazily — this is the highest-leverage cold-start work and it is entirely in your control.
- • Set per-function concurrency reservations so one function cannot consume the account ceiling and throttle the others.
- • Choose memory by measuring, not by guessing: raising memory often lowers total cost because duration falls faster than the per-millisecond rate rises.
- • Make handlers idempotent, because async triggers retry and timeouts leave partial work.
- • Track the runtime deprecation calendar; the upgrade is mandatory and dated by the provider.
- • A traffic spike produces a wave of cold starts, and p99 latency triples while p50 stays flat and every dashboard looks fine.
- • A slow downstream dependency pushes executions into the wall-clock limit, so a latency problem becomes a hard error with partially applied writes behind it.
- • The account concurrency ceiling is reached by a batch job, and unrelated user-facing functions start being throttled.
- • State cached in module scope produces wrong answers on reused environments only — a bug that reproduces one time in a hundred and never locally.
- • A retried async invocation duplicates a side effect that was not idempotent.
- • Scales with arrival rate to the concurrency ceiling, then stops scaling and starts rejecting.
- • Burst ramp is itself rate-limited, so the front edge of a spike is served with elevated latency even below the ceiling.
- • The real limit is nearly always downstream: database connections, third-party rate limits, or a partner quota.
- • Cost scales linearly with traffic with no floor and no volume break — the opposite curve from a reserved fleet.
- • A smaller attack surface at the host level: no SSH, no long-lived instance to patch, no persistent foothold across invocations.
- • A correspondingly larger dependency surface: the deployment package is the supply chain, and it runs with the function's role. See The Infrastructure Supply Chain.
- • Environment-variable configuration is the default and is the wrong place for secrets; fetch them from a secret store at initialization instead. See Secrets in Infrastructure.
- • Thin observability makes intrusion detection harder — there is no agent on the box because there is no box.
- • Cheapest at low duty cycle, where idle genuinely costs nothing.
- • Usually more expensive than a reserved instance at steady high traffic, because you keep paying for flexibility you have stopped using.
- • Provisioned concurrency converts part of the bill back to fixed, trading the model's core cost advantage for predictable latency.
- • Log ingestion, gateway requests and data transfer commonly exceed the compute line and are absent from every back-of-envelope comparison.
- • Cold-start count and init duration as their own series, separate from total duration.
- • Throttle count — the only signal that says you hit a ceiling rather than a bug.
- • p99 duration, split cold from warm; the split is the entire story.
- • The signal that lies: average duration and average cost per invocation. Both are dominated by warm, cheap invocations and stay flat while the tail degrades.
- • A right-sized instance or a small container fleet, when traffic is steady — cheaper, no cold start, and it can hold a connection pool.
- • A scale-to-zero container platform, when you want the no-machines property without the wall-clock limit and packaging constraints.
- • A queue plus a modest worker pool, when the burst is real but the latency requirement is not — the queue absorbs what the concurrency ceiling would have rejected.
- • Doing nothing: a cron entry on a host you already operate beats a new event platform for a single nightly task.
- • Buys zero idle cost; costs tail latency on every scale-out and after every quiet period.
- • Buys automatic scaling; costs you a hard ceiling that rejects rather than degrades.
- • Buys a tiny operational surface; costs depth of observability and any ability to profile a live process.
- • Buys speed of delivery; costs portability, because triggers, packaging and identity are the most provider-shaped layer you can adopt.
Cold start, concurrency limit, and 1000 connections
What people believe, and what is true
Serverless is always cheaper.
It is cheaper when utilization is low. At steady high traffic a reserved instance is usually cheaper, because a reservation trades flexibility for a discount and a function charges you for flexibility on every request.
Cold starts were solved.
They were reduced, and they can be pre-paid with provisioned concurrency — which restores the always-on cost. The initialization work is still real and still on the request path when an environment is new.
Functions cannot be over-provisioned.
Memory size is a provisioning decision that sets both CPU share and price. A default memory setting is as much a sizing mistake as a default instance type.