The question this answers
What sits between my application code and the electricity, and which layer is a given failure actually coming from?
An engineer is paged for "the API is slow". The application code has not changed. Something below it has, and there are seven candidate layers, each with a different owner and a different diagnostic.
A fixed ordering to reason with. Each layer provides a specific guarantee to the one above it and fails in a characteristic way, so a symptom can be attributed to a layer instead of investigated everywhere at once.
Seven layers, and what each one promises the layer above
The stack below is the domain's spine. It is not an OSI-style formalism; it is a debugging tool and a responsibility map. Read it downward to understand what you are standing on, and upward to understand how a low-level failure will present itself — because it never presents itself as what it is. A saturated disk on the host does not say "disk saturated"; it says "our p99 doubled and the database looks fine".
The critical property is that each layer hides the one below it, and hides it imperfectly. A container promises you an isolated filesystem and process tree, and does not promise you an isolated page cache or an uncontended CPU. A virtual machine promises you a machine, and does not promise you exclusive access to the physical cores — which is what steal time is, and why an instance can be slow while every metric inside it looks idle. Every abstraction leaks, and the leaks are where the interesting incidents come from.
The second property is that the owner changes partway down. Somewhere in this stack is a line where your responsibility ends and the provider's begins, and where that line sits depends on the compute model you chose. That is the subject of Shared Responsibility; the layer stack is what makes the line drawable.
A symptom, read down the stack
Take the page: "checkout p99 went from 180 ms to 2.4 s, no deploy". The layered stack turns an unbounded search into an ordered one, and the order matters because the cheap checks are at the top and the expensive ones are at the bottom.
Application: did anything change — a feature flag, a config value, a data volume crossing a threshold that flipped a query plan? Runtime: are GC pauses or event-loop lag correlated with the latency? Container: is the process being CPU-throttled against its limit, which shows as latency while utilization looks moderate — see OOM Kills and CPU Throttling. Host: are file descriptors, connections or disk space near a limit? Compute: is steal time non-zero, meaning another tenant is taking cycles? Fabric: has network-attached disk latency risen, or is one zone's path degraded?
The reason to walk this deliberately is that the top and bottom layers produce *identical* symptoms. Elevated latency with normal CPU is the signature of a slow query, a GC pause, CPU throttling, disk latency and a network partition. Five layers, one symptom. Without a layer model you check whichever one you know best, which is why teams with strong application skills spend hours in the profiler while the answer is a saturated network disk.
- Elevated latency with normal CPU is produced by at least five different layers. The metric does not identify the layer; the ordered walk does.
- A container reporting 60% CPU while being throttled is not lying — it is reporting its own consumption, not its quota denial. Different counter, different layer.
- Steal time is the only in-instance signal of a layer you do not own, which makes it disproportionately valuable and almost never on a dashboard.
- Network-attached storage latency is the most commonly missed cause of "the database got slow", because the database is innocent and its own metrics say so.
Where the abstraction leaks, and who owns the leak
Every layer boundary is a contract, and each one has a well-known clause that is not honoured the way people assume. Knowing these four or five leaks is most of what separates an engineer who can debug production from one who can only debug an application.
The other axis is ownership, and it moves with the compute model. On a virtual machine you own everything from the guest OS upward; on a managed container platform you own from the image upward; on a function platform you own from the code upward. Choosing a compute model is therefore choosing where in this stack your pager stops — which is the real content of Choosing a Compute Model, and why "serverless has no servers" is a statement about ownership rather than about physics.
The corollary is uncomfortable and worth stating: the layers you no longer own can still take you down, and you will have less visibility into them precisely because you handed them over. A managed platform's scheduler making a bad placement decision is invisible to you in a way that your own scheduler making the same decision would not be.
| Boundary | What it promises | The leak | Who owns it on VM / container platform / functions |
|---|---|---|---|
| App ↔ runtime | Memory is managed for you | GC pauses are latency your code did not cause and cannot see in a profiler sample | you / you / you |
| Runtime ↔ image | What you tested is what runs | Base image drift and unpinned dependencies — the image is only reproducible if you made it so | you / you / you |
| Image ↔ isolation | You get CPU and memory limits | Exceeding memory is an OOM kill with no application error; exceeding CPU is throttling, not an error at all | you / shared / provider |
| Isolation ↔ host | A private view of the machine | Page cache, kernel version, conntrack tables and file descriptors are shared or host-wide | you / provider / provider |
| Host ↔ physical | A machine of a given size | vCPUs are shares of physical cores; steal time is the leak — see Hypervisors and Shared Hosts | provider / provider / provider |
| Physical ↔ fabric | A disk and a network interface | Both are network services with their own queues, and a "disk" can have a bad minute for reasons inside the storage fleet | provider / provider / provider |
Key points
- Seven layers: application, runtime, image, isolation boundary, host OS, physical compute, and network/storage fabric on top of a data centre.
- Each layer hides the one below it imperfectly; the imperfections — GC pauses, throttling, steal time, network-disk latency — are where production incidents come from.
- One symptom maps to several layers. An ordered walk from cheapest to most expensive check is what turns an unbounded search into a bounded one.
- The compute model decides where in this stack ownership transfers to the provider, which is the same thing as deciding where your pager stops.
- Layers you no longer own can still take you down, and you have less visibility into them because you handed them over.
The loop, answered
Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.
- • Physical hardware in a facility is divided by a hypervisor into virtual machines, each with a guest kernel that believes it owns a machine.
- • Inside a guest, a container runtime uses kernel namespaces and cgroups to give a process group a private filesystem and process view plus enforced resource limits.
- • Inside a container, a language runtime maps application objects onto that limited memory and those scheduled cores.
- • Storage and network are not local devices but fabric services reached over the network, which is why their latency behaves like a network dependency rather than like a bus.
- • Metrics from at least three layers — application, container/instance, and host or fabric — or a symptom at one layer cannot be excluded from the others.
- • Knowing which layers you own for each workload, because it differs per compute model within the same organization.
- • Keeping the base image and guest OS patched on every layer you own, which is a continuous obligation and the one most often skipped — see Mutable Servers and Immutable Images.
- • A written debugging order, so that a 03:00 investigation follows the stack rather than the responder's specialism.
- • Latency attributed to the application for hours while the cause is CPU throttling against a container limit set six months ago.
- • A process disappearing with no log line and no stack trace: an OOM kill at the isolation layer, visible only in kernel or platform events.
- • A "database problem" that is network-attached disk latency, with every database-internal metric reporting normal.
- • An instance that is slow with idle-looking internal metrics, because another tenant on the physical host is consuming cycles.
- • A change at the fabric layer — a route, a firewall rule, a security group — presenting as a hung connection rather than a refused one, which is why it is misdiagnosed as an application timeout.
- • Each layer has its own ceiling and they are hit in a characteristic order: application connections, then container memory, then host file descriptors, then instance network bandwidth.
- • Adding capacity at the wrong layer is the most common wasted intervention — more instances do nothing when the constraint is a per-instance connection limit at the database.
- • The lower the layer, the coarser the scaling unit and the longer the lead time; you add application instances in seconds and physical capacity in weeks.
- • Each boundary is also a trust boundary, and their strengths differ substantially: a hypervisor boundary is far stronger than a container boundary, which is a shared-kernel boundary — see Containers vs Virtual Machines.
- • A vulnerability in a lower layer defeats every control above it, which is why patching the guest OS and base image is a security obligation rather than hygiene.
- • The layers you do not own are also the ones you cannot audit; provider attestations replace your own evidence there — see Shared Responsibility.
- • Multi-tenancy at the physical layer is real. It is well managed by providers and it is not zero risk, which is why regulated workloads sometimes pay for dedicated hosts.
- • You pay at the layer you rent: instance-hours on a VM, request-seconds on a function platform, and node-hours plus a control plane on a managed cluster.
- • Handing a layer to the provider converts operational cost into billed cost — usually a good trade for a small team and a poor one at large steady scale.
- • Fabric layers have their own meters that feel like they should be free: network-attached disk IOPS, cross-zone traffic and storage requests all bill separately from the compute they serve.
- • Application: error rate and latency percentiles. Runtime: GC pause time or event-loop lag. Container: throttled periods and OOM events.
- • Host: file descriptors, disk space, connection tracking table occupancy. Instance: steal time and network throughput against the instance ceiling.
- • Fabric: block-device latency and queue depth, which is the metric that exonerates the database.
- • The signal that lies: container CPU utilization during throttling. It reports what the process consumed, not what it was denied, so it looks comfortable while requests queue.
- • A managed platform that removes the bottom four layers from your responsibility entirely. For most teams this is the right trade — fewer layers to instrument, at the price of less visibility when the provider's layer misbehaves.
- • Fewer layers rather than better tools: running a process directly on a VM removes the container and orchestrator layers and removes their failure modes with them. For a single steady service this is a legitimate and underrated design.
- • Bare metal for workloads where the hypervisor leak is intolerable — deterministic-latency systems and some database workloads genuinely benefit, at the cost of everything the virtualization layer was providing.
- • Every layer added buys portability, density or automation and charges a new failure mode plus a new place to look at 03:00.
- • Handing layers to a provider reduces your operational surface and reduces your diagnostic reach into exactly the layers you can no longer fix.
- • A thorough layer model costs instrumentation at each level, which is real money and real setup time, and it is the difference between a 20-minute and a 6-hour incident.
The infrastructure layer stack
request path, top to bottom: application → runtime → container → host → compute → network → physical the layer you can observe is rarely the layer that broke
What people believe, and what is true
Serverless has no servers, so there is no stack.
The stack is identical; you just do not own most of it. Cold starts, memory limits and concurrency caps are the same isolation and host layers presenting through a different interface — see Serverless Trade-offs.
If application metrics look fine, the application is fine.
Application metrics are measured inside the top layer and cannot see throttling, steal time or fabric latency. They are necessary and nowhere near sufficient.
Containers isolate like virtual machines.
They share a kernel. That is a materially weaker boundary and a deliberate trade for density and startup speed — see Containers vs Virtual Machines.