OS + NetworkingAdvanced

What does "zero-copy" actually eliminate?

“Serving a static file over a socket: count the copies and mode switches in the naive `read()` + `send()` loop, then explain what `sendfile()`, `mmap()` and `splice()` change and when they matter.”

What this tests

  • Knowing where the copies are: disk → page cache → user buffer → socket buffer → NIC
  • The cost model: memory bandwidth and syscalls, not disk
  • Which mechanisms remove which copies
  • Judgement about when it is worth it

Answers by level

Read the beginner answer first and notice what is missing.

Naive loop: read() — DMA from disk into the page cache (copy 1, done by the device), then copy_to_user into the buffer (copy 2, CPU); send()copy_from_user into the socket buffer (copy 3, CPU), then DMA to the NIC (copy 4, device). Four copies, two of them CPU memcpys, and two syscalls per chunk, each a pair of mode switches.

sendfile(out_fd, in_fd, offset, count) does it in one syscall inside the kernel: page cache → socket buffer without the round trip through user space, so the two CPU copies collapse to one, or with scatter-gather NICs to zero — the socket only holds descriptors pointing at page-cache pages. mmap() removes the read copy by mapping the file, but send() still copies. splice() moves pages between a file and a pipe or socket via the pipe buffer.

It matters when throughput is bound by memory bandwidth and CPU, not disk — a CDN edge or a file server saturating 25–100 Gbit/s. For a typical API response of 2 kB it is noise; the syscall count and the TLS encryption (which must touch every byte in user space, unless kTLS) dominate.

Green flags · Red flags

Strong green flag · Mentions kTLS or MSG_ZEROCOPY and the pinning trade-off unprompted.
Green flags
  • Counts four copies and identifies which are CPU
  • Distinguishes sendfile, mmap and splice by what each removes
  • Knows TLS breaks the simple story
  • Says when it is not worth it
Red flags
  • Thinks zero-copy skips the disk or the NIC DMA
  • Believes it helps small API responses
  • Cannot place the page cache in the picture
  • Confuses mmap with zero-copy send

Follow-up questions

F1
Why is mmap + write not zero-copy?
F2
What does a mode switch cost and why does it matter here?

Scenario

A static file server hits 100% CPU at 8 Gbit/s on a 40 Gbit link while the disk is nearly idle. Where is the CPU going and what would you try, in order?

Learn this topic