Files & I/OIntermediate

What a file descriptor really is

“What is a file descriptor, really? What is descriptor 3, and why are sockets, pipes and epoll instances also descriptors?”

What this tests

  • The three-level structure: per-process table → open file description → underlying object
  • Lowest-available numbering and the 0/1/2 convention
  • Sharing semantics: dup, fork, and the shared offset
  • Limits and leaks: ulimit, EMFILE, CLOSE_WAIT, deleted-but-open files

Answers by level

Read the beginner answer first and notice what is missing.

The descriptor is an index into a per-process table owned by the kernel. Each slot points to an open file description — a kernel object holding the current offset, the access mode and status flags (O_NONBLOCK, O_APPEND) — which in turn points to the actual thing: an inode for a regular file, a socket, a pipe, a device, an epoll instance, a timerfd. The number is meaningless outside the process; descriptor 3 in one process and descriptor 3 in another are unrelated (File Descriptors).

open() returns the lowest unused slot, and processes start with 0, 1, 2 wired by convention — so 3 is simply the next free index, which is why shell redirection works by closing and reopening low numbers. Two open() calls on the same file create two open file descriptions with independent offsets; dup()/dup2() and fork() create new slots pointing at the same description, so the offset is shared — a parent and child both writing to an inherited log descriptor interleave correctly instead of overwriting each other.

Sockets, pipes and everything else are descriptors because the kernel exposes one interface — read, write, close, poll — over anything that can produce or consume bytes or events. That uniformity is what makes epoll possible: one wait over sockets, pipes, timers and signals at once (Everything Is I/O, The Socket: A Descriptor With Two Kernel Buffers Behind It).

The table is bounded: the soft limit per process (ulimit -n, historically 1,024, commonly raised to 65,536) and a hard limit above it; system-wide fs.file-max. Exhausting the per-process table gives EMFILE, the system table ENFILE (label: Linux/Unix). A descriptor leak is the usual way to hit it: a socket never closed after the peer hung up (visible as CLOSE_WAIT in ss), a file opened in an error path without close, a child process that inherited everything. ls /proc/<pid>/fd | wc -l over time tells you whether the count is growing.

Green flags · Red flags

Strong green flag · Reaches for /proc/<pid>/fd and lsof to classify before raising the limit.
Green flags
  • Describes the table → open file description → object structure
  • Knows dup/fork share the offset and two opens do not
  • Explains why sockets and epoll are descriptors
  • Names EMFILE vs ENFILE and the CLOSE_WAIT leak signature
  • Mentions CLOEXEC or deleted-but-open files
Red flags
  • Thinks the descriptor is a pointer to the file on disk
  • Believes closing a descriptor always closes the file
  • First response to EMFILE is "raise ulimit"

Follow-up questions

F1
Two processes each open the same file and read; do they share an offset?
F2
Why does df show a full disk while du finds nothing large?
F3
What does a growing count of CLOSE_WAIT sockets mean?

Scenario

A service’s descriptor count climbs by ~40 per hour and it dies after two days with EMFILE. lsof shows mostly IPv4 sockets to one upstream. Walk through the diagnosis and the fix.

Learn this topic