C++ Backend Services

1#include <sys/epoll.h>
2#include <unistd.h>
3#include <thread>
4#include <vector>
5#include <array>
6 
7// One loop per core. Each owns its own epoll instance and its own
8// connections. Nothing on the hot path is shared, so nothing on the
9// hot path needs a lock.
10class Reactor {
11public:
12  Reactor() : epfd_(::epoll_create1(0)) {}
13  ~Reactor() { if (epfd_ >= 0) ::close(epfd_); }   // RAII: the fd is
14                                                   // owned, so it is closed
15 
16  Reactor(const Reactor&) = delete;                // non-copyable: an fd
17  Reactor& operator=(const Reactor&) = delete;     // has one owner
18 
19  void add(int fd, uint32_t events) {
20    epoll_event ev{};
21    ev.events = events;
22    ev.data.fd = fd;
23    ::epoll_ctl(epfd_, EPOLL_CTL_ADD, fd, &ev);
24  }
25 
26  void run() {
27    std::array<epoll_event, 256> events{};
28    for (;;) {
29      int n = ::epoll_wait(epfd_, events.data(), events.size(), -1);
30      for (int i = 0; i < n; ++i) {
31        handle_ready(events[i].data.fd, events[i].events);
32        // Anything slow here costs this ENTIRE core: every connection
33        // owned by this loop waits. Same rule as any event loop --
34        // the unit that stalls is just one core instead of the process.
35      }
36    }
37  }
38 
39private:
40  void handle_ready(int fd, uint32_t events);
41  int epfd_;
42};
43 
44int main() {
45  const unsigned cores = std::thread::hardware_concurrency();
46  std::vector<Reactor> reactors(cores);
47  std::vector<std::thread> threads;
48  threads.reserve(cores);
49 
50  for (unsigned i = 0; i < cores; ++i) {
51    // Each thread runs one reactor. Connections are partitioned across
52    // them (SO_REUSEPORT gives each thread its own accept queue), so
53    // two cores never touch the same connection state.
54    threads.emplace_back([&reactors, i] { reactors[i].run(); });
55  }
56  for (auto& t : threads) t.join();
57}

Two things to take away even if you never write this: the loop-blocking rule from Node applies per core here, and the reason this design is fast is that nothing is shared, not that the language is C++.

// Garbage collector pauses land in the tail:
//   p50 unaffected, p99.9 shows the pauses
// Per-request allocation you do not control
// Memory overhead: object headers, heap slack
// JIT warm-up after every deploy or restart
// Less control over data layout and cache behaviour

// For a typical business service, ALL of this is
// invisible next to a 20ms database query.

// Memory-safety bugs that are security vulnerabilities
// Data races as undefined behaviour, not wrong answers
// Longer development and review cycles
// A much smaller pool of engineers who can be on call
// Ecosystem work: SDKs, auth, metrics, tracing
// Build and dependency management complexity

// For a proxy on the path of every request in the
// company, ALL of this can be worth paying.

Neither column is a defect list. They are the two bills, and which one you would rather pay depends on whether the component's cost is dominated by waiting on other systems or by its own CPU and its own tail. Measure that first; the language question answers itself afterwards.