How Do You Size a Thread Pool?

How many threads should a pool have?

The scenario

A Java service runs a fixed pool of 200 threads in a container limited to 2 CPUs, with an unbounded task queue in front. Under a traffic spike, latency climbs to 40 seconds and the heap fills with queued task objects.

What it tests

  • Whether the candidate refuses to give a universal number
  • Whether they reason from what the threads spend their time doing
  • Whether they consider the downstream resources the pool feeds

Answer it out loud first. Recognising a good answer is not the same skill as producing one.