Core, Hardware Thread, Software Thread
Three things routinely called "a thread", stacked on top of each other. A core is silicon that executes. A hardware thread is an execution context inside it. A software thread is an OS bookkeeping structure that gets mapped onto one. Conflating them is how "we have 16 threads" becomes a wrong capacity estimate.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
The three things, kept apart
A physical core is a complete execution engine: fetch, decode, execution units, private caches. It is the unit of actual compute throughput. Two cores can genuinely do two things at once.
A hardware thread, also called a logical CPU or a strand, is an architectural execution context inside a core: its own register file and program counter, so the core can hold more than one instruction stream and switch between them without OS involvement. On a core without SMT, one core is one hardware thread. On a core with two-way SMT, one core presents two.
A software thread is an OS object — a stack, a saved register set, scheduling metadata. There can be thousands of them. The scheduler multiplexes them onto the available hardware threads, giving each a time slice. Creating more software threads never creates more execution capacity; it only creates more things competing for the same capacity.
| Physical core | Hardware thread | Software thread | |
|---|---|---|---|
| What it is | Silicon that executes instructions | An execution context inside a core | An OS-scheduled entity |
| Typical count | Fixed by the chip | Cores × SMT width | Thousands, limited by memory |
| Created by | The manufacturer | The manufacturer | Your program, at runtime |
| Adds throughput? | Yes — real added capacity | Sometimes — fills stall cycles only | No — it only adds contenders |
| Owns private cache? | Yes, L1 and usually L2 | No — shares the core's caches | No — inherits whatever it runs on |
| Switching cost | n/a | Near zero, done in hardware | A context switch; see Context Switching |
How the counts stack up
Reading a machine correctly means reading all three numbers. A part with 8 physical cores and two-way SMT presents 16 logical CPUs. A program that starts 200 software threads on it has 200 runnable entities sharing 16 contexts sharing 8 cores. Nothing about starting the 200th thread made the machine capable of more work.
The practical consequence is that "number of CPUs" as reported by the OS is the *wrong* input for some decisions and the right one for others. For a CPU-bound thread pool, cores is usually the better bound, because two SMT siblings competing for one core's execution units do not deliver two cores' worth of throughput. For an I/O-bound pool, neither number is the constraint and both are the wrong question — see Hardware Threads Are Not OS Threads.
Where the terminology actually bites
The confusion is not academic. Sizing a CPU-bound worker pool from the logical CPU count on an SMT machine typically oversubscribes the execution units: the threads run, but each runs slower, and tail latency rises even though total throughput may be flat or slightly better. Sizing from the core count leaves SMT's stall-filling benefit unused. Which is right depends on the workload, and the only way to know is to measure both.
The same ambiguity appears in cloud instance specs, where a "vCPU" is usually a *logical* CPU — so an 8-vCPU instance is frequently 4 physical cores. Capacity estimates that assume 8 cores of compute will be wrong by close to a factor of two on CPU-bound work. Choosing a Compute Model treats the purchasing side of that; the hardware reason is here.
- Thread pool for CPU-bound work — start from physical cores, then measure whether adding SMT siblings helps this workload.
- Thread pool for I/O-bound work — neither count is the bound; concurrency is limited by the I/O, not the execution contexts.
- Cloud vCPU — usually a logical CPU, so halve it for a physical-core estimate on SMT parts, and check the instance documentation rather than guessing.
- Benchmark reporting — always state which count was used, or the result cannot be reproduced or compared.
Key points
- Core, hardware thread and software thread are three different quantities, and only the first is added execution capacity.
- Hardware threads share their core's execution units and caches; they are contexts, not extra cores.
- Software threads are OS objects multiplexed onto hardware threads — creating more never creates more capacity.
- The OS-reported "CPU count" is normally logical CPUs, which is the wrong bound for a CPU-bound pool on an SMT machine.
- A cloud vCPU is usually a logical CPU, so physical-core capacity is often half the advertised number.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Program → OS: a thread-creation call allocates a stack and a scheduling structure; no hardware is involved yet.
- 2OS scheduler → logical CPU: the scheduler picks a runnable software thread and loads its register state into a hardware context.
- 3Logical CPU → core front end: the context feeds instructions into the shared fetch and decode of its physical core.
- 4Core → execution units: SMT siblings interleave into the same execution units, filling cycles the other leaves idle.
- 5Core → private caches: both siblings share one L1 and L2, so their working sets compete for the same capacity.
- • "The machine says 16 CPUs, so I have 16 cores" — on an SMT part that is 8 cores presenting 16 contexts.
- • "More software threads means more parallelism" — past the hardware thread count it means more context switching.
- • "Hyper-threading gives me twice the CPU" — it gives extra contexts to fill stalls, not extra execution units.
- • "vCPU means core" — usually it means logical CPU, and the difference is the whole capacity estimate.
Consequences, controls and cost
- • A pool sized to logical CPUs on SMT hardware oversubscribes execution units and raises tail latency.
- • Capacity estimates from vCPU counts overstate CPU-bound throughput, often by close to a factor of two.
- • Two benchmarks quoting "16 threads" can mean 16 cores or 8 cores, and are not comparable.
- • Adding software threads past the hardware thread count adds context-switching overhead and nothing else.
- • Read all three counts from the machine before sizing anything, and record which one a benchmark used.
- • Size CPU-bound pools from physical cores as a starting point, then measure whether SMT siblings help.
- • For cloud capacity, check whether the instance family sells logical or physical CPUs rather than assuming.
- • Stop adding software threads once the runnable count exceeds the hardware thread count — past that, you are only adding switches.
- • Read the topology directly rather than trusting a single number: sockets, cores per socket, and threads per core.
- • Compare throughput at `cores` threads versus `logical CPUs` threads for your actual workload — the gap is SMT's real contribution.
- • Check whether sibling logical CPUs are being scheduled onto the same core when you expect them not to be.
- • For a cloud instance, benchmark single-thread throughput at full load versus idle; heavy degradation suggests shared physical cores.
- • Sizing to physical cores leaves SMT's stall-filling gains unused on workloads that would benefit.
- • Sizing to logical CPUs improves throughput on some workloads while degrading per-request latency on others.
- • Pinning to avoid sibling contention costs scheduler flexibility and can leave cores idle under imbalance.
Scope
§224 — what these claims are specific to.
- GENERALThe three-level distinction holds on every mainstream platform. What varies is SMT width — commonly two, but some server and accelerator designs use four or eight contexts per core.
- PLATFORM-SPECIFICHow vCPUs map to physical cores is a cloud-provider and instance-family property, not a hardware one.
Misconceptions
Where the rest of this lives
The hardware sets the ceiling; choosing a pool size for a given workload shape — CPU-bound, I/O-bound, mixed — is a concurrency decision that starts from these three counts.