Queue vs pub/sub vs log

A work queue delivers each message to one consumer: it is competing consumers over a shared queue, with ack, visibility timeout (redeliver if not acked in time), retry counts and a dead-letter queue for poison messages. Once acked, the message is gone. It fits background jobs — resize this image, send this email — where the point is to distribute work and never lose it.

Pub/sub delivers each message to every subscriber: OrderPlaced reaches Email, Inventory and Analytics, each with its own subscription and its own retry state. Subscribers that do not exist at publish time miss the message. It fits fan-out of facts. A log (Kafka) keeps messages for a retention period regardless of consumption; consumers track an offset per partition and can rewind. It gives fan-out and replay and per-partition ordering, at the cost of managing partitions, consumer groups, lag and rebalances.

The choice: distribute work → queue; fan-out facts to a few consumers → pub/sub; replay, ordering, high volume or many consumers reading the same stream at different speeds → log. Kafka for a job queue is a mismatch — no per-message ack, no visibility timeout, slow messages block a partition.