Why would you use Kafka in a data platform?
Whether the candidate can name the specific property a durable partitioned log provides that the alternatives do not, and whether they will decline it when nothing needs that property.
The situation behind the question
Interviewers ask this because it happened to them.
A team is designing ingestion for six sources. Someone has put a broker in the middle of the architecture diagram. Asked why, they say it is the standard pattern.
A strong answer
Flags
Green flags
- Names replay and fan-out as the reasons, and can say what each one buys concretely.
- Scopes the ordering guarantee to a partition and explains what happens when a key moves between partitions.
- Treats retention as a recovery-window decision rather than a storage-cost decision.
- Is willing to say that a broker is unnecessary for this platform, if that is the answer.
Red flags
- "Because Kafka is scalable." Scalability is not a reason to introduce a component; it is a property that matters only once a specific load exists, and it says nothing about which problem the log is solving here.
- "Kafka guarantees exactly-once." Delivery to a consumer is at-least-once by default; anything stronger requires the consumer's state update and output write to be transactional or idempotent, and that is a property of how you wrote the consumer.
- "Streaming is always better than batch." Freshness is bought with state, ordering, lateness handling and a harder debugging story, and it is worth buying only where a consumer has a latency requirement.
- Cannot describe what happens when a consumer is down longer than the retention window.
Follow-ups
Where the conversation goes if the first answer holds up.
- A consumer has been down for longer than retention. What are your options, and which of them recover the data?
- You need per-customer ordering. How do you get it, and what happens when you change the partition count?
- What would you use instead if there is exactly one consumer and a nightly schedule is acceptable?