What does "exactly-once" actually mean here?

Whether the candidate can decompose the phrase into the three places it might apply and name the assumption that buys each one.

Streaming

The situation behind the question

Interviewers ask this because it happened to them.

A vendor page and an internal design document both describe the streaming pipeline as exactly-once. The pipeline reads from a broker, updates aggregate state, and writes to a warehouse table that a dashboard reads.

A strong answer

Flags

Green flags
  • Decomposes the term rather than accepting or rejecting it wholesale.
  • Identifies the output write as the part that matters to consumers and the part usually unaddressed.
  • Reaches for idempotent writes on a business key as the practical mechanism.
  • Expects duplicates as a normal condition of the transport rather than as a fault.
Red flags
  • "Kafka guarantees exactly-once." The broker can offer transactional semantics within its own boundary; nothing it does makes an arbitrary external sink idempotent.
  • Uses the phrase without naming which of consumption, state or output it applies to.
  • Believes enabling a framework setting makes end-to-end duplicates impossible regardless of the sink.
  • Cannot describe what happens on a restart between the write and the offset commit.

Follow-ups

Where the conversation goes if the first answer holds up.

  • Your sink is an append-only warehouse table. What are your options for making the write idempotent?
  • The consumer restarts between writing output and committing its offset. What does the consumer of your output see?
  • Where would you put a deduplication step, and what does it cost in state?