Webhooks
Your contract running against someone else’s server: delivery states, retries, duplicate events, ordering you must not assume, and the signature that makes any of it trustworthy.
Every lesson below names the consumers, the design question and the guarantee before recommending anything. Recommendations come with what they cost, when not to use them, and how they evolve.
A webhook flips the roles: the provider becomes the client, calling an endpoint the consumer operates. Delivery is asynchronous and at-least-once, so the event envelope — event id, delivery id, type, timestamp — is what makes the stream usable, not the payload.
Every event delivery is a little state machine: queued → attempting → delivered, or failed → retrying → dead. The retry schedule, the definition of "delivered", and the dead-letter escape hatch are contract clauses both sides build against.
The provider promised at-least-once, so duplicates are not a bug — they are scheduled. Exactly-once processing is an illusion the consumer manufactures locally: record the event_id, process each id exactly once, and make the recording atomic with the effects.
Retries, parallel dispatch and redrives mean events arrive in whatever order the network permits — `order.shipped` before `order.paid` is routine. Consumers that apply event payloads as state, in arrival order, corrupt their data; the contract must say so and give them a defense.
A webhook receiver is an unauthenticated public POST endpoint that triggers business logic — unless the contract says how events are signed, how timestamps bound replay, and how secrets rotate. Signature verification is the consumer's only proof that an event is yours.