System designAdvanced
Design a Chat App
One-to-one and group messaging with delivery and read receipts, presence, and push notifications when the recipient is offline. The core difficulty is not storage, it is routing: a stateful WebSocket connection lives on one server, and the recipient is on another.
Functional requirements
- Send and receive text messages in 1:1 and group conversations (groups up to 500 members) in real time.
- Messages within a conversation appear in the same order for every participant.
- Sender sees three states per message: sent (server has it), delivered (recipient device has it), read.
- Presence: online / last seen, visible to contacts.
- History: open a conversation and page backwards through older messages.
- Offline recipients get a push notification; on reconnect the client receives everything it missed, exactly once from its point of view.
- Multiple devices per user stay in sync.
Non-functional requirements
Scale, latency, availability and durability targets — these decide the architecture.
- Message delivery latency p95 < 200 ms end-to-end when both parties are online.
- Scale: 50M daily active users, 10M concurrent connections at peak, 2B messages/day.
- Durability: an acked message is never lost; history retained indefinitely (or until the user deletes).
- Availability 99.95% for send/receive (4.4 h/yr); presence may degrade first.
- Ordering guarantee is per conversation only — no global ordering across conversations.
Back-of-the-envelope
Numbers first. Every component below has to be justified by one of these.
| Quantity | Value | Arithmetic |
|---|---|---|
| Message rate | ≈ 23k /s avg, 70k /s peak | 2B messages/day ÷ 86,400 s ≈ 23,150/s; peak 3× → ~70k/s. Each message is written once and delivered to every member: average 8 members per group → up to ~560k deliveries/s at peak. |
| Concurrent connections | 10M → ~100–200 connection servers | 20% of 50M DAU online at peak = 10M WebSockets. A tuned server holds ~100k idle connections at ~10 KB each (1 GB); plan 100 servers at capacity, 200 for headroom and rolling deploys. |
| Storage | ≈ 400 GB/day, 146 TB/yr | 2B × ~200 B (ids, seq, sender, body ~100 B, timestamps) = 400 GB/day. A wide-column store partitioned by conversation absorbs this; Postgres would need sharding within months. |
| Presence heartbeats | ≈ 330k writes/s | 10M connections × 1 heartbeat / 30 s = 333k SET presence:{user} EX 60/s. Redis does ~100k+ ops/s per node → a 4–6 node cluster just for presence; this is why presence gets its own store and a coarser update rate. |
| Session lookups | ≈ 560k /s peak | Every delivery needs session:{user_id} → server. Batch with MGET for groups (one round-trip per message, not per member) and cache hot sessions on the routing layer for a few seconds. |
| History reads | ≈ 12k /s | 50M DAU opening ~20 conversations/day = 1B reads/day ≈ 11,600/s, each a partition range scan of 50 rows — the natural shape of a (conv_id, seq DESC) clustering key. |
Interface
Endpoints, messages or events.
WS wss://chat.example.com/ws (auth: bearer token on upgrade)One connection per device. The server registers session:{user_id}:{device_id} → server_id on open and deletes it on close; a heartbeat frame every 30 s keeps both the TCP path and the registry entry alive.WS ↑ send { client_msg_id, conv_id, body } → ↓ ack { client_msg_id, msg_id, seq, ts }client_msg_id is a UUID generated on the device: a retry after a dropped connection reuses it and the server deduplicates (unique index on (conv_id, client_msg_id)). seq is the per-conversation order.WS ↓ message { msg_id, conv_id, seq, sender_id, body, ts }Server push to every online member device. The client acks with delivered { msg_id }; unacked messages are re-sent on reconnect from the client’s last_seq per conversation.WS ↑ read { conv_id, up_to_seq }Marks everything ≤ seq as read; naturally idempotent (a max). Fanned out to the sender as receipt { conv_id, user_id, read_seq }.GET /conversations/{id}/messages?before_seq=&limit=50 → { items[], next_before_seq }History paging by seq cursor, newest first. Also used on reconnect: after_seq=last_seq to fetch what was missed.POST /conversations { member_ids[] } → { conv_id }For 1:1, deterministic id from the sorted user pair so two clients creating "the same" chat simultaneously converge on one conversation.GET /presence?user_ids=… → { user_id: { online, last_seen } }Batched, read from Redis; clients subscribe to presence changes only for the contacts currently on screen.POST /devices { push_token, platform } → 204Registers APNs/FCM tokens for offline delivery; tokens are invalidated when the provider reports them dead.Build it one problem at a time
Each step names the problem first. Decide what you would add before revealing the reference answer.
1
Why not HTTP polling — persistent connections
Problem · 10M clients polling
GET /messages?since= every 2 s is 5M requests/s, almost all returning nothing, and a message still waits up to 2 s. Long polling halves the waste but keeps the reconnect churn. Real-time push needs a server-initiated channel.Work through every step to unlock the data model, the request walkthrough, scaling, failure modes and the open decisions.