Functional
- Send messages to one-to-one and group conversations.
- Synchronize history across a user's devices.
- Expose delivery/read state and best-effort presence.
End-to-end walkthrough
Chat System
Preserve conversation order and deliver messages across connected and offline devices.
Frame before solving
Requirements & boundaries
Quality attributes
- Online delivery p99 below 300 ms.
- Messages are durable before acknowledgement.
- Ordering is deterministic within a conversation.
Explicitly out of scope
- End-to-end encryption key management.
- Voice/video media transport.
Size the important constraints
Back-of-the-envelope estimates
Connections require a dedicated gateway tier.
Partition conversation logs and batch writes.
Use different strategies for ordinary and huge groups.
Hot recent history and archival storage need separate tiers.
These are reference assumptions, not universal facts. In an interview or architecture review, change them when the product context changes.
Define the contract
API & data model
Core operations
/v1/conversations/{id}/messagesSend with client_message_id for retry-safe submission./v1/conversations/{id}/messages?after=Page ordered history after a sequence./v1/connectReceive messages, receipts, and presence updates.Authoritative records
conversation_id+sequence PK, message_id, sender, body_ref, created_atConversation sequence defines visible order.user+device+conversation, delivered_through, read_throughCursors compress receipt state.Trace the critical path
Architecture & request flow
- 1Authenticate connection→
- 2Assign conversation sequence→
- 3Persist message→
- 4Fan out to online devices→
- 5Sync offline devices
Connection gateways
Maintain long-lived client sessions
Conversation service
Order and persist messages
Presence service
Track ephemeral device reachability
Fanout workers
Deliver and retry device updates
Reason about the hard parts
Critical design deep dives
Ordering and acknowledgement
Route a conversation to one logical sequencer or shard leader. Assign sequence, replicate the message, then acknowledge. Client-generated IDs deduplicate retries after ambiguous timeouts.
Fanout strategy
Fanout-on-write works for ordinary conversations; enormous groups store once and let recipients pull from the shared log. Hybrid thresholds avoid millions of per-user writes for celebrity groups.
Connection recovery
Gateways are disposable. On reconnect, a device presents durable cursors, fetches missing history, then resumes live delivery. Presence may be stale; history cannot depend on it.
Make trade-offs explicit
Architecture decisions
ChoiceWhyCost
Per-conversation orderingMatches the user-visible invariantHot group chats concentrate work
Store then fan outMakes history authoritativeDelivery may lag persistence
Failure-first review
What happens if…?
Gateway disconnects
Resume from the last acknowledged conversation sequence.
A celebrity group becomes hot
Shard fanout, batch membership reads, and degrade presence first.
Avoid premature complexity
How the design evolves
Small product
WebSocket gateways + relational message store
Move here when: Moderate connections and traffic.
Conversation sharding
Partitioned logs, cursor store, async fanout
Move here when: Write and connection scale exceed one cluster.
Global chat
Home-region conversations, edge gateways, archive tier
Move here when: Cross-region latency and residency requirements.
Test the reasoning
Interview follow-ups
What does 'sent' mean in this design?
Strong answer signal: Differentiate accepted, durably stored, delivered, and read.
How do you move a hot conversation?
Strong answer signal: Transfer sequence authority with fencing and a brief write pause/buffer.
Why not rely on timestamps for ordering?
Strong answer signal: Clock skew and concurrent sends require logical sequencing.
Build from primitives