Designing Data-Intensive Applications
Case 3

Delivery, Ordering, and Groups

Per-chat ordering, presence, and group fan-out define the server-side data model.

Group fan-out strategies

  • Small groups (< 256): push to every member inbox on send.
  • Large groups: store single copy; members pull on chat open.
  • Typing indicators: gossip or pub/sub channel, lossy is acceptable.
typescript — Per-chat sequence and ack
// Per-chat ordering with client ack
type ChatMessage = { id: string; chatId: string; seq: number; ciphertext: Buffer };

async function onClientAck(chatId: string, upToSeq: number) {
  await db.messages.deleteQueued({ chatId, seq: { lte: upToSeq } });
}
// Server assigns monotonic seq per chat; client dedupes by message id
Diagram

Step-by-step walkthrough

Send path — persist message

  • ① WebSocket SEND — Sender posts ciphertext over an open connection; payload stays small (no JSON bloat).
  • ② Append seq=N — Chat server assigns a monotonic sequence number and appends to the replicated message store.
  • ③ Update presence — Sender marked online in Redis with TTL; typing indicators use the same channel.

Delivery path — online or offline

  • ④a Online: WebSocket PUSH — If receiver is connected, server pushes the message on their socket immediately.
  • ④b Offline: notify — If receiver is disconnected, server triggers APNs/FCM instead of holding sockets open.
  • ⑤ Push alert — Mobile OS shows notification; message ciphertext waits in the durable store.
  • ⑥ ACK seq — Receiver acknowledges; server trims delivered messages from the hot queue.
Key Takeaways
  • Each chat has a monotonic server or hybrid logical clock for ordering.
  • Acks from clients let servers trim durable queues.
  • Group messages: write once, fan out to member inboxes or connection maps.
  • Large groups may use partial fan-out + pull on open (hybrid like Meta feeds).
  • Presence (online/typing) is ephemeral — Redis with TTL, not Postgres.
  • Multi-device sync shares the same message log keyed by account.
fan-outsequencegroupspresenceRedisQPSconnections