Designing Data-Intensive Applications
Case 1

Back-of-the-Envelope Estimation

Rough numbers prevent impossible architectures. Estimate QPS, storage, bandwidth, and cache size before you draw shards.

You do not need exact numbers. Order-of-magnitude checks catch designs that cannot work: serving 4K video from a single Postgres row, or indexing every WhatsApp message in Elasticsearch in real time.

Worked example: short-video app

  • 50M DAU, each watches 20 videos/day → 1B views/day ≈ 12k views/s average.
  • Peak ≈ 5× average → ~60k read QPS for metadata; bytes served mostly from CDN.
  • 100k uploads/day, 50 MB average raw → 5 TB/day ingest; keep hot in object storage.
  • Metadata row ~2 KB × 500M videos → ~1 TB before indexes and replicas.
In practice

YouTube and TikTok napkin math drove separate upload paths (write-heavy) from feed reads (CDN-heavy). WhatsApp optimized for small text payloads and connection fan-in, not storage per message.

typescript — Peak QPS from DAU
// Napkin math: DAU → peak QPS
const DAU = 50_000_000;
const actionsPerUserPerDay = 20;
const avgQps = (DAU * actionsPerUserPerDay) / 86_400;
const peakQps = avgQps * 5; // burst factor for prime time
console.log({ avgQps: Math.round(avgQps), peakQps: Math.round(peakQps) });
Diagram

Step-by-step walkthrough

Write path — upload and transcode

  • ① Upload chunks — Creator sends resumable multipart uploads; API never buffers the full file in RAM.
  • ② PUT raw video — Upload API streams bytes directly to Object Storage (S3/GCS).
  • ③ INSERT status=PROCESSING — Metadata DB row created so creators see upload received, not yet playable.
  • ④ Publish event — video.uploaded event to Kafka; upload HTTP response returns immediately.
  • ⑤ Transcode job — FFmpeg worker consumes the event from the queue.
  • ⑥ Read raw / write HLS — Worker reads original, writes adaptive bitrate segments back to Object Storage.
  • ⑦ UPDATE status=READY — Metadata row updated; manifest URL becomes valid for viewers.

Read path — metadata and playback

  • ① GET /videos/:id — Viewer requests title, channel, and manifest URL from Metadata API.
  • ② SELECT title, manifest URL — API reads Postgres; hot rows cached in Redis.
  • ③ GET .m3u8 + .ts segments — Player fetches HLS manifest and video segments from CDN edge.
  • Cache miss → origin — CDN pulls segment from Object Storage on miss; viral content stays at the edge.
Key Takeaways
  • Convert daily active users into peak requests per second with a burst factor.
  • Storage = records × average object size × replication factor × retention.
  • Video and images dominate bandwidth — metadata is usually smaller.
  • 1 day of write volume often fits in RAM; plan for 30–90 days on disk.
  • Estimates justify CDN, batch transcoding, and read replicas early.
  • Google, Meta, and AWS whitepapers all start with napkin math.
estimationQPSstoragebandwidthCDN