Case 1
Back-of-the-Envelope Estimation
Rough numbers prevent impossible architectures. Estimate QPS, storage, bandwidth, and cache size before you draw shards.
You do not need exact numbers. Order-of-magnitude checks catch designs that cannot work: serving 4K video from a single Postgres row, or indexing every WhatsApp message in Elasticsearch in real time.
Worked example: short-video app
- 50M DAU, each watches 20 videos/day → 1B views/day ≈ 12k views/s average.
- Peak ≈ 5× average → ~60k read QPS for metadata; bytes served mostly from CDN.
- 100k uploads/day, 50 MB average raw → 5 TB/day ingest; keep hot in object storage.
- Metadata row ~2 KB × 500M videos → ~1 TB before indexes and replicas.
Step-by-step walkthrough
Write path — upload and transcode
- ① Upload chunks — Creator sends resumable multipart uploads; API never buffers the full file in RAM.
- ② PUT raw video — Upload API streams bytes directly to Object Storage (S3/GCS).
- ③ INSERT status=PROCESSING — Metadata DB row created so creators see upload received, not yet playable.
- ④ Publish event — video.uploaded event to Kafka; upload HTTP response returns immediately.
- ⑤ Transcode job — FFmpeg worker consumes the event from the queue.
- ⑥ Read raw / write HLS — Worker reads original, writes adaptive bitrate segments back to Object Storage.
- ⑦ UPDATE status=READY — Metadata row updated; manifest URL becomes valid for viewers.
Read path — metadata and playback
- ① GET /videos/:id — Viewer requests title, channel, and manifest URL from Metadata API.
- ② SELECT title, manifest URL — API reads Postgres; hot rows cached in Redis.
- ③ GET .m3u8 + .ts segments — Player fetches HLS manifest and video segments from CDN edge.
- Cache miss → origin — CDN pulls segment from Object Storage on miss; viral content stays at the edge.
estimationQPSstoragebandwidthCDN