Upload, Transcode, and Serve

Write path

• Client → upload API → object storage (raw).
• Event → transcoder fleet → object storage (segments + manifests).
• Metadata service marks video READY and exposes playback URL.

Read path

• Client requests manifest from API (cached).
• Player fetches segments from CDN edge; miss goes to origin.
• View counts batched to stream processor (approximate is OK).

// Resumable upload → transcode queue (YouTube-style)
async function onUploadComplete(videoId: string, rawKey: string) {
  await db.videos.update({ id: videoId, status: "PROCESSING" });
  await kafka.publish("video.uploaded", {
    videoId,
    rawKey,
    renditions: ["360p", "720p", "1080p"],
  });
}
// Workers write HLS segments to object storage; API serves manifest URL

Step-by-step walkthrough

Write path — upload and transcode

• ① Upload chunks — Creator sends resumable multipart uploads; API never buffers the full file in RAM.
• ② PUT raw video — Upload API streams bytes directly to Object Storage (S3/GCS).
• ③ INSERT status=PROCESSING — Metadata DB row created so creators see upload received, not yet playable.
• ④ Publish event — video.uploaded event to Kafka; upload HTTP response returns immediately.
• ⑤ Transcode job — FFmpeg worker consumes the event from the queue.
• ⑥ Read raw / write HLS — Worker reads original, writes adaptive bitrate segments back to Object Storage.
• ⑦ UPDATE status=READY — Metadata row updated; manifest URL becomes valid for viewers.

Read path — metadata and playback

• ① GET /videos/:id — Viewer requests title, channel, and manifest URL from Metadata API.
• ② SELECT title, manifest URL — API reads Postgres; hot rows cached in Redis.
• ③ GET .m3u8 + .ts segments — Player fetches HLS manifest and video segments from CDN edge.
• Cache miss → origin — CDN pulls segment from Object Storage on miss; viral content stays at the edge.