Designing Data-Intensive Applications
Case 2

Caching, Popularity, and Cost

Virality shifts traffic from long-tail to hot objects. Design caching and storage tiers for both.

A video with 10 views and one with 10M views share the same architecture but wildly different cache hit ratios. Popularity-aware prefetch and tiered storage prevent paying CDN prices for cold content.

In practice

Netflix Open Connect caches hot titles on ISP appliances. YouTube's long tail lives in cheaper storage classes; only trending manifests get aggressive edge warming.

Google at scale

Search ranking and YouTube recommendations share DNA: offline batch features + online low-latency scoring. The serving path never scans raw watch history per request.

typescript — Singleflight for trending metadata
// Prevent thundering herd on trending metadata
const inflight = new Map<string, Promise<TrendingPage>>();

async function getTrending(region: string): Promise<TrendingPage> {
  const cached = await redis.get(`trending:${region}`);
  if (cached) return JSON.parse(cached);
  if (!inflight.has(region)) {
    inflight.set(region, loadTrendingFromDb(region).finally(() => inflight.delete(region)));
  }
  return inflight.get(region)!;
}
Key Takeaways
  • Long-tail videos: CDN still helps, but most bytes sit cold in cheap storage.
  • Viral videos: push to more edge POPs; protect origin with shield tiers.
  • Popular metadata (trending page) needs application-level cache + singleflight.
  • Recommendation features precomputed offline; online path does light reranking.
  • Cost control: transcode once, store lifecycle policies for unwatched uploads.
  • Multi-region metadata replicas with sticky routing for editors.
CDNcachelong tailrecommendationscostQPSbandwidth