Designing Data-Intensive Applications
Ch. 3

Column-Oriented Storage

Column stores excel at analytics by reading only the columns a query needs.

In a row store, all fields of a record sit together — great for fetching a whole user profile. In a column store, all values for a given field sit together — great for SUM(revenue) across millions of rows.

In practice

Snowflake and ClickHouse store columns separately for analytics queries that touch few fields across billions of rows. Apache Parquet files on AWS S3 use columnar layout for cheap cold storage. PostgreSQL remains row-oriented — which is why you offload analytics to a warehouse instead of scanning production tables.

Google at scale

Google's web index and ad analytics read only the columns each query needs — URL, rank, bid price — across petabytes. Columnar Parquet on GCS and BigQuery column stores are the production descendants of this layout.

typescript — Column-pruned warehouse query
// Google BigQuery / Snowflake — declarative OLAP over columnar storage
const revenueByRegion = await snowflake.execute(`
  SELECT d.region, SUM(f.revenue_cents) AS total
  FROM fact_orders f
  JOIN dim_date d ON f.order_date = d.date_key
  WHERE d.year = 2025
  GROUP BY d.region
  ORDER BY total DESC
`);
// Optimizer reads only region + revenue columns — not full rows
Diagram
  • Read only columns referenced in SELECT — less I/O.
  • Run-length and dictionary encoding compress repetitive column data.
  • Vectorized execution processes columns in SIMD-friendly batches.
Key Takeaways
  • Row-oriented storage reads entire rows even when you need one column.
  • Column-oriented storage stores each column contiguously on disk.
  • Column compression is far more effective due to similar values in a column.
  • Sort order in column storage enables efficient range scans and merges.
  • Materialized views and data cubes pre-aggregate common queries.
  • Snowflake, ClickHouse, and Apache Parquet on S3 are column-oriented in production.
column storeSnowflakeClickHouseParquetcolumn compressionmaterialized view