An interactive study companion for Martin Kleppmann's classic on reliable, scalable, and maintainable systems.
Partitioning splits a dataset across nodes so each machine handles a manageable subset.
A single machine cannot hold all data forever. Partitioning (sharding) divides the key space so each node owns a subset. The partition scheme determines load balance and query efficiency.
Kafka topics split into partitions for parallelism — each partition is an ordered log. MongoDB sharding routes documents by shard key through mongos routers. DynamoDB partitions by primary key hash; a hot partition (e.g., celebrity user ID) can throttle the whole table until you add a random suffix.
Trip events partition by ride_id so all state transitions for one trip land on the same Kafka partition — preserving order. Driver location updates hash by geo-cell to spread load across shards.
// Kafka / DynamoDB-style partition routing
function partitionForKey(userId: string, partitionCount: number): number {
let hash = 0;
for (const ch of userId) hash = (hash * 31 + ch.charCodeAt(0)) >>> 0;
return hash % partitionCount;
}
// WhatsApp routes messages by chat_id to the same shard for ordering
const partition = partitionForKey(chatId, 128);