DDIA Learn — Designing Data-Intensive Applications

A database holds current state; a stream holds the history of changes. CDC bridges them by tailing the replication log. Event sourcing inverts the model: the log is primary, and current state is a derived view rebuilt by replay.

In practice

Debezium reads PostgreSQL WAL changes and publishes to Kafka — powering search index updates, cache invalidation, and data warehouses without dual writes. The transactional outbox pattern writes events to an outbox table in the same PostgreSQL transaction as the business row, then a relay process publishes to Kafka. EventStoreDB and Marten (.NET) implement full event sourcing for audit-heavy domains.

Shopify at scale

Order rows and outbox events commit in the same PostgreSQL transaction — a relay publishes OrderPlaced to Kafka for search indexing and warehouse sync without risky dual writes.

typescript — Transactional outbox pattern

// Transactional outbox — Airbnb / Shopify event publishing
await prisma.$transaction(async (tx) => {
  const order = await tx.order.create({ data: orderInput });
  await tx.outboxEvent.create({
    data: {
      aggregateId: order.id,
      type: "OrderPlaced",
      payload: JSON.stringify(order),
    },
  });
});
// Separate relay process reads outbox → publishes to Kafka

Diagram

Databases and Streams