Siddhant Deval
Siddhant Deval
backend18 min read

Event Sourcing with Kafka: Append-Only Truth, Snapshot Optimization & Schema Evolution

Event Sourcing stores the history of state changes — not current state — making the event log the single source of truth from which any projection can be derived. Master Kafka as the append-only event store, snapshot optimization against O(n) replay, and Avro schema evolution with the Confluent Schema Registry.

Event Sourcing with Kafka: Append-Only Truth, Snapshot Optimization & Schema Evolution

Every boundary is a failure isolation decision — and one of the most profound boundaries in distributed system design is the boundary between the current state you store and the history of changes you discard. Traditional systems store rows in a users table and update them in place. When a bug corrupts 10,000 user records, you write a fix-up script, apply it, and hope nothing was missed. When a business analyst asks "what was the order status at 3PM on Tuesday?", the answer is "we don't have that data." Event Sourcing inverts this model: you store the full history of state changes — not current state — and current state is derived by replaying that history. This means you can answer any temporal query, reconstruct any point-in-time snapshot, and deploy a bug-fix by simply correcting the projection logic and replaying from the beginning.

Architectural Note

Series positioning: This is Part 6 of the Distributed Architecture & System Design series. Building directly on the CQRS foundations established in Part 5: Separating Write Models from Read Projections, this article explores Event Sourcing, append-only domain event streams, snapshotting optimizations, and Avro schema evolution before examining projection engines in Part 7: Materialized Views & Event-Driven Projections.


1. The Event Store as Source of Truth

TYPESCRIPT
// ❌ Traditional: store current state — history is discarded
await db.query(`
  UPDATE orders SET status = 'shipped', shipped_at = NOW() WHERE id = $1
`, [orderId])
// The previous 'confirmed' status is gone. What changed it? When? Who?

// ✅ Event Sourcing: append domain events — current state is derived
await eventStore.append('order', orderId, [
  {
    type: 'OrderShipped',
    orderId,
    trackingNumber,
    carrier,
    shippedAt: new Date().toISOString(),
    version: currentVersion + 1,
  }
])
// Current state = replay of all events for this order
// History: OrderPlaced(v1) → PaymentCharged(v2) → OrderConfirmed(v3) → OrderShipped(v4)
// Any version reconstructable at any time
Traditional mutable database state with destructive overwriting updates (left) versus Event Sourcing append-only immutable event log where current state is derived by replaying history (right).
Traditional mutable database state with destructive overwriting updates (left) versus Event Sourcing append-only immutable event log where current state is d…

1.1 The Append-Only Event Log

Mental Model Check

In Event Sourcing, DELETE and UPDATE do not exist at the event level. Every business fact that happened is appended. To "undo" something, you append a compensating event: OrderCancelled after OrderPlaced. The history is preserved; the derived state reflects the compensation.


2. Implementing the Event Store with Kafka

2.1 Kafka as an Append-Only Event Store

TYPESCRIPT
// ✅ Event store backed by Kafka — one topic per aggregate type, key = aggregateId
const ORDER_EVENTS_TOPIC = 'order.event-store'

class KafkaEventStore {
  async append(aggregateId: string, events: DomainEvent[]): Promise<void> {
    await this.producer.send({
      topic: ORDER_EVENTS_TOPIC,
      messages: events.map(event => ({
        key: aggregateId,           // same aggregateId always routes to same partition
        value: JSON.stringify(event),
        headers: {
          'event-type': event.type,
          'aggregate-version': event.version.toString(),
          'occurred-at': event.occurredAt,
        }
      }))
    })
  }

  async loadEvents(aggregateId: string, fromVersion = 0): Promise<DomainEvent[]> {
    // Subscribe to the partition owning this aggregateId
    // Replay all events for this key from the beginning of the partition
    const consumer = this.kafka.consumer({ groupId: `load-${aggregateId}-${Date.now()}` })
    await consumer.subscribe({ topic: ORDER_EVENTS_TOPIC, fromBeginning: true })

    const events: DomainEvent[] = []
    await consumer.run({
      eachMessage: async ({ message }) => {
        if (message.key?.toString() === aggregateId) {
          const event = JSON.parse(message.value!.toString()) as DomainEvent
          if (event.version >= fromVersion) events.push(event)
        }
      }
    })
    return events.sort((a, b) => a.version - b.version)
  }
}

2.2 Aggregate Reconstruction

TYPESCRIPT
// ✅ Reconstruct current state by replaying all events for an aggregate
class OrderAggregate {
  static reconstruct(events: DomainEvent[]): OrderAggregate {
    const order = new OrderAggregate()
    for (const event of events) {
      order.apply(event) // each event mutates in-memory state
    }
    return order
  }

  private apply(event: DomainEvent): void {
    switch (event.type) {
      case 'OrderPlaced':
        this.orderId    = event.orderId
        this.userId     = event.userId
        this.lineItems  = event.lineItems
        this.totalCents = event.totalCents
        this.status     = 'pending'
        this.version    = event.version
        break
      case 'PaymentCharged':
        this.chargeId = event.chargeId
        this.status   = 'payment_confirmed'
        this.version  = event.version
        break
      case 'OrderShipped':
        this.trackingNumber = event.trackingNumber
        this.status         = 'shipped'
        this.version        = event.version
        break
      case 'OrderCancelled':
        this.status = 'cancelled'
        this.version = event.version
        break
    }
  }
}

3. Snapshot Optimization

As an aggregate accumulates events over time, O(n) replay becomes expensive. A snapshot captures current state at a version checkpoint, reducing reconstruction to O(snapshot_gap):

TYPESCRIPT
// ✅ Snapshot-based reconstruction — O(snapshot_gap) instead of O(all events)
class SnapshotStore {
  async saveSnapshot(aggregateId: string, state: OrderAggregate): Promise<void> {
    await db.query(`
      INSERT INTO order_snapshots (aggregate_id, version, state, created_at)
      VALUES ($1, $2, $3, NOW())
      ON CONFLICT (aggregate_id)
      DO UPDATE SET version = $2, state = $3, created_at = NOW()
    `, [aggregateId, state.version, JSON.stringify(state.toJSON())])
  }

  async loadSnapshot(aggregateId: string): Promise<{ state: OrderAggregate; version: number } | null> {
    const row = await db.query(
      `SELECT state, version FROM order_snapshots WHERE aggregate_id = $1`,
      [aggregateId]
    )
    if (!row.rows[0]) return null
    return {
      state: OrderAggregate.fromJSON(row.rows[0].state),
      version: row.rows[0].version
    }
  }
}

// ✅ Reconstruction with snapshot shortcut
async function loadOrder(aggregateId: string): Promise<OrderAggregate> {
  const snapshot = await snapshotStore.loadSnapshot(aggregateId)
  const fromVersion = snapshot?.version ?? 0

  const recentEvents = await eventStore.loadEvents(aggregateId, fromVersion + 1)

  if (snapshot && recentEvents.length === 0) {
    return snapshot.state // fully up-to-date snapshot — no replay needed
  }

  const base = snapshot?.state ?? new OrderAggregate()
  for (const event of recentEvents) {
    base.apply(event)
  }

  // Refresh snapshot every 50 events to keep replay gap bounded
  if (recentEvents.length >= 50) {
    await snapshotStore.saveSnapshot(aggregateId, base)
  }

  return base
}

3.1 Snapshot Strategy

Aggregate Event Rate Snapshot Threshold
Order (short lifecycle) Low (5–10 events total) Not required
User account (long-lived) Medium (grows over years) Every 100 events
Trading position (high-frequency) Very high (thousands/day) Every 500 events or 1 hour
Event sourcing snapshot optimization showing full replay bottleneck from event 0 avoided by loading aggregate state from snapshot at event 500 and replaying only tail events 501 to 504.
Event sourcing snapshot optimization showing full replay bottleneck from event 0 avoided by loading aggregate state from snapshot at event 500 and replaying…

4. Schema Evolution & the Confluent Schema Registry

Domain events are permanent. Unlike database rows that you can migrate in place, published events that are stored in Kafka will be read by consumers weeks or months later. Schema evolution requires explicit versioning from day one.

4.1 Avro Schema Definition

JSON
{
  "type": "record",
  "name": "OrderPlaced",
  "namespace": "com.company.orders.v1",
  "fields": [
    { "name": "orderId",     "type": "string" },
    { "name": "userId",      "type": "string" },
    { "name": "totalCents",  "type": "long" },
    { "name": "placedAt",    "type": "string" },
    { "name": "currency",    "type": "string", "default": "USD" }
  ]
}

4.2 Schema Compatibility Rules

TYPESCRIPT
// ✅ Backward-compatible evolution — add optional fields with defaults
// Old consumers (compiled against v1) ignore new fields → no breaking change
const OrderPlacedV2 = {
  type: 'record',
  name: 'OrderPlaced',
  namespace: 'com.company.orders.v1',
  fields: [
    { name: 'orderId',    type: 'string' },
    { name: 'userId',     type: 'string' },
    { name: 'totalCents', type: 'long' },
    { name: 'placedAt',   type: 'string' },
    { name: 'currency',   type: 'string', default: 'USD' },
    // NEW: optional field with default — backward compatible ✅
    { name: 'promoCode',  type: ['null', 'string'], default: null },
  ]
}

// ❌ Breaking changes — never do these in the same schema version
// - Removing an existing field (old events no longer deserialize)
// - Renaming a field (field number/name change)
// - Changing a field type (int → string)
Change Type Compatibility Action
Add optional field with default Backward + Forward ✅ Safe to publish immediately
Add required field (no default) Breaking ❌ New schema version required
Remove existing field Breaking ❌ New schema version + migration period
Rename field Breaking ❌ Add new field, deprecate old with alias

4.3 Schema Registry Integration

TYPESCRIPT
// ✅ Schema Registry enforces compatibility at publish time
const schemaRegistry = new SchemaRegistry({ host: 'http://schema-registry:8081' })

// Producer: schema validated and registered before first publish
const { id } = await schemaRegistry.register({
  type: SchemaType.AVRO,
  schema: JSON.stringify(OrderPlacedV2),
}, { subject: 'order.event-store-value' })

await producer.send({
  topic: 'order.event-store',
  messages: [{ key: orderId, value: await schemaRegistry.encode(id, event) }]
})

// Consumer: schema fetched from registry and used to decode
await consumer.run({
  eachMessage: async ({ message }) => {
    const event = await schemaRegistry.decode(message.value!)
    // event is correctly typed and decoded regardless of which schema version was used at publish
  }
})

5. Event Sourcing Anti-Patterns

5.1 Mutable Events

TYPESCRIPT
// ❌ Never modify or delete events — they are the source of truth
await kafkaAdmin.deleteRecords([{
  topic: 'order.event-store',
  partitions: [{ partition: 0, offset: '42' }] // ← NEVER do this in production
}])
// This corrupts the aggregate history and breaks all projections that read past that offset

// ✅ Correct: append a compensating event
await eventStore.append(orderId, [{
  type: 'DataCorrectionApplied',
  orderId,
  correctedField: 'totalCents',
  oldValue: 4999,
  newValue: 4499,
  reason: 'Pricing bug on 2026-09-01',
  correctedAt: new Date().toISOString(),
  version: currentVersion + 1,
}])

5.2 Anemic Events

TYPESCRIPT
// ❌ Anemic event — missing context, forces consumer to re-query state
{ type: 'OrderUpdated', orderId: 'abc123' } // what was updated? what did it change to?

// ✅ Self-contained event — all required context embedded
{
  type: 'OrderShipped',
  orderId: 'abc123',
  trackingNumber: 'UPS-123456',
  carrier: 'UPS',
  estimatedDelivery: '2026-09-08',
  shippedFrom: 'warehouse-us-east-1',
  shippedAt: '2026-09-05T14:30:00Z',
  version: 4
}

Summary

Architectural Concern Production Rule
Immutable Event Log The source of truth is the ordered event history; current state is a derived artifact rehydrated on demand.
Self-Contained Events Each event must carry all context required to reconstruct state; avoid forcing consumers to make re-queries.
Snapshot Optimization Periodic snapshots cap $O(n)$ replay to $O(\text{snapshot_gap})$; mandatory for long-lived aggregates.
Schema Evolution Add optional fields with defaults; enforce schema registry compatibility to prevent consumer deserialization failures.
Compensating Events State corrections are appended as new domain events; the historical log is never mutated or deleted.

What's Next

Now that we have explored event-sourced persistence and snapshot strategies, Part 7: Materialized Views & Event-Driven Projections examines how to maintain denormalized projection stores, execute blue-green rebuilds without downtime, and monitor projection lag.

Research & Synthesis Note

This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.

#Event Sourcing#Kafka#CQRS#Distributed Systems#Backend
Siddhant Deval

Written by Siddhant Deval

Senior Full-Stack Engineer building high-scale architectures, browser performance engineering systems, and SaaS platforms.