Saltar a contenido

ADR-005: Stream processing con command sourcing

  • Status: Accepted
  • Date: 2026-05-14
  • Tags: core, architecture, pattern

Context and Problem Statement

¿Cómo modelar el state del engine y sus transformaciones? Las opciones tradicionales son CRUD-centric (mutar tablas directamente) vs event-sourcing (state derivado de events). Camunda eligió command sourcing — variante donde commands también se persisten al log, no solo events.

Decision Drivers

  • Replay determinism es requirement (testing, recovery, debugging)
  • Single source of truth para audit/compliance
  • Failover requires reconstrucción de state desde log
  • Patterns proven en Camunda 8 production a 22M instances/15h

Considered Options

  1. Command sourcing (commands + events al log, state derivado)
  2. Event sourcing puro (solo events persisten)
  3. CRUD tradicional (UPDATE/INSERT directos)
  4. Hybrid CRUD + event stream (state CRUD + event log para audit)

Decision Outcome

Chosen option: Command sourcing porque: - Determinism: cualquier follower puede re-derivar events desde commands - Failover safety: commands en log antes de procesar elimina ambiguity - Validated empíricamente por Camunda (18-26x mejora vs C7 según Intuit) - Replay testeable (ver ADR-019)

Positive Consequences

  • State reconstructible from log
  • Failover sin ambiguity (qué events generó el leader fallido)
  • Audit trail completo
  • Replay determinism testeable
  • Backup-restore vía log replay
  • Time-travel debugging posible

Negative Consequences

  • Log volume mayor (commands + events vs solo events)
  • Commands rechazados también en log (COMMAND_REJECTION)
  • Storage overhead permanente
  • Complexity vs CRUD (más conceptos)

Pros and Cons of the Options

Command sourcing

Pros: - Replay determinism garantizable - Failover sin ambiguity - Audit completo - State reconstructible - Time-travel debugging - Pattern probado at-scale (Camunda 8)

Cons: - Log volume mayor - Commands rechazados también persisten - Complexity inherente del pattern

Event sourcing puro

Pros: - Log volume menor (solo events) - Simpler conceptualmente

Cons: - Failover ambiguity (leader genera events; si falla mid-processing, ¿qué events ya generó?) - No audit de commands rechazados - Recovery más complejo en partial failures

CRUD tradicional

Pros: - Simple, familiar - Performance directa - Querying trivial

Cons: - Sin audit trail (a menos que se agregue separado) - Sin replay determinism - Failover requires sync state snapshots - No time-travel debugging - Compliance harder

Hybrid

Pros: - Best of both worlds

Cons: - Dos sources of truth (state CRUD vs event log) - Sync issues inevitable - Bug magnet (qué dice CRUD vs qué dice log)

Implementación en el MVP

-- Command log
CREATE TABLE command_log (
    position BIGSERIAL PRIMARY KEY,
    asqn BIGINT,                    -- application sequence number
    tenant_id TEXT NOT NULL,
    intent TEXT NOT NULL,
    payload JSONB NOT NULL,
    timestamp TIMESTAMPTZ DEFAULT NOW(),
    processed_at TIMESTAMPTZ,
    rejection_type TEXT,            -- NULL si succeeded
    rejection_reason TEXT
);

-- Event log
CREATE TABLE event_log (
    position BIGSERIAL PRIMARY KEY,
    command_position BIGINT REFERENCES command_log(position),
    tenant_id TEXT NOT NULL,
    intent TEXT NOT NULL,
    payload JSONB NOT NULL,
    timestamp TIMESTAMPTZ DEFAULT NOW()
);

-- State tables (derived from events)
CREATE TABLE process_instances (...);
CREATE TABLE element_instances (...);
CREATE TABLE variables (...);
-- etc.

Flow: 

1. Client → Engine: CreateInstanceCommand
2. Engine: INSERT INTO command_log VALUES (...)
3. Engine: process command → generate events
4. Engine: BEGIN TRANSACTION
     INSERT INTO event_log VALUES (...)
     INSERT/UPDATE state tables
   COMMIT
5. Engine: UPDATE command_log SET processed_at = NOW()
6. Engine: respond to client

Atomicity: state mutations + event_log inserts en una transaction. Si transaction falla, todo rolled back.