← Story

Migration plan · Cutover 2026-11-02

Cut over one partition at a time and keep the original queue recoverable

Readiness closes on 2026-10-28; the old queue remains available through handoff on 2026-11-20.

Cutover clock

  1. Dual-write in staging

    Shadow-read for one week and reconcile event IDs.

  2. Route to ingest-a

    Verify acceptance and warehouse rows.

  3. Route to ingest-b

    Repeat verification independently.

  4. Retire eligibility

    Keep ingest until hypercare closes.

RFC 014 change view: one queue becomes two Today, producers reach the warehouse through the gateway, the single ingest queue, and one consumer group. RFC 014 adds a stateless dispatcher after the gateway that hashes the event ID to one of two queues, ingest-a and ingest-b, each with its own consumer group. The accent marks the dispatcher. A change ledger lists each change by node id: dispatcher added, ingest-a and ingest-b added, consumer group changed, ingest removed once the shadow week closes; producers, gateway and warehouse are unchanged. TODAY removed RFC 014 · PROPOSED added added changed Producers Gateway Queue ingest Consumers 3 replicas Warehouse Producers Gateway Dispatcher by event ID Queue ingest-a Queue ingest-b Consumers group a Consumers group b Warehouse CHANGE LEDGER dispatcher added Stateless; hashes the event ID to exactly one queue. ingest-a, ingest-b added Independently recoverable; 24-hour retention unchanged. consumer group changed One group per queue; each scales and recovers alone. ingest removed Stays recoverable until the shadow week closes. Unchanged: producer API, authentication, payload, warehouse schema, batch path.
What cuts over: the dispatcher and two queues replace the single ingest path; gates 1 and 2 move one queue each.

Rollback trigger: data mismatch, stalled lag, or any crash loop. Route back to ingest; never delete a queue.

Validation gates

Data

No duplicate warehouse row.

Latency

POST /events p99 stays below 200 ms.

Recovery

Lag falls for ten minutes.