RFC 014 · for the Platform director

Split ingestion before the next halt

Four of six incidents ended as a full stop. One queue still has no failover.

Platform · presented 2026-08-12 · data as of the same day

This hour ends in a decision

  1. 01 What failed
  2. 02 Why split, not enlarge
  3. 03 The ask

Wording matches the dividers. A decision is needed by 2026-09-01.

Four of the last six incidents ended the same way.

Queue fill → gateway block → every producer stops.

01

What failed

The last halt lasted 1 hour 52 minutes

Incidents that traced to the single ingestion queue
Date (UTC) Detected Recovered Duration
2026-03-1409:4011:051h 25m
2026-04-0216:1217:481h 36m
2026-06-1902:0803:411h 33m
2026-07-3014:1216:041h 52m

Four of six incidents in 2026. Detection still waits on 120s of lag.

A halt takes down 41% of the footprint

41%
Of included units behind one queue
1.84M of 2.50M · as of 2026-08-12
1h 52m
Last full stop
detect 14:12 → recover 16:04 UTC
~1 / qtr
If nothing changes
four incidents, two quarters
Ingestion share of footprint 410 of 1000 units of track, 41 percent, marked as the focal segment.
IngestionRest of platform

Same snapshot as the inventory report · included population only

Ingestion holds 41% of the footprint

Ingestion holds 41% of the footprint Five functions ranked by current size. Ingestion holds 41 percent of the 1.8 million unit total, more than the next three combined.IngestionTransformServingArchiveOther0510152025303540Share of included total (%) →

Denominator: 1,842,110 included units · as of 2026-08-12

Every producer shares a path with no failover

Current ingestion path Checkout, Catalog, and Inventory call the gateway, which writes one queue with no failover, then a consumer, then the warehouse. The queue is the focal failure. Checkout Catalog Inventory Gateway POST /events Queue ingest no failover Consumer Warehouse

Current state. Dispatcher and ingest-a/b are proposed, not drawn.

02

Why split, not enlarge

A larger queue changes when it fails, not whether.

Capacity is not a second path.

Split is the only option that removes the halt

Do nothing, a larger queue, and splitting the path compared on the same four criteria
Criterion Do nothing Larger queue Split the path
On failure Full stop Full stop Degrades
Failure mode Kept Kept Removed
Producer API Unchanged Unchanged Unchanged
This quarter No extra work Cheaper Two engineers

Accent marks the recommendation.

Two queues, one stateless dispatcher

Proposed ingestion path Gateway to a stateless dispatcher that writes ingest-a or ingest-b by event-id hash, each with its own consumer, then the warehouse. Gateway Dispatcher stateless ingest-a ingest-b Consumer A Consumer B Warehouse even hash odd hash

Partition on event ID — ADR 009. Producer contract at /events does not change.

The quarter costs two engineers and a slipped backfill

2
Engineers
Platform · twelve weeks
12
Weeks
Decision needed by 2026-09-01
1q
Catalog backfill slips
The opportunity cost of the quarter

Ongoing: one extra queue and consumer group.

03 · the ask

Approve the split by 1 September

  1. Approve RFC 014 — not a larger single queue.
  2. Owner: Platform. Decision needed by 2026-09-01.
  3. No decision is do nothing. Record it as one.

Detail lives in RFC 014 and the 2026-07-30 postmortem. This deck is presented, not sent.