Engineering

Signal over noise: rebuilding our event pipeline

We rewrote ingestion around one idea — a product event should arrive already explained.

Priya Raman

Head of Engineering

7 min read

Our first ingestion pipeline was honest about one thing: it moved rows. It did not know what any of them meant, so every question started with a join and ended with a guess.

Events should arrive explained

We moved identity resolution, schema validation and enrichment to the edge of the pipeline. By the time an event lands, it already knows who caused it, which release it belongs to, and which flow it interrupted.

What changed in practice

Queries that used to take a day of careful SQL now resolve in seconds, and the answers survive contact with a second person asking the same question.

Create a free website with Framer, the website builder loved by startups, designers and agencies.