Real-time data pipeline serving 40M events/day
Replaced a brittle ETL setup with a lineage-tracked warehouse that finance and product trust.
To maintain client confidentiality, the company and industry in this case study have been anonymized. The underlying solution is the same.
The problem
The client had a fragile ETL pipeline that no one fully understood anymore. Jobs would fail silently, data lineage was undocumented, and the finance team had stopped trusting the numbers. Product decisions were being made on stale or incorrect data.
What we built
We replaced the legacy ETL with a streaming pipeline built for observability from day one. Every transformation is lineage-tracked. Schema changes are versioned. Failures surface immediately with enough context to diagnose and fix without a war room.
The warehouse now serves 40M events per day with consistent sub-second query times on the reporting tables finance and product rely on.
Results
99.9% uptime SLA maintained since launch. Query performance improved 3× on the core reporting views. More importantly: the finance team trusts the numbers again, which means product decisions are no longer blocked on “can we verify this?”