Finance Intelligence Data - Building scalable, reliable data pipelines

turned-on black iPad

86%

Pipeline success rate

62%

Reduction in manual reprocessing

2%

Data error rate after validation

Client
A leading media and telecom company, offering television, broadband, mobile, and streaming services to millions of customers across the UK and Europe.
Industry
Telecom
Company Size
10,000+
Location
London, United Kingdom
Project Duration
28 months (Aug 2023 - Dec 2025)
Framework
  • Airflow
  • BigQuery
Visit Website

Designed and implemented a large scale data orchestration and validation framework for a telecom data platform, ensuring reliable delivery of highquality datasets to analytics and downstream systems. This project replaced fragmented legacy processes with an automated solution, and enhanced trust in the data.

The client’s data environment was complex, with separate teams maintaining their own definitions, calculations, and reporting logic. This resulted in inconsistent metrics, duplicate transformation efforts, and frequent disputes over “which numbers were correct.”

I led a cross-functional project to design and deliver a unified data model that aligned on business logic across analytics, finance, marketing, and operations teams. This model, built in BigQuery and orchestrated with Airflow and dbt, served as the gold layer, the single source of truth for all critical reporting and analytics.

To ensure accuracy and trust, we implemented a robust data validation framework that ran automated checks at every stage of the pipeline. Using SQL tests, Datafold, and GCP-native monitoring, the system detected anomalies, schema changes, and logic mismatches before they could reach production.

A critical part of the work was stakeholder collaboration, holding workshops with each department to validate the unified business logic, fact-check the transformed data against source systems, and gain full buy-in on the definitions.