All case studies
Data Engineering£85KUtilities

Anglian Water Services

Enterprise Data Platform

0Products Controlled
Timeline: 9 monthsTeam: 3 engineers

Summary

Built a comprehensive enterprise data platform using Microsoft Fabric and Databricks, controlling 250+ data products with automated ETL/ELT architecture and Delta-Parquet format in OneLake.

The Challenge

Anglian Water needed a unified data platform to manage data from hundreds of operational systems across eastern England. Legacy systems were siloed, reporting was manual, and data quality was inconsistent across 250+ data products serving different business units.

Our Solution

Designed and implemented a Microsoft Fabric-based data platform with Databricks as the compute engine. Implemented central storage in OneLake using Delta-Parquet format for both raw and transformed data. Built Curated and Gold layers using a customizable Microsoft framework with automated ETL/ELT architecture.

Architecture Flow

Landing Zone (Bronze)

Raw data ingestion250+ source systemsDelta-Parquet formatOneLake storage

Curated Layer (Silver)

Data cleansingSchema validationBusiness rulesDatabricks notebooks

Serving Layer (Gold)

Business aggregationsPower BI datasetsAPI endpointsStar schema models

Our Process

1

Data Audit & Discovery

Mapped 250+ data sources across operational systems, identified quality gaps, and defined target medallion architecture.

Microsoft PurviewAzure Data Catalog
2

OneLake Foundation

Implemented central storage in OneLake with Delta-Parquet format. Designed Landing, Curated, and Gold layer structure.

OneLakeDelta-ParquetMicrosoft Fabric
3

Pipeline Automation

Built automated ETL/ELT pipelines with Azure Data Factory and Databricks notebooks. Framework approach for 250+ products.

DatabricksPySparkAzure Data Factory
4

Gold Layer & BI

Built Curated and Gold layers using customizable Microsoft framework. Delivered Power BI dashboards integrated with Fabric.

Power BIDAXSSAS Tabular
5

DevOps & Governance

CI/CD pipelines with Azure DevOps. Automated testing, data lineage tracking, and governance with Microsoft Purview.

Azure DevOpsCI/CD YAMLMicrosoft Purview

Results & Impact

0
Products Managed

All data products controlled through a single framework

0
Data Accuracy

Automated validation at every pipeline stage

0
Faster Reporting

From days to hours for complex operational reports

0
Cost Reduction

Optimized compute and storage architecture

Technology Stack

Microsoft FabricDatabricksDelta LakePySparkAzure Data FactoryPower BIAzure DevOps CI/CDOneLakeDelta-Parquet

CODES AI delivered an exceptional data platform that has transformed how we manage our operations. Their expertise in data engineering is truly world-class.

James Whitfield

IT Director, Anglian Water Services

Start a similar project

Let's discuss how we can deliver the same results for your business. Free consultation, no commitment.

More Case Studies