The challenge
A national coordination body in the waste-management sector had no unified, reliable data platform for internal and government-mandated reporting. Data was scattered across a legacy billing system, Excel files and Access databases, duplicated and inconsistent, and slow manual reporting could not meet regulatory and operational demands at national scale.
At a glance
StackHPE Ezmeral, Kafka Connect, Spark, PostgreSQL, Superset, Zeppelin
SourceDB2 billing system
Duration1 year
What we did
SWICON delivered a multi-layer data lake on HPE Ezmeral Data Fabric: delta extraction from the DB2 billing system via Kafka Connect into a Stage layer, data-quality assessment and cleansing into a normalised Store layer, and reporting-optimized datamarts. The reporting layer was deliberately built on open-source tooling (Apache Superset, Zeppelin) to minimize license cost and maximize transparency. A dedicated data-quality datamart tracks source-system errors.
The result
Five mandatory regulatory reports and five business reports went live on the new platform after UAT and user training, through a phased Dev/Test/Prod rollout with a full documentation package. Historical data for the last full calendar year and the open year was loaded.
