Introducing Data Quality Testing at a Global Energy Company

Incomplete specification: A complete specification for the data products was not available at the initial stage of the project, so the requirements were developed through ongoing consultations. United Consult responded with a flexible team structure: at different stages of the project, we expanded the team's competencies, such as business analysts, as needed, to support the clarification of requirements.

Constantly changing business needs: Business needs changed regularly, which required frequent configuration and deployment changes. The flexible testing framework used throughout the project allowed for rapid adaptation, but constant changes increased the risks of the project, which we managed with continuous monitoring and adaptive solutions.

Integrating testing into the development process: It was an important expectation that data quality testing should be closely aligned with the development and release processes of data products. Through continuous customer engagement, iterative development and continuous fine-tuning, the United Consult team has developed a testing process that can be fully integrated into the existing development environment.

Unburdening the data engineering team: One of the most important results of the project was that the customer's data engineering team was able to free up significant resources that could be devoted to actual development tasks. This significantly accelerated the progress of the projects.

Early identification of data errors: The solution introduced enabled the identification of critical data errors at field, data source and supplier level. As a result, errors were often corrected in the source systems, ensuring data quality at the beginning of the processing process.