A photonics manufacturer wanted to speed up the delivering reports and dashboards to their business users, after migrating to a cloud database provider and a SaaS data warehouse automation tool.
About the Client
Problem Statement
After modernizing both the database platform and the automation tool, the manufacturer had trouble leveraging the advantages of the new platforms. An existing Data Vault 2.0 implementation did not strictly follow the standards, which led to a few problems:
- Repeatability and transparency of implementation was lacking
- Time to deliver new business reports was too long
The Challenge
A major challenge was a lack of common understanding of the Data Vault 2.0 standard, especially regarding implementation. The Data Warehouse Automation was based on loading patterns provided by another consulting company. Because these were not completely working out of the box, they needed to be changed over time. After many adjustments, the team ended up with a poorly documented, mainly self-built solution. This led to confusion about using and configuring the code.
The Solution
Our suggested solution included using our publicly available dbt package “datavault4dbt” to have a well-documented and highly-configurable set of Data Vault loading patterns. This suggestion is supported by the following actions:
- Successful implementation of datavault4dbt in a selection of examples during the workshop
- Comparing loading performance of old and new solution
- Explored documentation and user-interface of suggested solution
- Recommendations for naming conventions and sprint organization
- Optimization of the deployment process
Tangible Results for the Client
The defined standards minimized the room for interpretation when team members worked on new development tasks. Therefore, tangible results can be delivered to business users faster.
The usage of datavault4dbt reduced time for maintaining and extending the loading patterns, allowing the team to focus on delivering business value.
The usage of datavault4dbt reduced time for maintaining and extending the loading patterns, allowing the team to focus on delivering business value.
Technologies involved
- Snowflake
- Dbt Cloud
- Datavault4dbt
