The client, a leading technology company working in electrification, automation, and digitalization, had a Data Vault prototype already in place and needed recommendations to migrate it to the cloud.
About the Client
Problem Statement
Seeking expert advice, our client encountered problems in collaboration approaches, technical setup, data quality, governance, change management, and methodology within their Data Vault initiative, entailing a best practice guide for improvement and efficiency to use for further implementations. Furthermore, input was needed on how to move the prototype to the cloud.
The Challenge
Some of the challenges the customer faced included:
- Team members limited availability and junior staff seeking Data Vault 2.0 proficiency
- Data governance issues
- No quality guidelines
- Lack of automation in the project
- Use case driven approach (costly and less effective than a centralized EDW) with lack of cross-team collaboration
The Solution
The main deliverable was a report to guide the client in further Data Vault initiatives.
Key recommendations from Scalefree encompassed:
- Implementation of a persistent staging area
- Usage of TurboVault4dbt to integrate the organizational metadata with the enterprise data warehouse and allow for automation
- Definition of clear data security, data privacy, and data quality strategies
- Implementation of a shared raw data vault to allow for data sharing and eased collaboration across teams
- Simplification of data architecture
Tangible Results for the Client
The review accelerated and supported solving important problems the client faced, mainly by:
- Providing a best practices guide
- Guiding the client through the migration process from dbt core to dbt cloud
- Responding to questions on the Data Vault 2.0 approach
- Supporting the company through consulting services following the review
Technologies involved
- Oracle data sink
- Azure
- Amazon S3
- Snowflake
- Dbt core
