The client is a leading company in insurance and asset management, committed to securing lives and fostering global growth for individuals as well as businesses with excellence and integrity.
About the Client
Problem Statement
The client sought an efficient, effective solution for storing and managing their Data, specifically transactional data to better project the cash flow while balancing long-term storage needs with compliance requirements, including IFRS standards and specific retention policies.
The client faced the following problems:
The client faced the following problems:
- Insufficient proficiency in Data Vault 2.0 methodologies resulted in lack of quality of the Data Model
- Data lineage was fragmented, lacking a complete end-to-end traceability
- No GDPR compliance implementation
The Challenge
The challenge was ensuring consistency and accuracy across datasets, as well as harmonizing data from diverse source systems posed significant hurdles. Moreover, the client faced difficulties in efficiently accommodating and managing various source file formats, underscoring the need for a more streamlined approach to data handling and processing.
Therefore, the main challenges revolve around:
Therefore, the main challenges revolve around:
- Data reconciliation
- Data integration from various different source systems
- Different file format management
- Integration of manual bookings
The Solution
After conducting a thorough review and analysis of the current system, Scalefree consultants proposed a new target architecture to the client that enhanced efficiency and streamline processes. Although this suggestion has not been fully executed, the implementation of the recommendations has been initiated to improve the Data Warehouse.
The actions undertaken by Scalefree consultants encompass:
The actions undertaken by Scalefree consultants encompass:
- Proposing a new target architecture
- Maintaining data lineage
- Ensuring GDPR compliance through the development and implementation of a GDPR pseudo-routine
Tangible Results for the Client
Compliance requirements, including IFRS standards and specific retention policies, have been satisfied by implementing part of the new target architecture. This accomplishment is attributed to:
- Achieving GDPR compliance
- Sustaining data lineage leading to:
- Optimizing runtimes
- Consolidating artifacts
- Standardizing load patterns
- Boosting the system’s performance and efficiency
Technologies involved
- Snowflake
- Wherescape
- PoC dbt Core with datavault4dbt
- Collibra
- JIRA
- Confluence
