One common requirement in data warehouse projects is to provide data lineage from end-to-end. However, custom solutions (for example custom Meta Marts for self-developed Data Vault generators) or tools from different vendors often break such end-to-end data lineage.
Unlike business or technical metadata, which is provided by the business or source applications, process execution metadata is generated by the data warehouse team and provides insights into the ETL processing for maintenance. The data is used by the data warehouse team or by end-users to better understand the data warehouse performance and results presented in the information marts. One type of process execution metadata is the control flow metadata which executes one or more data flows among other tasks. Logging the process execution provides a valuable tool for maintaining or debugging the ETL processes of the data warehouse because it provided information about the data lineage of all elements of the data warehouse.
MANTA is a global startup, born in Prague, Czech Republic and delivers detailed and accurate data lineage to the table and helps customers to regain trust in their data. This means data lineage is a must for all companies with complex structured data warehouses, that need to have everything all nice and tidy for regulatory compliance purposes, or companies that just want to optimize their data warehouse. How exactly does MANTA do it?
The Magic Trick
MANTA goes in your data warehouse and reads it all – like – super fast. It analyzes all the databases, scripts, stored procedures and cracks all your custom code – yes, even those parts that you don’t really remember who wrote them like that, but they sure do cause trouble. MANTA dives deep into map the very core. Not to mention it does it all in a fragment of the time a real human employee would, saving you lots of money.
Then, MANTA weaves all the information in a neat visualization, showing end-to-end data lineage flows or pushes it into 3rd party metadata solution. It can find data flows that you forgot you even had. They are a match for almost any tool you already have.
Any Lineage Necessary
In the latest release, they have now added support for multiple layers of abstraction, meaning physical, logical, business and pretty much any other you like. The automatic generation of logical and business lineage based on physical lineage and provided mappings will be highly appreciated by both your business and technical guys.
What they do is they can stick together the physical metadata from existing MANTA connectors with business terms their users provide them with. They are the first ones to be able to give you data lineage that suits the needs of your business users as well as your BI team from information that your company already owns!
MANTA still pulls physical metadata from your programming code but is now able to use existing business or logical mappings to prepare also a different perspective – a simplified one, using easier to understand names and descriptions, but still accurate, complete and fully automated. It allows to easily integrate with the not-so-technical solutions. (e.g. Collibra, IBM Information Governance Catalog, Alation, Data3Sixty, or Axon).
Let’s Be Complete, Shall We?
MANTA completes also many data governance solutions, to give them the little strength and lots of support they need. Together with them, they form a strong data governance platform that can meet up to your expectations and satisfy your needs.
In today’s world, data is the key. And taking care of it and its’ safety should be a priority. That’s why MANTA has an on premise version, so your data never has to leave the building. But they do have a remote version as well – just in case you are willing to fly to the clouds.
They have a nice online demo, and if you like what you see, don’t hesitate to ask for a free trial version!
