As a bit of a movie geek, one of my favourite quotes came from the 1987 classic, The Untouchables. “never bring a knife to a gun fight”. It may seem strange to launch into an editorial on Enterprise Data Warehousing by quoting lines from Brian De Palma’s, mobster master piece but, as I thought about where we now are as an industry, it seemed fitting.
When you look at the fight we are now in as data professionals, the odds are overwhelmingly against us. On one side, there is the irrefutable explosion and growing complexity of data. And then on the other, there is the exponentially growing rate of disruption and organisational change.
Faced with these dual and compounding forces, it appears that we are caught in a pincer manoeuvre and left asking the question “ can any Data Warehouse realistically be expected to keep up with the changing needs of a business’?
The problem is that the playing field has changed so significantly, most of us are using methodologies that were originally developed over 20 years ago. And whilst these approaches have continued to evolve, their underlying architecture was not originally conceived to meet the needs of big data, or the new agile business methodologies they now serve.
This means that the odds are stacked against us. Most of us find ourselves with our backs against the wall, and now stuck “with a knife in a gun fight”
So what’s the answer, do we just throw the towel in? Well as Special Agent Eliot Ness said “Never stop fighting till the fight is done’.
As I look at the data warehouse landscape, there are a myriad of solutions and tools available to provide point-solutions or quick fixes but they all fail to address the real problem, the foundational data that they are built upon. Continuing to layer solutions on poor data is just creating a house of cards that will eventually come tumbling down.
However, there is a different way. Data Vault 2.0 is a unique approach to data warehousing that was created from the ground up to deal with the real-world data challenges that most businesses are facing today. DV 2.0 delivers unprecedented Total Cost Ownership - typically less than 50% of traditional Data Warehouse methodologies, greatly enhanced operational agility, and traceable data governance.
The Data Vault 2.0 methodology is different from traditional approaches: offering the ability to support business intelligence with both governance and agility. It achieves this by abstracting your business rules from the datasets beneath it. This means that as your business changes, or you want to test, model or query, you can easily modify the rules and quickly generate the results you need. This is a stark contrast from traditional processes that bake the business rules into the data as it’s loaded.
Because of the simplified model, automatic code generation, and abstracted business rules, loading new data into a Data Vault is significantly less labour intensive. The speed of load, combined with the ease with which new reports can be created (typically around half the time) delivers a solid improvement in total cost of ownership. By adopting DV 2.0 you can typically halve your IT headcount across development and testing.
Data Vault 2.0 is also effectively tool-agnostic: you can deploy it on the same database and with the same software you currently use – avoiding expensive re-tooling and re-training.
Additionally, when you need to extend your Data Vault 2.0 data warehouse to include a data lake the methodology remains the same - meaning that data is easily ratified and shared across both.
In addition to increased load and change speed, the accuracy of the data is also significantly improved. Data Vault 2.0 delivers full auditability by protecting the integrity and origin of the data. Whilst traditional methods only present a “single version of the truth,” cleansing or removing what does not fit with predefined business rules; a Data Vault can offer all the in-scope data all the time. This provides improved accuracy and data integrity, allowing an auditor to trace values back to the source.
The unique architecture of Data Vault 2.0 means that it cannot only manage the complexity of today’s big data workloads, but also fully support business agility and transformation.
It’s time to bring a gun, to the gun fight. If you are looking at a simple way to get back into the fray, Certus has just developed The Certus DV 2.0 Platform. A fully managed Platform as a Service (PaaS) offering that provides a best practice environment for Enterprise.
The Certus Data Vault Platform combines the Data Vault 2.0 Methodology with the latest capabilities in managed cloud infrastructure, best of breed technologies, ETL automation, metadata management and governance & data quality as well as end to end managed services. Matched with a delivery framework including Agile Delivery Methodologies, project & design templates as well as resource training and coaching to provide a holistic approach to modernising your Information Management capabilities covering People Process and Technology.
Deploying Data Vault 2.0 allows you to unlock the power of your data and improve TCO, whilst transforming antiquated EDW’s into agile platforms capable of driving strategic decision-making and ultimately business growth. Learn more here.
