The four “Vs” of data are well known – volume, velocity, variety and veracity. However, data warehousing infrastructure in many organisations is no longer equipped to handle these. The fifth elusive “V” – value – is even more evasive. Meeting these challenges at the scale of data that modern organisations have requires a new approach – and automation is the bedrock.
For CDOs, it’s all about finding methods of using data for value creation and revenue generation, which occupies 45% of their time. This means harnessing the growing beast that is data in a way that is practical, manageable and useful. That’s where the data warehouse comes in, providing a centralised space for enterprise data that business users, including the CDO, can use to derive insights.
Creating a successful data warehouse is critical for CDOs to succeed in monetising data within their organisation. However, the traditional waterfall approach to data warehousing, first introduced in the 1970s, delivers only a fraction of the value that it could potentially offer. Instead, the approach needs to evolve to become more responsive as organisational needs change, addressing new data sources and adapting to business demand.
Practical steps for the successful implementation of an automated data warehouse
As IT departments are expected to do much more with much less, processes need to change. Data warehouses can no longer be created “artisinally” – IT teams need to focus on producing an adaptable decision support infrastructure. Here are five steps for CDOs to help their company achieve this:
Understand the desired outcomes: Before making any decisions as to the future of your data warehouse infrastructure, CDOs need to ensure they understand the specific challenges the business teams are facing where data could help. In essence, the data warehouse automation and modernisation program needs to be built around enabling decision-making that will lead to differentiation in the market place.
According to 39% of respondents a TDWI survey, re-alignment to business objects is the top reason for data warehouse modernisation, selected by. By enabling collaboration between business teams and IT teams, the CDO helps chart the course for how business goals and technology meet. In turn, this will lead to overall business transformation, accelerated through the new data warehouse’s approach to data-driven decisions.
Understand what you have already: Most organisations already have sophisticated data management tools deployed as part of their infrastructure – however these may not be working to the fullest of their abilities. Organisations already using SQL Server, Oracle, or Teradata, for example, have a range of data management and data movement tools, already within their IT real estate, which can be automated and leveraged more effectively as part of a data warehouse automation push.
However, in that inventorying process, CDOs should be ensuring they have considered the capacity requirements of their data warehouse. Data will continue growing exponentially, so while the data warehouse may be fit for purpose today, it’s important that the automation processes, storage requirements and general infrastructure are capable of handling this in the future too.
As part of this, data warehouse automation needs to integrate with the business as it is, rather than the business as the IT teams wish it might be. CDOs need to encourage their teams to understand the data that is available, and the automated analytics and evaluation processes which can be used to meet specific business priorities. The data warehouse automation strategy needs to be designed not just for an ideal set up of data, expertly managed and curated, but for the realistic “messiness” of the business data landscape.
Automate efficiently: Data warehouse automation, as with any other large-scale transformation project, requires resources – and these are often scarce due to strict budgets and competing priorities. This means that CDOs need to think hard about what actually should be automated in order to free up man-hours in the future. In particular, these should be systematic processes, where data warehouse automation can either eliminate the need for human involvement or dramatically accelerate the process.
Embrace change: CDOs should look at data warehouse modernisation and automation as an avenue of constant, on-going development. As business needs change and new data sources emerge, CDOs need to be able to re-strategise different parts of the infrastructure to match. Similarly, to minimise disruption and ease the transition for business users, CDOs should take a staged approach to the initial automation and modernisation process, with a set schedule of when different requirements will be met. Post-production change is inevitable due to evolving business needs, new technologies used and continuous improvement desired. Change needs to be planned for.
At the same time CDOs need to prepare for the human change that automation will create. In business teams, users can be re-deployed on analysing business intelligence and translating insight into business value. In the IT teams, automation provides new capacity to plan for the future – looking at new analytics tools, or planning for smarter, better ways to deliver on business priorities further down the line.
A data warehouse automation mentality
Data warehouse automation is not solely software you buy. It’s a philosophy and culture you implement. Tools and technologies form the bedrock of the processes, but a data warehouse strategy requires strong leadership, a transparent process, and an unrelenting focus on the business’s end goals in order to succeed.
Without robust data warehouse automation, businesses will struggle to capitalise on the potential of data and its associated technologies. As the strategic lead for data-driven transformation, and the change agents across both business and IT teams, the responsibility falls to CDOs. Professionals in this role need to understand, strategise, and execute on the way that large-scale data usage will influence future business decisions. The adaptability of the supporting data infrastructure can either be a CDO’s greatest weakness or greatest asset. Use the four steps above to ensure it is the latter, and to achieve the ultimate goal of any business investment – value.