Data Driven: more than technology

Crystal Clear

It all starts with having a clear vision. Personally I believe that analogies in which data are called “the new gold” or “the new oil” create an image that is too exclusive, although I understand that “gold” refers to the value that can be obtained from data. I prefer the analogy in which data is compared to “water”: it should be available in crystal clear formatting and quality and easy to use by different types of users (e.g. for drinking, showering or washing dishes).

Concepts like data quality don’t mean anything without the context of the application: to determine if only having quality “rainwater” is a problem, one needs to know whether it will be used for flushing the toilet or making coffee.

The analogy is clear: the water company has to create water that meets the standards in an auditable way (data governance processes). Within our houses we have a pipeline network (the data architecture) that distributes water to the using devices in order to make it ready to use by simply turning on the tap (compare to time to market for a new application). Of course, make sure that you know what you are doing (skills)! The guy (or girl) who connects your shower is preferably a plumber, the one who uses your espresso machine a barista (personally I don’t have my private barista so I followed a course on it – being a plumber however will never match my talents). Different skills for different players in the chain: if the barista must have too many plumbing skills, he will neither be very effective nor will he like making coffee.

Sounds easy. Then why don’t all companies succeed in this transformation? Because most companies have legacy systems, architectures and processes. And legacy people and legacy rewarding systems…

Data Governance

The first part is creating data that meet the standards. But who determines these standards and who is responsible for meeting them? Probably in many companies there will not be a clear separation between data, functionality, systems and process. So when introducing data ownership, the person who is system owner will be the first one who is asked to “volunteer”. He (or she!) might tell you: “well, my system is used for process X,Y,Z and the KPI’s on that processes are green, so we don’t have a problem with data quality. I don’t think that I should be a data owner, and if I were one, I would argue that we have no problems. Everything is going fine here!”. You, as a data governance professional, might argue: “but we don’t want to think starting from your system or processes, the quintessence is that we decouple data and give it a value that exists independently from systems and processes in order to be able to use the data not only for processes X,Y,Z. Perhaps tomorrow we want to use the data that comes from your system for another purpose”. I bet you will get an answer like “but I can never be responsible for that, that’s not part of my targets”. Perhaps, witty as you are, you try the colleague who is responsible for the operational processes that use the data. But, perhaps slightly different to some extent, you get the same answer.

So, simply spoken, as long as you don’t find anyone who changes the targets of one of these colleagues or forces them to look beyond their traditional responsibility, you will not find a data owner. That is the first reason why you need executive sponsorship.

Data Architecture

A flexible architecture decouples functionality from data and data from system context, not necessarily physically. Anyhow metadata is the crucial core: what data do I have, what does it mean, where/how do I get it and how do I deliver it to users and applications. Many different solutions might work, virtualization, physically storing data in a standardized language and so on, but these solutions have one thing in common: they differ very much from the traditional monolithic architecture and they are very expensive to realize. And alignment with the IT strategy is crucial, or, even better, both are integrated. At the beginning you might trust some vendor who says that he can do magic but at some point in time you find out that the complexity is within the legacy itself and it takes hardcore, unscalable knowledge of your company data, systems and processes to do the trick. If you are very unlucky this knowledge exists only within the heads of people that were fired in the last reorganization…

Your boss might have asked you to write a business case. But where to start? The marketing department knows that they will do “something” with the flexible data architecture but they didn’t figure out yet what exactly (“data is the new gold, we will make new business models like we saw on our business trip to Uber”). And your gut feeling says that there is also a colossal business case on making your traditional processes more efficient and flexible, but you cannot fund this case on just doing things differently (it works as it is right now, doesn’t it?).

To make it even more complex, some department that really wants to get on with innovation already hired ten data scientists and they did some really cool pilots. But they need both the new architecture and the prepared data from the traditional systems to operationalize it. Half a year later, half of them quitted their job.

This is not a one year project. It will cost your company a lot of money. And every path you take, can be criticized: if you focus on the traditional processes it will take long before you deliver value (in the sense that you can deliver a product or service you couldn’t deliver before), if you focus on innovation you will lose the connection with running business and be stuck with the image of wasting money on cool things whereas the rest of the company has to economize, and not even delivering fast enough on that.

So this is the second reason why you need executive sponsorship. Someone at board level who understands the impact and helps you clear the way. It is not about THE best architecture, but about the understanding that it will be a long journey and creating a broadly supported path, anchored within the corporate strategy and corporate business planning, is the crucial success factor.

But how?

Large companies are often managed on a solid control basis of finance and risk. This might disconnect with the need for “belief” as described above. I find it interesting to see that “doing nothing” is often not seen as an alternative that also bears risks. It reminds me of the slogan I read on Linked-In: Q: “what if we train our people and they leave?”, A: “what if we don’t and they stay?”. I guess the same holds here: not transforming to a data driven company is also a choice with consequences. Of course, depending on your situation, it might be a reasonable choice but don’t make it implicitly by not making another one.

First of all you need to be lucky that a board member or a whole board really wants to understand the journey and is willing to listen to what you have to say. And then comes the part where we as data professionals really have to be able to convince. And frankly spoken, that is not always our core competence. We like details, we like to explain why we found THE brilliant architecture or THE perfect data ownership model that exactly states what everyone has to do. We talk in a potentially confusing “slang” (let’s mention a bit of a bullshit bingo like Hadoop, RDBMS, DAMA DMBOK, AI, attributes, data modelling, integration and interoperability, API, ….).

I think that it is crucial to be able to close the gap between data professionals and board members, directors, managers and colleagues at different levels, combining bottom-up and top-down awareness and a shift from push (professionals explaining why the transition is important) to pull (business units asking for it). There is no “one-size-fits-all” recipe for that. You will need sponsorship and you will have to work very hard to get (and maintain) it. And of course, you will need good ideas on governance and architecture. But it will be the people that make the difference…

Tim Schulteis will present a keynote during the Data Warehousing & Business Intelligence Summit on June 9th: Data Driven: meer dan technologie.