Data Management, Waste, and You

Data catastrophes usually come in the form of clumsy fingers. Sometimes it’s a user. Sometimes it’s an IT administrator. The recovery process is straight forward, right? Find the backup. Restore it. Resume eating Bonbons. Of course not. Technically, backups are easy. In practice, they’re amazingly difficult. And it has nothing to do with technology.

Backup sites tend to behave like digital versions of the junk drawer at your home. Just on a massive scale. The business, unwilling to part with data, just wants to back up everything. IT is left trying to sort out massive amounts of data, much of which it knows nothing very little about in terms of importance and relevance. Time to recovery? Point of recovery? Forget about it! And the money spent on storing it? Dizzying. Organizations that don’t treat their data as an asset (complete with value depreciation) often fall into either of two camps: those that continue to pour money into saving everything or those that are reluctant to spend money on storage. If you’re in former camp, good luck finding anything when you’re saving everything. If you’re in the latter camp, you’re going to eventually inadvertently push important data out of the system, losing it forever and not knowing it until time comes to use it. Both approaches are costly, one just defers the cost until later. It’s in every business’ best interest to manage their data as an enterprise class asset.

If our offices reflected the way we managed data, we’d find ourselves unable to navigate to the restroom with all the clutter. The individual practice of saving everything you’ve ever done on a computer rolls up into massive and gnarly enterprise problems. We also institutionalize it within our databases. Some of the best minds in the world have worked on trying to assist in resolving this problem of the human condition, this need to hoard data and save it forever, ignoring the diminishing return on value that much of it represents. Since we all struggle with data management, there’s an entire industry built around helping businesses with it. So, with costs starting somewhere in the low six figures, you can acquire yourself a data deduplication system where you can deploy some of the best mathematics in the world to assist you in managing this problem. This buys you only time at the cost of extra money as data deduplication is costly. The dike is built but the waters continue to rise.

Like backups, the value of deduplication is marginalized without a coherent plan to address on how to manage data as an asset. And managing data as an asset is a challenge unless one takes a systems approach to data management. The imperative of a systems approach to data management is that IT is laterally positioned within the business. We’ve had a number of years of experience within the industry with service-oriented architectures and even longer with extract-transform-load operations. We’ve built data warehouses. We’ve written an uncountable number of reports. We’ve long recognized the flow of data throughout the organization. There’s no one better positioned than us to lead our businesses into these brave blue waters. What is required of us is that we evolve that model to understand the flow of work, or even better, the flow of value creation throughout the enterprise. We need to understand what is required by each of the subunits to do precisely the job they need to do. As any analyst can attest, what is required likely doesn’t match what is done. This is how IT must provide the leadership necessary to put together a comprehensive plan of data management that treats data as an enterprise asset replete with backup schedules, archival schedules, records management, and more.

To that end, it’s time we resurfaced the Viable System Model and applied it to data management. Stafford Beer’s model dissects the organizational model from a systems perspective and describes the necessary subsystems required for an organization to be viable; that is, to remain effective within its particular ecosystem (or market). An excellent introduction to VSM can be found here.

As we progress into our respective data management programs, we will need to align each organizational unit with their appropriate system or systems, discover what states each system requires to maintain control (referred to as variety) and how each system interacts with the others. That leads right to discovering what data is needed to support control and what data needs to be ignored (referred to as attenuation). Understanding what data can be attenuated is key. Coordination and control systems can consume only so much information at a time. Oversupply of information leads to costly delays and waste. Most questions about data aren’t necessarily binary. That is, it’s not usually just a question of keep versus discard but, rather, how long to keep it and in what state.

If you’ve built a data warehouse, you’ve likely asked many of these questions already. The goal would be to expand that work so that the data warehouse isn’t the fixed point that everything traces back to. One of the tricky tasks of IT leadership is to decouple programs from the technologies that support them. Failure to do so leads to very costly evolutionary programs or over-reliance on obscure systems. Think about California and the amount of money they’re spending on COBOL programmers to support their legacy systems today. Take the work you’ve done in building data warehouses and expand upon it with the VSM. Generalize it.

What is particularly attractive about applying the Viable System Model to data management is its neutrality in language and flexibility in application. It has a lexicon that is neither rooted in technology nor business language. It is intuitive to understand and is malleable enough to be utilized in most any situation.

There’s more to this than just backup schedules. Backups are just the start. Your programmers and analysts, once involved, are going to start revealing data waste all over the organization. Like a compass, that data waste will point you to institutional waste baked into stagnant business processes. You’ll be a catalyst for business process improvement. Or just business process management if you’re starting humbly. What it takes is a common framework and language that sets the stage for asking the right questions.

It’s critical for you to start this now. Why? Because we can no longer afford to blindly throw money at problems that require planning and management. We’ll let Peter Sondergaard from Gartner be the final punctuation mark.

Tagged , , ,

2 thoughts on “Data Management, Waste, and You

  1. Eric Emerton says:

    Awesome post Peter. Data sprawl is definitely part of the human condition. I agree that IT as a business unit (what a concept!) needs to work with the firm to develop strategies around data storage. What has helped in my company has been bringing the records department to the forefront. Understand the difference between working files and long-term business documents is a simple mindset that helps users from worrying about keeping their room clean. A relatively inexpensive records management system (ie Symantec Enterprise Vault) can add de-dupe and allow users to manage the “important stuff”.

    My biggest frustration of late is when vendors change the game. Microsoft removed single-instance storage from Exchange 2010. That can make huge impacts in expensive SAN storage for companies without a good email archiving system.

    Even though the best option would be to fix the way business works, that never works in our industry. There’s a sweet spot between gradual user change and smart technology.

  2. phouseak says:

    Boy, Eric, you really hit the nose there. A records management initiative is a great way to introduce a data management initiative. It certainly can shake your users out of their habits of saving everything and with consequence too. Legal and policy consequences hang over anyone out of compliance. I believe a generalized VSM approach to data management will be a key differentiator between successful, mature organizations and those that persist in mediocrity or worse. It certainly comes with gradual change though. Anyone who tries for overnight change will only be successful in the short term. It’s like crash dieting as opposed to gradually change the way we approach food and diet.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: