Approaching data management through the lens of the Viable System Model for the first time may feel overwhelming. It’s an entirely reasonable feeling to have (if you’re unfamiliar read Trevor Hilder’s excellent introduction). Learning the Viable System Model is like a learning new language. Learning it and then applying it to your work isn’t too different from taking a French class and then being asked to write a pen-pal in Paris.
I’d like to put together a scenario that we can use to walk through this. Let’s play pretend. Microsoft publishes a sample Visio database diagram that models a fictional bicycle businesss named AdventureWorks that we can use. You may want to download it to play along.
You’re the new IT manager for a moderately successful bike manufacturer, AdventureWorks. After meeting with your staff to learn what challenges IT is having, you’ve decided that one of your biggest problems is data sprawl. You have massive amounts of uncataloged and unmanaged data that may or may not be serving the business. You have data sprawl challenges in terms of depth and breadth. You suspect that years of unmanaged data in your database has led to your applications performing slowly. Furthermore, it’s reported that different business units each have their own applications, each containing some amount of duplicate data. There is no agreement on what information is canonical and there are problems with updated information not reaching everyone. The good news is that Sales, Human Resources, Purchasing, and Production have all been sharing an enterprise class application that uses the AdventureWorks database for some of their work. Starting with the AdventureWorks database seems like a good place to start. Without a lot of information at hand and desperate for something, you ask your database administrator if she has a database diagram and it turns out that we have one and it’s up to date. At a high level, the diagram looks like this:
Scanning the diagram, you’re able to piece together a bit about at least what the database reflects about the business. You also look for boundary changes. Boundary changes, or foreign key relations between the schemas that reflect business domains, such as between Purchasing and Production, implies channels between systems. You’ve decided that Production will be your starting point. Your team, having been versed in the VSM, starts with Production. Production appears to be a type 1 system, that is, it’s firmly in the role of operations.
Your analysts set up a couple of meetings with the folks in Production. Of course, the staff, having had bad relations with IT in the past with poorly implemented tools that either don’t do the job or do it inefficiently, are a little reticent but they accept the invitation. The first thing they bring up is the poor performance of the AdventureWorks database. It was only fast during the first couple of years but over time, it’s turned into a real dog and now sometimes they will get up and get a cup of coffee while they wait for a screen to load. Your team, knowing that the database is running on premium hardware and knowing that no data has ever been archived out of the system, starts asking questions about how long bike part information is needed for production. Production, whose focus is on absorbing variety from Sales and delivering on customer expectations, really uses most of the bike part information for tracking inventory and forecasting for parts purchases. Digging deeper, your analysts find that not all the inventory is tracked in the database. Because of poor relations with IT, newer business attributes associated with changes in how they do work are being tracked in an Excel spreadsheet. It was easier to use Office than try and ask for changes to the enterprise application. The work order module of the application isn’t being used either. Instead Production uses email to let assembly teams know when work is ready for them. People sometimes forget to check their email or they get buried under other emails. This leads to unpredictable delays and other challenges. As your team starts to wind down the meeting, Production mentions that they have challenges meeting the special requests from Sales. The Sales department frequently oversells limited issue models, ignores recently discontinued models, or special requests. Your team certainly has their work cut out for them.
Let’s go over what your team discovered in that meeting. The AdventureWorks database and application has issues attenuating data, absorbing variety, and amplifying effort. Attenuating data could be pretty straight forward. Your team will need to have a couple of meetings about how long certain data needs to persist and in what state with the coordination, control, intelligence, and policy systems that have channels with Production. Some information likely can be archived in a report somewhere and purged. Or it may be transformed in some other way that meets the variety needs of other systems.
The problem of the application absorbing the appropriate variety is a little stickier. Shadow solutions, like the Excel spreadsheet they’re using, are in place because of inefficiencies between Production and IT. Investigating how those processes work and fixing them will take some later brainstorming but, for the time being, you can document and implement the new tables and import the spreadsheet into the database. This information too, may or may not be necessary for other systems to do their jobs. Your team will need to investigate, create, and document as necessary.
The biggest area of issue will be how to amplify Production’s efforts. The application’s work order module is so painful to use that the Production team would rather roll the dice on email. In those inboxes, you have “unstructured data” relating to work orders intermingling with many other types of information. Sorting and prioritization are challenges. Assembly teams are out of sync as communication and teamwork is out of sync. This is a coordination nightmare and is likely being felt in other parts of the business. For example, Sales may be having to deal with failure demand, “Where’s my stupid bike!?” This could beget direct commands from a control system, an unsavory but potentially necessary experience. The easy question for your team to ask is how to make the application’s work order module work better so they don’t have to use email. The better question is, in general, what does Production need to improve the flow of work? The application’s work order module may end up just getting deep sixed. Saving a poorly built piece of software does no one any favors. In this context, flow is what matters.
There’s also the specter of the residual variety that Production continues to absorb from Sales in terms of difficult or impossible to fulfill orders being placed. This likely also impacts Purchasing as well. There are data and communication issues here that need to be investigated.
Your team, informed of a systems approach, will have come away with more questions than answers. This is a good thing because systems thinking is a holistic approach that isn’t satisfied with addressing symptoms. It will compel one to find root cause.
What the Viable System Model does in this case is provide the appropriate model and concepts for data management. By understanding who or what in your organization is responsible for operations, coordination, control, intelligence, and policy you can start to ask the right questions on what they need. You will also need to employ the concepts of variety, amplification, attenuation, as well as algedonic signals and the Law of Requisite Variety. In this way, you can carry on a conversation with them about their needs instead of continuing the time-honored IT tradition of mere order placement and fulfillment. Having those conversations is critical because the false dichotomy between business and IT is vanishing (some places faster than others).
Data management isn’t just about finding and deleting data that isn’t used anymore. Data management is about ensuring that the data needs of the organization are met. To be effective, data management needs to partner with business process management. It’s very difficult to discuss one without encountering the other. Employing systems thinking and the Viable System Model will go a long way toward doing the hardest work of all, asking the right questions instead of merely looking for right answers.