An Approach for Robust Data Management
Building a robust data management environment is in many ways like building a house. There are three components to building a good house. First of all, there are some fundamental questions you need to ask before doing anything. Why are you building the house in the first place? What are the important goals and benefits you want to enjoy? What other things are you willing to trade off to realize those benefits? Asking and answering those questions will help with the second component: building a model, or architectural blueprint. There are many different ways to build a house (or a data management system). Not all of them will be right for the needs you have in mind. There are efficiencies to designing and building in certain ways – and, as always, there are trade-offs with any given choice. Finally, once you have established a workable model, it’s time to build out the infrastructure. That starts with the plumbing. Nothing else in the house is going to work well without good plumbing which, seamlessly and unobserved, harnesses the flow of water (or data, in our analogy) to efficient uses as and when needed. Then comes the foundation – the platform to support the house according to your model. Think of the plumbing and the foundation as the transmission pipes, the controls to regulate the flow of information, the storage repositories and the other critical supports for your data management platform.
robust systems need good blueprints
Asking the Questions that Matter for You
It’s hard to imagine that someone would build a home without first asking and answering some basic questions about what purpose the home is meant to serve. But all too often enterprise managers think of their data intelligence needs in terms of generic, one-size-fits-all products and solutions. They may be driven by the perceived urgency of getting immediate results, so they do not put the extra time into thinking through all the details that have to be in place in order for a solution to best meet their targeted needs. They build up organizational IT resources but fail to adequately integrate these resources into business decision-making processes so that business goals and technological capabilities are aligned. By not asking the right questions up front, managers increase the likelihood that their IT investment will fail to achieve the specified goals.
The place to start is by asking what kind of business you are. What are the core elements of your business strategy? What kind of information – about customers, products, channels, territories, campaigns, transaction dates and so on – do you need to pull into your organization every day in order to make decisions that help accomplish that strategy? What performance measures do you seek to optimize – for example gross margin, market share or something else? Is there a single overriding imperative for all territories or are there more nuanced considerations for local markets? Who needs access to the data, both within the organization and among trade partners? These broad-based questions can then lead into more detailed questions around specific tactical opportunities that can help support the overall strategic goals.
Modeling Around the Right Questions
One homebuilder wants a large kitchen with lots of counter space for preparing gourmet meals. Another wants to optimize for green initiatives –creating spaces for solar panels and the like. Every homebuilder needs an architectural blueprint and model that defines the best tools, materials and processes to accomplish the job at hand. These decisions should flow naturally from the questions asked and answered in the previous stage.
Data models are designed for different purposes. Some place emphasis on providing a continual stream of information for decision-making in real time. Others focus on high quality analytical tools applied to batch data that are aggregated for a particular time period – for example daily, weekly or monthly. A critical driver of data architecture is the volume of information the system is expected to handle and the types of activities users expect to be performing when they access the databases. Information processing has a cost, which includes a time component and a money component. Thinking back to the homebuilder who wants a state-of-the-art kitchen space, the architectural blueprint for that kitchen has to think through all the activities our homebuilder-chef is going to perform – how to optimize the process of retrieving ingredients, setting up preparation work stations, pulling out pots and pans and chef’s knives, and cooking on the stove or in the oven. The data architecture similarly has to visualize the user retrieving various bits of data from different locations at different times for different purposes, and figure out the configuration that works best.
Laying Pipes and Building a Platform
The most elegant model in the world will be of little use if the infrastructure is not solid. Living in a house we tend not to think about the plumbing as long as it is working – we wash dishes, water the plants and fill the ice trays without wondering about how the system works. But we will certainly notice when it doesn’t work – when water doesn’t come out of the faucet or when it floods the basement. Likewise a business user at a computer workstation is probably not going to think about how all that information flowed from somewhere out there to show up at her desk – unless it doesn’t show up or is in an unusable format. Laying the transmission conduits and controls that regulate the flow of data to where, when and how it is needed in the organization is a serious undertaking.
When information comes into the organization from “out there” it is unfortunately a bit more complex in form than the municipal water that enters your home’s plumbing system. It is easily corruptible and subject to misuse if robust protocols and procedures are not in place to keep it clean and usable. In a surprisingly large number of cases data “formats” consist of little more than unprotected Excel spreadsheets with inconsistent definitions and naming conventions for product categories, subsets, SKUs and customer types. That will likely prove to be of little use in facilitating actionable analysis. To be usable data need to be clean and granular – in other words, accurate, in the right format and at the right level of detail for the analysis to be performed.
Organization Matters, Too
Business managers do not always take into account the organizational aspect of data management. The solutions architecture will not be effective if the enterprise is unable to successfully weave it into the fabric of the organizational culture and existing decision making processes. Empirical evidence suggests that rigorous analytical methods play a relatively minor role in the decision making practices of many organizations – especially those at higher strategic levels but also those related to daily tactical opportunities. Ask yourself how different levels of decisions are made in your organization. What role, if any, does data analysis play? How persuasive are data-supported arguments – for example in comparison with rigid top-down planning imperatives, an ad hoc “firefighting” culture or political and personality-centered negotiations? What needs to change in terms of both formal processes and the informal cultural context for actionable solutions architecture to make a lasting impact and lead to more informed decisions?
Like a house, a data management system is best if it can last for a long time and have the flexibility to adapt to changing needs and circumstances. Business strategies will change, as will technology. Far better that a system can adapt to this change without requiring an entire rebuilding effort from scratch. Start with the right questions, and you have a much better chance of building a data management capability that can successfully evolve with your organization.