Category Archives: Tech Trends

The systematic challenges associated with forming insight on market behavior and consumer dynamics given massive quantities of data and elusive patterns of market behavior/consumer dynamics along with sparseness of data at the micro-market level.

Business Intelligence: Five Questions to Ask Your Technology Provider

Posted on by

Business intelligence can provide enterprises with deep visibility into their data and to extend the range of vision otherwise available to them. Business intelligence tools for better customer relationship management should help enterprises all along the demand chain to have a full and coherent picture of the market for their products and services. The specific requirements will be different from business to business: manufacturers, distributors and retail operators face different challenges and need different solutions from their business intelligence technology. But any enterprise can get a good start in identifying their own particular needs by considering the five following questions when evaluating the merits of a particular BI solution:

#1: Does the solution provide a means for the enterprise to do more with its data than it currently is able to do? How does it deliver this capability, and what makes this approach better than alternatives?

Any business intelligence solution provider is likely to answer the first part of this question in the affirmative: of course we give you the tools to do more with your data. That’s why the second part of the question is such an important follow-up. How does it deliver the capability? What the business user sees when using a BI user interface tool is just the end of a long series of events that starts with the extracting of data from different sources, transforming the data and making it available for the specific discovery, analysis and decision-making activities in which the business users will be engaging. There are a great many decisions and trade-offs that have to be made about the technology that supports these ETL (extract-transform-load) activities. Some choices will make it easier for the vendor to develop and package the solution, but at the expense of the business user’s ease of use. Can your provider show convincing evidence that the technology choices put the interests of the business user ahead of those of the technology developer?

#2: Does the solution provide different stakeholders with a unified picture of demand, at a level of detail to gain a personalized understanding of the preferences, tastes and needs of each customer?

If you’re using business intelligence for better product and customer relationship management then it’s important that they key stakeholders with an interest in the technology will have a single view of the market environment. That’s easier said than done. The problem is not always the massive volume of information, but rather the multiple streams of data that come in from different sources and providers, in different formats and structures. Threading the many strands into a single coherent picture is a significant technology challenge, and you need to be sure that the technology you adopt is up to the challenge. Beyond that, you want to ensure that you will have information at the right level of detail. Beware the “flaw of averages”, where you see information aggregated to a level that loses critical knowledge about the individual customer’s needs and preferences. That loss of knowledge has a measurable cost in money left on the table with each customer-product interaction.

#3: Does it supply the requisite level of domain expertise to provide intuitive business definition addressed to the language, processes and measurement techniques understood by your business users?

Let’s face it: business professionals at any given company, in any given industry sector, speak a particular language. It’s the language of their products, their customers, their processes and their performance measurement techniques. A good business intelligence solution needs to be rooted in a sufficient level of domain expertise to translate the language of the technology into the language of the business. That doesn’t necessarily mean that your BI solution provider has to be a boutique single-industry expert. But it does mean that the core technology capabilities are sufficiently customizable to easily adapt to the functional language of your business environment. Be cautious about off-the-shelf solutions that claim to work as turnkey offerings in any business environment.

#4: Does it allow for rapid time to value without the need for additional investment into technology and resources on your part?

How much pain before the gain? You want to see results early and often, and you don’t want to worry about the need to acquire additional infrastructure or build out an internal IT team to support the technology. In today’s Big Data environment you are probably going to want a hosted solution where the data is warehoused and managed in the cloud. The most important thing to know about a cloud-based solution is that it passes the responsibility of day-to-day management of the data from you to your technology provider, meaning less of an internal administrative/IT burden for you. Then you can focus on the other part of this question: understanding the time frame in which you will be able to see and measure the value delivered by the solution.

#5: Does it adapt and make continuous improvements and facilitate those improvements back into the system for still more informed insights?

The only thing certain about your market environment is that it will change, and often in ways unanticipated when you invest in a new technology. Technology that cannot adapt to new business requirements can go stale very quickly. Does your provider offer a market aware technology? Market awareness is when a technology platform has the capability to learn from new market insights as they evolve, and feed those insights back into the core models and algorithms that power the technology. Market awareness gives you more confidence that your business intelligence solution will be able to rise to the challenges of tomorrow as effectively as it can to the challenges of today.

Selling in the Age of Big Data

Posted on by

Every day we see more evidence that the age of Big Data has arrived. With the data inevitably comes automation – the steadily increasing automation of business activities that used to be performed by human beings. For companies with a strong sales culture this presents a deep-seated challenge. Business analytics and data-driven sales & marketing tools can offer deeper insight into your demand environment and provide recommendations to make better decisions around your products and customers. But as an organization you worry – do these data-driven capabilities really make your organization better if they threaten the very foundation of your longstanding sales culture? You don’t want a technology solution to replace your sales force. What you want is a technology solution that can make your sales professionals better at the job they already do.   Continue reading →

Big Data: It’s Not Just About Size

Posted on by

Are you ready for the era of Big Data?

The phrase “Big Data” is now firmly part of the mainstream business lexicon. You hear about it with increasing frequency, but what does it really mean for you and your business? The answer is: a great deal. Big Data not just another lazy buzzword. It has the potential to revolutionize your industry value chain and upend practices that have been in place for years. The revolution has not yet taken root as firmly in many B2B markets as it has among retail consumer-facing enterprises, but that is fast changing. How do you get ahead of the curve and be in the position to profit from the changing role of technology in business decision making? A good place to start is by understanding what Big Data actually is, and how it relates to your company, your customers (and their customers), your suppliers (and their suppliers) and all your other key business partners. Next is to understand what capabilities you currently have and what gaps exist that need to be filled for a full-fledged Big Data capability consistent with the specific characteristics of your demand environment. Continue reading →

Managing Information in the Age of Big Data

Posted on by

Big Data presents challenges for business managers and IT pros alike

The Big Data revolution taking place across many industry sectors is presenting challenges to business managers in traditional line functions like production and marketing, and to corporate IT professionals alike. Big Data goes to the heart of the IT professional’s domain – the way that the enterprise warehouses and manages information – and potentially upends long-established systems and practices. At the same time it presents business manages with a new level of urgency to incorporating deep data analytics into their daily array of tactical business decisions. As important as it has always been for business and technology professionals to be on the same page, it is now more important than ever. Both sides also need to understand the specific concerns and challenges that are keeping the other up at night, and work together to solve them and identify critical gaps. Continue reading →

Before You Build, Ask the Right Questions

Posted on by

An Approach for Robust Data Management

Building a robust data management environment is in many ways like building a house. There are three components to building a good house. First of all, there are some fundamental questions you need to ask before doing anything. Why are you building the house in the first place? What are the important goals and benefits you want to enjoy? What other things are you willing to trade off to realize those benefits? Asking and answering those questions will help with the second component: building a model, or architectural blueprint. There are many different ways to build a house (or a data management system). Not all of them will be right for the needs you have in mind. There are efficiencies to designing and building in certain ways – and, as always, there are trade-offs with any given choice. Finally, once you have established a workable model, it’s time to build out the infrastructure. That starts with the plumbing. Nothing else in the house is going to work well without good plumbing which, seamlessly and unobserved, harnesses the flow of water (or data, in our analogy) to efficient uses as and when needed. Then comes the foundation – the platform to support the house according to your model. Think of the plumbing and the foundation as the transmission pipes, the controls to regulate the flow of information, the storage repositories and the other critical supports for your data management platform.

a blueprint is an architectural model

robust systems need good blueprints


Asking the Questions that Matter for You

It’s hard to imagine that someone would build a home without first asking and answering some basic questions about what purpose the home is meant to serve. But all too often enterprise managers think of their data intelligence needs in terms of generic, one-size-fits-all products and solutions. They may be driven by the perceived urgency of getting immediate results, so they do not put the extra time into thinking through all the details that have to be in place in order for a solution to best meet their targeted needs. They build up organizational IT resources but fail to adequately integrate these resources into business decision-making processes so that business goals and technological capabilities are aligned. By not asking the right questions up front, managers increase the likelihood that their IT investment will fail to achieve the specified goals. Continue reading →

The Changing Landscape of the Foodservice Industry – Part 2

Posted on by

This is the second installment in a two-part series on major changes taking place in the US foodservice industry. In the first installment we looked at some of the key challenges, deriving from traditional industry practices in sales & marketing that impede optimal performance by manufacturers, distributors and operators in the sector. This second installment will take a closer look at converging technologies that are poised to shake up the industry, and look at ways for industry players to benefit from these developments with intelligent, coordinated approaches to technology-driven solutions.

For manufacturers of foodservice products an important and often elusive goal is to gain visibility into the factors shaping and influencing downstream demand. The view from upstream is obscured by one or more layers of intermediation separating products from their end customers. Manufacturers typically set aside the largest part of their sales and marketing budgets for payments to trade partners, but evidence suggests that these expenditures do little to improve their understanding of actual downstream demand. Whether on their own or in collaboration with trade partners, manufacturers need to make better use of the data that can provide accurate intelligence about what is happening downstream. The good news is that the data are available, and new technologies are converging to enable manufacturers to capture information from which to make better sales & marketing decisions. The challenge is to get around the obstacles that are preventing this from happening. Continue reading →

Missing the Ocean for the Stream: What We Can and Cannot Learn from IBM’s New Breakthrough

Posted on by

As part of its perpetual quest to reinvent and perfect its business model, IBM has made an aggressive push into the analytics market in the last half-dozen or so years. The company’s slick, though occasionally confusing ad campaigns (remember those ads with the mysterious red box being unveiled?) often announce its new initiatives, though it is not always clear that a new announcement is indeed a major one. In the analytics space, however, Big Blue does mean business. The announcement of its sizable new business analytics and optimization division is clearly intended to prove as much. Shortly after its announcement, IBM also unveiled a new stream computing platform called “System S” to much fanfare. The breathless enthusiasm of business journalists, technology bloggers and investment analysts has been palpable. But what exactly does this technological advancement do, and what does it mean for your business?

To answer this question, let’s begin briefly by dissecting what IBM has introduced. Imagine that you are receiving a continuous stream of data, such as stock prices on the Nasdaq. These figures must be quickly analyzed so that the proper buy and sell orders can be placed. Suppose that you also need to base your decisions not just on the Nasdaq prices but also the numbers figures coming in from dozens of other exchanges. Continue reading →

Cheating Your Way into Business Visibility

Posted on by

Several weeks ago, I wrote a post about how the pace at which the world is accumulating information exceeds our ability to critically evaluate it. For companies that make thousands or millions of marketing decisions every day in the form of price offerings, advertising placements and so on, this translates into making decisions that perpetually involve a greater amount of uncertainty relative to the amount of information we have. The root cause of this problem is a technological one: we do not have the computing power to slice and dice massive datasets in order to glean insight in time to support decisions. An even deeper explanation is that the gap between the rate of information accumulation in businesses and the pace of information transfer improvements will continue to widen at an increasing rate. This poses serious challenges to the capabilities offered by Business Intelligence, not to mention our ability to determine optimal prices. Continue reading →

What Happens When We Can’t Keep Up with Information

Posted on by

I ran into a former colleague the other day who, as it turns out, recently left his job and presently spends his days building options pricing models and trading from home on his own accounts. In turn, I described to him some of the recent work that we have done in revenue optimization and particularly the breakthroughs that we have engineered for processing data. His face scrunched up a bit, and his response was uncharacteristically blunt: “You can always process numbers quickly if you need to,” he smirked.

Not so, in fact. When you start asking extremely detailed questions that require combing through years of detailed historical data and then performing mathematical transformations on each of those figures, you will find out rather quickly the limits of processing speed when your results finish compiling in a week or so. The thing is that most of us never push up against the processing speed frontier. We can see that every year computers get faster, chips get smaller, and Excel seems to have more rows. Moore’s Law prevails. The trouble is that all the while the rate at which the data universe expands is screaming past advances in processing capabilities, and that rate does not fluctuate with the economic downturn. Consider the markets for microprocessors, which allow us to perform those calculations and manipulate data, and hard drives, which allow for storage of data. Microprocessor sales have been dealt a sharp blow by the global downturn as computer sales have slowed, but worldwide shipments of hard disk drives (HDDs) roughly maintained 2007 levels even in the worst quarters of the recession (and the drives themselves contain more memory). Solid state and flash memory shipments were down, but the evidence suggests that this is due to consumers substituting HDDs for other types of memory, rather than simply not storing more information. The demand for data storage, while not completely recession-proof, is nonetheless of the hardier variety.

Simply put, information of all kinds accumulates faster than we can analyze it. We are losing the race, and the gap is widening, not shrinking. As for what this ultimately means, I will now make a rather dour point. A fashionable explanation for the recession among both politicians and many “Main Street” types is that greed is what did us in. The greed of the bankers, the hedge funds, the fat cats, the small cats, whomever – greed is the culprit. But that doesn’t explain everything by a long shot. Even the greediest person doesn’t want the party to end and the money to stop coming in. Might it be possible that they weren’t able to ask the questions that might have led to certain debt instruments having never been created? Financial services employees have more information available to them than decision makers any other industry, and still here we find ourselves. Think about how many times each day similarly misinformed decisions are made inside corporations all across the world. The information is there, but we are more often than not letting it rot on the docks.

Wanted: Intelligence (Information Need Not Apply)

Posted on by

Professionals in the foreign intelligence community take pains to distinguish between information and bona fide intelligence. Any piece of knowledge, no matter how trivial or irrelevant, is information. Intelligence, by contrast, is the subset of information valued for its relevance rather than simply its level of detail. That distinction is often lost in sector of the enterprise technology industry that is somewhat loosely referred to as Business Intelligence, or BI. This has become a bit of a catchall term for many different software applications and platforms that have widely different intended uses. I would argue that many BI tools that aggregate and organize a company’s information, such as transaction history or customer lists, more often provide information than intelligence. The lexicon is what it is, but calling something “intelligence” does not give it any more value. In order to sustainably outperform the competition, a company needs more than a meticulously organized and well-structured view of its history. Decision makers at all levels need a boost when making decisions amidst uncertainty and where many variables are exerting influence. They need what I would call predictive intelligence, or PI – the ability to narrow down the relevant variables for analysis and accurately measure their impact on the probability of a range of outcomes.

What makes the distinction between information and intelligence critical is that information is getting more accessible by the day. This democratization of BI is evidenced nowhere more so than at Microsoft. In 2008, Microsoft unveiled several projects in the late stages of development that it claims will put BI capabilities at the fingertips of more users than ever before. “Project Madison” will massively increase Microsoft’s information storage capabilities, while the “Kilimanjaro” and “Gemini” projects together will provide easy-to-use reporting and analysis tools designed to drastically reduce the complexity of using traditional BI tools – all at very low cost compared to large-scale ERP implementation. The possibilities abound. But I still ask the question: what are all of these newly empowered users going to do with all of this information once they can access it at the push of a button?

I am excited by the idea of so many more information workers being able to ask the questions that end up driving businesses to continuously reinvent and perfect themselves, but I worry about relevance. Will these capabilities actually increase the amount of intelligence available to decision makers? Any business decision can be thought of as a bet that some desired future state will materialize as a result of a present course of action. Business intelligence tools as we know them more often than not do not help us make more intelligent bets when it comes to the future. The problem is that we think they do. More data often makes the task of identifying the true predictors of business success and isolating their effects more difficult. In order for a company to get the most out of its data, it needs PI as well as BI capabilities at the fingertips of decision makers. For marketing and pricing to become a more fact-driven corporate discipline, we must recognize the need not for more data, but ways of evaluating the probability of outcomes based on only the factors that matter. This is not child’s play. Information alone, however well-groomed, is simply not sufficient to meet this need.