Avoiding the “Irrelevant Elegance” Trap: Modeling for Practical Business Outcomes

Quantitative modeling is a creative process. There is as much art to modeling as there is science – choices about what relationships you want to express and how to express them. And just as with anything creative, the authors of quantitative models can take pride in the beauty of their creations. In the words of my colleague Ali Mahani, Sentrana’s senior quantitative modeler, models can be truly elegant – they can be things of beauty. But he adds that they can also be irrelevant – irrelevant to the particular business goals they are intended to serve. That presents a problem for enterprises seeking to elevate the role of quantitative insights in their decision making processes. Data and analytical methods are important tools in the arsenal of a modern enterprise. But decision makers would be wise to heed my colleague Ali’s advice: in using these tools, make sure to avoid the trap of “irrelevant elegance”.

Elegance does not always lead to the best outcomes

Elegance in modeling is expressed in the appearance of simplicity – rendering sprawlingly complex interrelationships in the real word into the clarity of precise mathematical formulae. Simplicity and elegance are all well and good, unless in the quest for this holy grail you wind up dramatically misrepresenting how things actually work in the environment you are trying to model. This can result in not only failing to solve the business problem at hand, but actually making matters worse than status quo ante by facilitating decisions based on incorrect assumptions. We have a real world example of just how much worse this can be in the financial markets debacle of 2008, when the elegant models crafted by the best and brightest quantitative experts Wall Street had to offer proved to be fatally flawed in the assumptions and heuristics they used to express the variables affecting housing prices, interest rates and mortgage payment trends. Perhaps modelers need to live by something like the Hippocratic oath taken by medical doctors: first of all, do no harm. Read the rest of this entry »

Subscribe   |  

Quantitative Intuition II: The Bayesian Brain’s Achilles Heel

In a previous posting (“Quantitative Intuition: It’s Not Counterintuitive”) I described some of the advancements that have been made in bringing together the disparate worlds of quantitative methods and human intuition, ending on the rather happy note that advanced scientific micromarketing models today are capable of introducing qualitative human judgment and experience into quantitative models, such that the models are able to “learn” from humans about important factors such as competitive threats, nuanced negotiation strategies and even meteorological vagaries – factors that traditionally have been difficult to crunch into the binary 1s and 0s of machine language.  The human brain works in a hierarchical manner, embedding propositions within propositions to think a potentially infinite number of thoughts.   In the example I used in the last posting, a sales rep who reads about a national wholesaler coming to town to open a discount distribution center can nearly instantaneously form a series of mental propositions to evaluate the importance of that news and the probability of potential outcomes that may (or may not) require decisive competitive action from the sales rep’s firm. Read the rest of this entry »

Subscribe   |  

In Economic Modeling, Can Hindsight Lead to Foresight?

In thinking more about my last posting here on failed Wall Street quant models and the dimensionality curse I started to wonder whether we could ever be more than the archetypal Monday morning quarterbacks:  commenting brilliantly on all the reasons why X should never have happened, after X has already happened and done its damage.  Can the mistakes of hindsight lead to foresight?  In other words, can we apply foresight to develop “good” economic models that won’t blow up in our faces?

In trying to answer this postulation we must go back to examine the eternal challenge of good modeling: how to create a simplified representation of reality that in ignoring many real-world features still manages to convey an inherently robust facsimile of the real thing.  For example, one of those maps of New England you buy at Exxon gas stations can serve as a good model for getting you from Hartford, CT to Boston, MA even if it ignores most of the streets and alleyways and other real-world detail that exist along the route.  In their book “Complex Adaptive Systems: An Introduction to Computational Models of Social Life” John H. Miller and Scott E. Page observe that the “ability to ignore is a crucial component of scientific progress”, using the image of a parent’s being able to respond to the incessant “why” questions of a three year old child by saying “just because”.  The trick, as the authors point out, is knowing when (and perhaps more importantly when not) to say “just because”.

While I wholeheartedly agree with that assertion I don’t think that it quite gets us to an adequate level of comfort in applying foresight to the creation of good models.  In his fascinating book “The Origin of Wealth” Eric D. Beinhocker points out that economic modeling took what many consider to be a wrong turn back in the latter years of the 19th century when leading thinkers of the day like Leon Walras and William Stanley Jevons borrowed heavily from the referential context of physics to create models for explaining economic activity, including such notable concepts as a mathematically representable state of equilibrium that continue to serve as the conceptual foundations of modern economics textbooks.  As Beinhocker elaborates, the problem with these models was that some of their fundamental assumptions – like the perfect, robot-like rationality of human beings in making economic choices – didn’t seem to simplify reality as much as contradict reality.  Thus we find ourselves in the present ruminating over the precise, mathematically elegant language of physics and the complex, evolutionary language of biology and debating whether a choice of the wrong science by the founding fathers of economics back in the 19th century led to the failure of models to adequately explain much of what is going on in the economy today and in particular the string of boom-bust upheavals that have become part and parcel of the last 20-odd years of economic activity.

I still don’t think we are there yet in getting closure on the foresight question, but we may be getting closer.  To tie in the strands of thought presented by Miller & Page and Beinhocker, when we get to those basic defining assumptions, Read the rest of this entry »

Subscribe   |