Physics Envy: Pervasive, But Not Incurable

Everywhere you look, it seems, people are talking about “physics envy”.  This derisive term mocks the attempt of economists and other social sciences practitioners to imbue their disciplines with the equations and mathematical rigor of physics – a rigor that many believe fails when applied to the messy environments of disciplines like sociology or economics.  It’s not a new term – economist Philip Mirowski contributed to the Finnish Economic Papers series way back in 1992 with a piece entitled “Do Economists Suffer from Physics Envy?”

kinetic energy, not supply & demand

Eighteen years later the answer from many observation posts along the byways of public discourse appears to be: yes, they most certainly do, and so do their fellow travelers, business and financial markets experts.  After all, we just barely survived the most devastating economic event of our times, deeper and more far-reaching than any downturn since the Great Depression, and all the high priests of the field can do is shake their heads and say “wow, I sure didn’t see that coming.”  Distrust of fancy math is rampant in all walks of business life.  That presents a real problem for enterprise decision-makers at a time when they need smart quantitative tools – yes, fancy math and all – more than ever.  Markets are more complex than at any time in human history.  Giant waves of transactional data inundate marketing managers with new information every day.  Managers need science to help them gain valuable insights into the markets for their products and services – but how do they know that the growing number and variety of scientific marketing tools out there aren’t infected with the nasty symptoms of physics envy? Read the rest of this entry »

Subscribe   |  

Quantitative Intuition II: The Bayesian Brain’s Achilles Heel

In a previous posting (“Quantitative Intuition: It’s Not Counterintuitive”) I described some of the advancements that have been made in bringing together the disparate worlds of quantitative methods and human intuition, ending on the rather happy note that advanced scientific micromarketing models today are capable of introducing qualitative human judgment and experience into quantitative models, such that the models are able to “learn” from humans about important factors such as competitive threats, nuanced negotiation strategies and even meteorological vagaries – factors that traditionally have been difficult to crunch into the binary 1s and 0s of machine language.  The human brain works in a hierarchical manner, embedding propositions within propositions to think a potentially infinite number of thoughts.   In the example I used in the last posting, a sales rep who reads about a national wholesaler coming to town to open a discount distribution center can nearly instantaneously form a series of mental propositions to evaluate the importance of that news and the probability of potential outcomes that may (or may not) require decisive competitive action from the sales rep’s firm. Read the rest of this entry »

Subscribe   |  

Quantitative Intuition: It’s Not Counterintuitive (Nor an Oxymoron)

Think of the best salesperson you know: if you’re fortunate, perhaps someone in your company or, less happily, in a competitor’s firm.  What are the qualities that make this person excel at the job of sales?  In a classic Harvard Business Review article “What Makes a Great Salesperson” (July-August 1964) David Mayer and Herbert Greenberg likened a star salesperson to a heat-seeking missile: “Sensing what customers are feeling, they [the sales stars] are able to change pace, double back on the track, and make whatever creative modifications might be necessary to home in on the target and close the sale.”   Whereas most of us have intuitive abilities to a greater or lesser extent, excellent salespeople lever this intuition with strong empathy skills (sensing what the customer’s needs are) and the relentless personal drive necessary to cross the finish line.  If they could, managers would bottle this elusive elixir of talents and have all their salespeople drink it, every morning of every day. Read the rest of this entry »

Subscribe   |  

Models Didn’t Bring Down Wall Street; People Brought Down Wall Street

“Burn the mathematics” wrote economist Alfred Marshall in a letter to a friend, musing about the proper role of mathematics and scientific inquiry in the field of economics.  That 19th century cogitation would seem to be a prêt-a-porter soundbite for these latter days of the 21st century’s first decade – a time in which the mathematical infrastructure that underpins longstanding economic and financial theories stands accused of all manner of malfeasance, particularly given its presumed role in the decade’s signature economic event – the financial market meltdown of 2008.  The logic behind the accusation goes roughly thus: More complex (but not necessarily more “accurate”) models allow for more complex instruments to be created. Increased complexity means it takes more time to process and then fully comprehend what the numbers may be telling you. At the same time, though, technology allows buy and sell orders to be executed almost instantaneously through electronic trading systems. Time is of the essence, and ponderously complex computations simply won’t do.  A seemingly elegant (and fast, and commercially viable) shortcut is discovered and becomes the currency of the day. The models’ outputs come to be trusted blindly simply because there is no time to question them (and too much money to be made by using them). The impenetrable Greek letters obfuscate the sensitivity of the models to changes in important assumptions – which is fine for a few years because those assumptions (e.g. rising housing prices) don’t change – but then all of a sudden they do. The models start losing more money than they make. Then the chasm widens further as the high levels of leverage in the system make themselves felt. The losses accelerate dramatically, wiping out years of profits in just a few months. Burn the mathematics, indeed.

But let’s take a different look at this apparent tight coupling of mathematics and dire outcomes. Our recent correspondence with an author who has been widely published on the subject of Wall Street’s use of mathematical models recently offered to us an interesting opinion. His point was that the problem with the models was not so much their complexity, but rather that they were models in the first place. His argument was that you can’t ever perfectly hedge model risk.  Now, I agree with that observation: a model by definition selects some aspects of reality to represent and omits others, and the choice of what to include and what to omit is subject to human error, therefore fallible and not perfectly hedgable.  But I take issue with the idea that the fault lies in the existence of the models themselves.  Models can be misused – I think that much is clear. But the notion that models are all doomed to failure obscures a deeper truth about the goals of predictive modeling; namely that you can seek either to reduce the world or truly explain it. By trying to elegantly reduce the world to as few predictor variables as possible, you are more likely to be sowing the seeds of future failure, because complexity and actual drivers of outcomes are taken out of the equations to make them more solvable (or perhaps sellable, as in the case of the Gaussian copula function that was behind Wall Street’s demise, as we discussed in a previous posting “You Can’t Punt Away the Dimensionality Curse”). Predictive modelers don’t have to go down that road, however: they can also set out with the goal not of reducing an entire system to a single neat, tractable equation, but to quantify and explain all of the relationships that dictate outcomes to the absolute fullest extent possible. Tractability and computability are things to address later in the process, through technological means, but they should not dictate the fundamental mathematical approach at the outset. Read the rest of this entry »

Subscribe   |  

In Economic Modeling, Can Hindsight Lead to Foresight?

In thinking more about my last posting here on failed Wall Street quant models and the dimensionality curse I started to wonder whether we could ever be more than the archetypal Monday morning quarterbacks:  commenting brilliantly on all the reasons why X should never have happened, after X has already happened and done its damage.  Can the mistakes of hindsight lead to foresight?  In other words, can we apply foresight to develop “good” economic models that won’t blow up in our faces?

In trying to answer this postulation we must go back to examine the eternal challenge of good modeling: how to create a simplified representation of reality that in ignoring many real-world features still manages to convey an inherently robust facsimile of the real thing.  For example, one of those maps of New England you buy at Exxon gas stations can serve as a good model for getting you from Hartford, CT to Boston, MA even if it ignores most of the streets and alleyways and other real-world detail that exist along the route.  In their book “Complex Adaptive Systems: An Introduction to Computational Models of Social Life” John H. Miller and Scott E. Page observe that the “ability to ignore is a crucial component of scientific progress”, using the image of a parent’s being able to respond to the incessant “why” questions of a three year old child by saying “just because”.  The trick, as the authors point out, is knowing when (and perhaps more importantly when not) to say “just because”.

While I wholeheartedly agree with that assertion I don’t think that it quite gets us to an adequate level of comfort in applying foresight to the creation of good models.  In his fascinating book “The Origin of Wealth” Eric D. Beinhocker points out that economic modeling took what many consider to be a wrong turn back in the latter years of the 19th century when leading thinkers of the day like Leon Walras and William Stanley Jevons borrowed heavily from the referential context of physics to create models for explaining economic activity, including such notable concepts as a mathematically representable state of equilibrium that continue to serve as the conceptual foundations of modern economics textbooks.  As Beinhocker elaborates, the problem with these models was that some of their fundamental assumptions – like the perfect, robot-like rationality of human beings in making economic choices – didn’t seem to simplify reality as much as contradict reality.  Thus we find ourselves in the present ruminating over the precise, mathematically elegant language of physics and the complex, evolutionary language of biology and debating whether a choice of the wrong science by the founding fathers of economics back in the 19th century led to the failure of models to adequately explain much of what is going on in the economy today and in particular the string of boom-bust upheavals that have become part and parcel of the last 20-odd years of economic activity.

I still don’t think we are there yet in getting closure on the foresight question, but we may be getting closer.  To tie in the strands of thought presented by Miller & Page and Beinhocker, when we get to those basic defining assumptions, Read the rest of this entry »

Subscribe   |  

You Can’t Punt Away the Dimensionality Curse

A single mathematical formula brought ruin to the global financial markets.  What happened was not a failure of quantitative methods per se but rather a lesson in the perils of ignoring real-world complexities in favor of deceptively elegant shortcuts.

The fault, dear investor, lies not in the head of AIG’s Financial Products Group or members of the Bear Stearns Investment Committee or any other anthropomorphic entity: rather it was a single mathematical formula that apparently felled the pillars of global finance.  That’s the gist of a recent article in the 3.17 edition of Wired magazine entitled “Recipe for Disaster: The Formula that Killed Wall Street” by Felix Salmon.  The formula, known as a Gaussian copula function (when is the last time that term became a fixture of the public discourse?), purported to solve the mother of all securitization problems: establishing default correlation factors between the many constituents of the pools of mortgages and other credit obligations whose cash flows served as the underpinning for the complex derivative securities known as collateralized debt obligations (CDOs).  Awareness of the potential in this arcane formula helped power the CDO market to some $4.7 trillion in volume over the course of the housing bubble years of this decade.  As the Wired article explains, the explosive commercial viability of this formula can be explained by its use of a simple sleight of hand.  Rather than modeling out the default correlation implications of pools of thousands upon thousands of individual mortgage obligations – an extremely complex undertaking requiring powerful algorithms and massively robust computational processing technology – the CDO market’s Wall Street practitioners used a shortcut that appeared elegant but proved deadly: using the market price of credit default swaps (CDSs) as a proxy for the actual historical data.

What happened in essence was that the CDO market ran up against one of the most challenging of quantitative modeling problems: the dimensionality curse.  This refers to what happens in complex environments where numerous variables interact with each other and all of the resulting combinatorial possibilities influence the economic value.  The addition of an incremental variable to the pool exerts an exponential effect on the number of possible outcomes.  Think of a simple case: if you have a pool of two variables then the number of potential outcomes is four: add a third dimension (variable) to the mix and the potential outcomes expand to nine, and so on.  In an environment like pools of thousands of mortgage obligations or credit card receivables influenced by a bevy of macro- and micro-economic, behavioral, seasonal and other random factors there are literally billions of combinatorial outcomes that could affect the incidence, magnitude  and frequency of default events and hence the price of the CDOs whose economic value derives from those pools.   Getting to the right answers – and doing so with enough speed to satisfy the blistering pace of 24-7 investment markets every day – is a daunting challenge to say the least.  So when Daniel X. Li, a quantitative analyst at JPMorgan Chase, posited the use of CDS prices as a proxy for historical data in a 2000 paper published in the Journal of Fixed Income Securities, the CDO market rejoiced and basically punted away the dimensionality curse by using this shortcut.  The reasoning and the assumptions employed proved to be flawed and the disastrous results are entirely visible to the naked eye in all their graphic detail.

In quantitative methods as in life there are no free lunches.  You can’t simply punt away the dimensionality curse – you have to embrace it and try to achieve mastery over it using all the knowledge and technology tools at your disposal.  At Sentrana we deal with dimensionality curse problems every day – the demand markets for the products and services our clients sell are highly complex environments: tens of thousands of products for thousands of customers in hundreds of locations reachable by any number of marketing vehicles and sales channels.  Modeling these environments is not for the faint-hearted: but the problems are not impossible.  The computational technology does exist, as does the modeling science.  The critical ingredient is the will and determination of those who practice quantitative methods in business to forego the easy outs and stay focused on solving the real problems, however daunting.

Perhaps the field of quantitative methods needs a variation of the medical profession’s Hippocratic Oath: First of all, do no harm.  Clearly the Wall Street experiment egregiously failed that standard.  Let’s hope that the next time some arcane mathematical formula figures into the cultural Zeitgeist it will be for better, not for worse.

Subscribe   |