You Can’t Punt Away the Dimensionality Curse

A single mathematical formula brought ruin to the global financial markets.  What happened was not a failure of quantitative methods per se but rather a lesson in the perils of ignoring real-world complexities in favor of deceptively elegant shortcuts.

The fault, dear investor, lies not in the head of AIG’s Financial Products Group or members of the Bear Stearns Investment Committee or any other anthropomorphic entity: rather it was a single mathematical formula that apparently felled the pillars of global finance.  That’s the gist of a recent article in the 3.17 edition of Wired magazine entitled “Recipe for Disaster: The Formula that Killed Wall Street” by Felix Salmon.  The formula, known as a Gaussian copula function (when is the last time that term became a fixture of the public discourse?), purported to solve the mother of all securitization problems: establishing default correlation factors between the many constituents of the pools of mortgages and other credit obligations whose cash flows served as the underpinning for the complex derivative securities known as collateralized debt obligations (CDOs).  Awareness of the potential in this arcane formula helped power the CDO market to some $4.7 trillion in volume over the course of the housing bubble years of this decade.  As the Wired article explains, the explosive commercial viability of this formula can be explained by its use of a simple sleight of hand.  Rather than modeling out the default correlation implications of pools of thousands upon thousands of individual mortgage obligations – an extremely complex undertaking requiring powerful algorithms and massively robust computational processing technology – the CDO market’s Wall Street practitioners used a shortcut that appeared elegant but proved deadly: using the market price of credit default swaps (CDSs) as a proxy for the actual historical data.

What happened in essence was that the CDO market ran up against one of the most challenging of quantitative modeling problems: the dimensionality curse.  This refers to what happens in complex environments where numerous variables interact with each other and all of the resulting combinatorial possibilities influence the economic value.  The addition of an incremental variable to the pool exerts an exponential effect on the number of possible outcomes.  Think of a simple case: if you have a pool of two variables then the number of potential outcomes is four: add a third dimension (variable) to the mix and the potential outcomes expand to nine, and so on.  In an environment like pools of thousands of mortgage obligations or credit card receivables influenced by a bevy of macro- and micro-economic, behavioral, seasonal and other random factors there are literally billions of combinatorial outcomes that could affect the incidence, magnitude  and frequency of default events and hence the price of the CDOs whose economic value derives from those pools.   Getting to the right answers – and doing so with enough speed to satisfy the blistering pace of 24-7 investment markets every day – is a daunting challenge to say the least.  So when Daniel X. Li, a quantitative analyst at JPMorgan Chase, posited the use of CDS prices as a proxy for historical data in a 2000 paper published in the Journal of Fixed Income Securities, the CDO market rejoiced and basically punted away the dimensionality curse by using this shortcut.  The reasoning and the assumptions employed proved to be flawed and the disastrous results are entirely visible to the naked eye in all their graphic detail.

In quantitative methods as in life there are no free lunches.  You can’t simply punt away the dimensionality curse – you have to embrace it and try to achieve mastery over it using all the knowledge and technology tools at your disposal.  At Sentrana we deal with dimensionality curse problems every day – the demand markets for the products and services our clients sell are highly complex environments: tens of thousands of products for thousands of customers in hundreds of locations reachable by any number of marketing vehicles and sales channels.  Modeling these environments is not for the faint-hearted: but the problems are not impossible.  The computational technology does exist, as does the modeling science.  The critical ingredient is the will and determination of those who practice quantitative methods in business to forego the easy outs and stay focused on solving the real problems, however daunting.

Perhaps the field of quantitative methods needs a variation of the medical profession’s Hippocratic Oath: First of all, do no harm.  Clearly the Wall Street experiment egregiously failed that standard.  Let’s hope that the next time some arcane mathematical formula figures into the cultural Zeitgeist it will be for better, not for worse.

Subscribe   |