Home > Finance and money > Super Quants and Super Models

Super Quants and Super Models

Some of you may have already heard of the “Quants”, the supposed supermen of financial engineering. The majestic modellers of risk management and liquidity management that within their black box algorithmic alchemy they alone hold the key to unlocking the wondrous stability and growth of the late 20th / early 21st century.

Well, it seems that their black box “black magic”, might have turned out to be little more than complete fiction:

“Scientists struggle with models in many fields—including climate science, coastal erosion and nuclear safety—in which the phenomena they describe are very complex, or information is hard to come by, or, as is the case with financial models, both. But in no area of human activity is so much faith placed in such flimsy science as finance.”

States an article just published in the Scientific American:

“Calibrating a complex model for which parameters can’t be directly measured usually involves taking historical data, and, enlisting various computational techniques, adjusting the parameters so that the model would have “predicted” that historical data. At that point the model is considered calibrated, and should predict in theory what will happen going forward.”

It seems that a scientist (of the genuine variety) undertook a test whereby he created some dummy data from an underlying statistical trend model and then using the output data generated by this model tried to re-create the original formula; much as in the way a financial quant would study past trends of financial data and create a well fitting trend formula to predict the future.

So what did he find? Well, he found that he wound up not with one correct answer, but numerous, almost correct answers:

“while these different versions of the model might all match the historical data, they would in general generate different predictions going forward”

In other words the models “seemed” OK when tested against the data they were “trained on”, but then failed when tested against new data, it’s future forecasts.

So when the new data arrives, and it doesn’t match the orginal forecast, what do the Quants do? Well they don’t say their model was incorrect, they just say that it needs “re-calibrating”:

“When you have to keep recalibrating a model, something is wrong with it. If you had to readjust the constant in Newton’s law of gravity every time you got out of bed in the morning in order for it to agree with your scale, it wouldn’t be much of a law. But in finance they just keep on recalibrating and pretending that the models work.”

Yes, so much “faith” is placed on these financial geeks, and their flimsy “science” and flimsy “models” by a naïve public and gullible politicians.

In fact referring to what financial quants call “models” is an insult not just to science but to real models like Jordan and Kate Moss too.

———————

NB: Scientific American may be a little late to the party in critiquing the Quants, as this film from 2010 explores the dubious underpinnings of high finance:

Quants – Alchemists of Wall St

However, the beauty of the Scientific American article is the way that the investigator used a pure laboratory experiment to conclusively prove the folly of assuming that formulae created from past data is not always sufficient to conclusively predict the future.

About these ads
Categories: Finance and money
  1. Charles Jurcich
    November 3, 2011 at 12:47 am

    Here is the reason why most “mainstream” economic models and forecasts fail:

    Since the 1970′s when mainstream economic theory gained credance, these economists relied on the idea that macroeconomics was just microeconomics, but scaled up. Thus they concluded that what was good for the individual had to be good for all individuals, and therefore the whole economy.

    This was in direct opposition to Keynes, who being something of a scientist, was aware of the scientific principle called ‘fallacy of composition’, which states that in many natural systems, if what benefits the individual is applied to all individuals, then the group is actually neutrally or negatively affected overall.

    Example: If I am sitting down watching a hockey game, I may choose to stand-up to get a better look. If everyone one watching the game chose to do the same, then no one would be any better off.

    In the real world, and with a fixed money supply, some people can save their money for a rainy day, but if everyone tried equally hard to save, no one would be able to save. Those that typically succeed, do so because others go into debt.

    Simply aggregating from micro to macro does not work – you have to stand back and see the bigger picture. This is why Keynes said that macroeconomics should be studied as a seperate discipline, because most of it’s conclusions are actually opposite to what happens to individuals and businesses. When individual agents start saving overall, government has to spend extra to compensate for the private sector’s lack of spending.

  2. November 4, 2011 at 8:44 am

    Charles

    That’s an interesting point. I think we can add that to the (growing) list of flaws of mainstream economics!

    See this post for more detail on the lack of understanding of where overall macroeconomic growth actually comes from:

    http://forensicstatistician.wordpress.com/2011/05/27/is-economics-a-real-science/

    Much as I respect Keynes, even he didn’t place growth in the context of energy and resources.

  3. December 15, 2011 at 9:32 am

    More detail on the flawed statistics at the heart of modern finance:

    “Recipe for Disaster: The Formula That Killed Wall Street”

    http://www.wired.com/techbiz/it/magazine/17-03/wp_quant?currentPage=all

  4. May 10, 2012 at 11:15 am

    A further description of this article from Scientific Amercian:

    http://www.scientificamerican.com/article.cfm?id=finance-why-economic-models-are-always-wrong

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: