When Time magazine economics columnist Justin Fox began his extensive research into the efficient market hypothesis, its role in the evolution of modern finance, risk management and investing, and its steep fall from grace, he had no idea that the world was about to be blindsided by a financial crisis of monumental proportions - or that the once-celebrated academic theory would wind up at the centre of the storm.
Now, his detailed history of an idea gone awry, The Myth of the Rational Market: A History of Risk, Reward, and Delusion on Wall Street (HarperCollins) , has the writer of Time's popular "Curious Capitalist" column trading quips with the likes of Jon Stewart on The Daily Show and sharing his thoughts with us on everything from learned investing lessons to the dangers of rapid financial innovation.
You have been working on this book for several years. Did what happened in the past year or so come as a surprise or was it a logical outcome?
It was more of a surprise than it should have been, looking back on the research I did. It's just hard to conceive of everything suddenly changing so quickly. But obviously, it all fits very well into the tale.
What's amazing to me is that since 1987, it should have been clear that there's this pretty much fatal flaw with quantitative risk management. Or else, there's an extreme limit to it. It works until something big happens. And yet people kept going back to doing it, each time with a bit more sophistication.
Is that because of the huge profits?
I guess so. I guess what it is is that you can make a lot of money in between [the market disasters]
Maybe they should bar the finance academics who created all these computerized models from setting up shop in the real world.
In general, they have a pretty poor record, percentage-wise. ... A lot of those who go out there and try to beat the market or tame risk run into trouble.
You point out that disasters have often stemmed from the introduction of new financial products for which there's really no analysis or history. Where are we heading now?
That's one of the crucial questions. There's this tradeoff between financial innovation and the risk of total financial collapse. And I think that was widely ignored for the past couple of decades.
You can go back to long before there were derivatives or computers. You look at crashes and downturns in the past, and very often it had to do with some new way of doing things financially. Because it had never been tested by a crisis, people didn't know how bad things could get.
My sense is that financial innovation is okay, and we want it. But we don't need to encourage it to go as fast as possible.
What's your assessment of the capacity of governments to regulate these vehicles, given their own shortage of talent?
And even if they did [have enough capable people] the regulators might often be subject to the same mood swings as the markets. It seems like the simplest answer is just keep leverage down. Then, everything else is more likely to take care of itself. You had the dot-com boom and bust, and it was definitely painful. But it was nowhere near an economic disaster. And it was partly because it was all built on equity.
It's when you combine these market swings with debt - when people get to borrow money based on crazy asset values, which happened with stocks in the late 1920s and with real estate now - that's when you have the big trouble.
It sounds as if the cure might be simpler than a lot of people realize: Just keep a lid on leverage.
Instead of trying to fine-tune things and have this perfect government regulation that avoids bubbles or whatever else, just accept that there are going to be occasional moments of irrational exuberance, but have some sort of simple rules for debt growth.
Don't we need global regulation to do that effectively?
If you have a global financial system, you have to have some amount of global agreement on some basics. I don't know that there needs to be one global regulator, if you could get all the big players to agree. It would be bad if you had extremely detailed global regulation. But if you had some simple rules, the harder it would be to game it, because it would be easier to see that people are doing that.
You weren't as hard on Alan Greenspan and his role in this mess as many others have been. Why is that?
In general, I'm a nice guy, I guess. I've met Greenspan a few times and find him very amusing to talk to.
How do you see his successor emerging from all this?
I think [Federal Reserve chairman Ben]Bernanke will come out badly in terms of his performance very early on. But since then, the most endearing thing about him is that he has been this realist. And his testimony has been kind of fun at times, because he really does acknowledge the limits of his knowledge.
You can judge people on long-run economic policies. But with crisis management, you have these "what ifs?" I have this sense that things could have been vastly worse, given the sort of imbalances that had been built up. Therefore, the Fed's performance ... is reasonably okay. But, obviously, if you don't buy that, then what he has done is a lot less impressive.
Is there a middle ground between the rational and the behavioural that would serve as an explanatory theory of how and why the markets work the way they do?
I'm sure there is, because even the pretty die-hard efficient markets people realize that prices aren't right all the time. I mean, they definitely realize it now. And even behaviourists don't see the world as this place where it's all about emotion and there's no relation to fundamental values of securities.
In reality, prices are always somewhere on this continuum between rational and crazy; and I think our perception of markets is always somewhere on that continuum.
One of the big lessons from this book is there is no one model of how the world works. And it just seems like the biggest mistake you can make is think that there is and just assume this explains everything.
What personal investing lessons did you take away from your research?
I actually moved more of my assets into index funds over the course of doing the book. ... To a certain extent, one of the great lessons of all of this is: No, the market's not unbeatable. There are actually people who possess the skill to beat it.
And it's not necessarily a skill that you have to have massive resources to exercise. But most of us can't.