Make no mistake: Mr. Market is smarter than you

The average investor doesn't get much respect.

If we aren't being accused of inflating bubbles and chasing trendy stocks, we're denigrated as panicky when the market corrects and contrarian indicators when the market moves higher.

But if that's the case, why is the stock market so hard to beat? After all, the market reflects the actions of all investors. And if individual investors are always doing the wrong things, it can't be too hard to triumph over them.

It's a question that many investors reject outright, simply because they don't see the market as a difficult hurdle to clear.

Since small investors are rubes, they believe, and the pros are often hampered by restrictive mandates and quarterly performance benchmarks, success is relatively easy to achieve with a little know-how.

Don't believe it. The evidence is overwhelming that the market is one tough opponent.

Standard & Poor's has been tracking the performance of active mutual funds for more than 10 years, and found that over five-year time horizons nearly 62 per cent of actively managed funds failed to beat the S&P 500, after accounting for fees. Small-cap funds have fared no better.

As for individual investors, the results are worse, even without having to account for fees.

"The vast majority of individual investors will not beat the market through skill," said Terrance Odean, a finance professor at the University of California, Berkeley, who has studied investor behaviour extensively.

"So if they beat the market, it will be through luck – and luck can go either way."

Besides lacking the skills to outperform, the instincts of individual investors also tend to work against them. That's why Mr. Odean found that investors who traded stocks most actively underperformed the market more than those who traded infrequently.

Nonetheless, you would think that by making a few good decisions – avoid Facebook Inc.'s initial public offering and other stocks that trade at more than 100 times earnings, hold solid dividend-paying stocks and pounce on steep market downturns – you might have an edge.

In the United States, though, the stock market is dominated by mutual funds, hedge funds, pension funds and endowments, run by smart people working full-time and employing sophisticated trading techniques. Canada is similar.

According to some estimates, high frequency trading – essentially robots that can move in and out of stocks in a fraction of a second – now account for about 60 per cent of all trading volume.

Put another way, the pros define the market. And if they have a tough time beating it, after accounting for fees, the chances of an individual whistling his or her way to success with little more than a laptop and an hour or two in the evening is even more remote.

Here's a particularly nerve-rattling way to think about it: Buy a stock and an institutional investor is probably selling it to you, and aiming to profit from the exchange. Sell a stock and an institutional investor is probably buying it from you.

Discouraged? Don't be.

Since beating the market consistently over the long term is nearly impossible for most investors, the best solution is to give up. Simply buy a low-cost mutual fund or exchange-traded fund that tracks an index, such as the S&P 500 or the S&P/TSX composite index.

"If an investor bought an index fund tied to the S&P 500, I would say that that investor is a very good investor, but with a zero-per-cent chance of beating the market," Mr. Odean said.

"In other words, I don't think beating the market should be the objective of most investors."