Skip to main content

Editor's note: Statistics Canada announced on Tuesday, Aug. 12, that there was an error in the jobs report data they released on Aug. 8. Read more.

Did you catch that horrible jobs report? The mere 200 jobs created across all of Canada in July fell spectacularly short of economists' predictions for 20,000 new positions, adding to evidence that a) Canada's jobs-generating machine is in rough shape, and b) economists' forecast-generating machine is in even rougher shape.

The first is a surprise; the second, not so much.

Canada's best and brightest prognosticators have had an underwhelming track record, especially since the financial crisis. The Bank of Canada, for instance, has confidently predicted that the national economy will return to potential over the next two years. The only problem is that it's been saying that for four-and-a-half years.

To be fair, the Department of Finance's survey of private-sector forecasters show that most of them were also confident back in 2010 and 2011 that growth would be bounding ahead in 2012 and 2013. Oh, if only.

The official explanation for the misses is that any forecast contains a margin of error. Or to put that into layman's terms: We guessed wrong.

Of course, part of the forecasting problem is the inherent difficulty of predicting the course of anything so complex as a national economy. But there are other land mines here that might escape the attention of anybody but those annoying folks who make a habit of reading footnotes to statistical reports.

Let's look, for instance, at those dismal jobs numbers.

The mere fact that a national survey can say 200 jobs were created – not 199! not 201! – leads to the impression that employment growth is being monitored with laser-like precision. In fact, economists are dealing with randomness, inside volatility, wrapped in uncertainty.

As Statistics Canada will tell you, the Labour Force Survey is based on a monthly survey of about 54,000 households. While it's representative of the working-age population, it's not perfect. The standard error in the survey – a measure of the sampling error – is equivalent to about 28,900 jobs.

Statistics fans will know what that means: There's a 68 per cent chance that the true number of new jobs in any month is within 28,900 of the reported figure. So the reported figure of 200 new jobs in July gives us reason to think that the true figure was somewhere between a loss of 28,700 jobs or a gain of 29,100 jobs. Probably.

It's not surprising that economists misfire when trying to pluck a forecast out of the air for a number that is not all that precise. But their problems are indicative of a more general issue with expert forecasting: It doesn't work.

Philip Tetlock of the University of Pennsylvania studied almost 28,000 predictions from 284 economists, political scientists, intelligence analysts and journalists. He found their average forecasts to be only slightly better than random guessing.

The lack of forecasting accuracy is especially noticeable at key turning points. By and large, the economics profession failed to provide any warning of the financial crisis.

Does that mean we should give up forecasting? Not at all. Forecasts provide a reality check on our assumptions and give us a chance to test our mental models of how the economy is working (or not).

So, by all means, let's forecast. Let's also address Canada's faltering jobs market. But, in doing so, let's make a forecast that is sure to be accurate: Predictions will fail.