It's disconcerting to learn that Statistics Canada miscalculated its latest employment data so badly that it had to rescind one of its most important and influential economic reports just days after issuing it. But even once Statscan fixes what it called an "error in processing" and reissues the July Labour Force Survey (LFS) this Friday, the estimate of the number of jobs gained or lost likely still won't be all that accurate. Month in and month out, Statscan's flagship labour market report is showing us an entire dartboard – and largely guessing where the bull's eye is.
Statscan readily admits this. Yes, it's in the fine print of the report, but it's no secret that the statistical margin of error on the monthly LFS – essentially a telephone survey of 54,000 households – routinely dwarfs the actual reported change in the number of jobs from month to month.
The standard error in the monthly job count – the margin by which, statistically speaking, the survey sample could differ from the reality across the entire population – is 28,900. That means the July count of a meagre 200 net new jobs, which now appears to have been erroneous, could have been well off the mark anyway – from a gain of 29,100 jobs to a loss of 28,700 jobs, and anything in between.
And even at that, there is only a 68-per-cent statistical probability that the actual number was within this range. Want something approaching certainty? There's a 90-per-cent chance that the actual job count was within plus- or minus-46,000 of what was reported.
In fact, Statscan advises that you should only consider the change in job count to be meaningful if it exceeds the margin of error. Otherwise, the data may be no more than a bit of statistical noise. And in the past two years, you can count on one hand the number of months that the job gains or losses exceeded 46,000.
There are other problems. About 10 per cent of all households contacted don't provide responses. And participants may not always provide the most accurate interpretation of their own employment status – no small concern when nearly 70 per cent of Canada's job gains in the first six months of this year came from people describing themselves as "self-employed."
It should be noted that the United States doesn't rely on a household survey to estimate its monthly job numbers. Oh, it conducts one, as a secondary measure; but its primary labour market numbers come from payroll data gathered from employers and government agencies. As such, they are a more accurate measure of actual payroll employment – they reflect things like paycheques issued and taxes paid.
Actually, Statscan does this, too – it's called the Survey of Employment, Payrolls and Hours (SEPH), and it, too, is published monthly. It relies even more heavily on government payroll-deduction data than the U.S. report, and is considered a highly reliable measure of the labour market.
Notably, the SEPH has diverged considerably from the LFS in recent months. For the first five months of this year, the SEPH shows that employment rose by just 33,600 jobs, or an average of 6,700 a month. But according to the LFS, employment, the gains were almost double that – 62,200, or an average 12,400 a month.
The SEPH's big drawback is its lack of timeliness. SEPH data lag behind the LFS release by nearly two months; Statscan won't publish the SEPH for June until Aug. 28. Markets don't want to wait that long for such a key measure of economic health.
David Watt, chief economist at HSBC Bank Canada, suggests that Statscan should deliver the SEPH at the same time as the LFS, as the U.S. does in its employment report.
He said the SEPH would probably have to rely more on its survey data from businesses and less on Canada Revenue Agency payroll-deduction information if it were to be released that much earlier, which might make it somewhat less precise than it is in the current format. "However, we believe that releasing LFS and SEPH concurrently would provide valuable insight into the state of the job market and more than compensate for that loss of precision," he said in a note to clients.
Regardless of how it's achieved, there's a strong argument both for giving the SEPH more prominence, and for Statscan to find a way to bring forward its release time. As it stands now, the LFS's timeliness is seriously compromised by its imprecision.