Statistics Canada's release Tuesday of its latest Job Vacancy Survey (JVS) for January, 2014, provides an opportunity to revisit the federal Finance Department's Budget 2014 companion Jobs Report.
Despite Finance's job vacancy rate being thoroughly discredited as unreliable (largely due to the revelation that it was bolstered by postings from free classified-ad website Kijiji), Employment Minister Jason Kenney insists a measure of online job ads can be a viable proxy for labour demand, provided a few minor technical concerns are worked out.
But given the questionable methodologies and disparate numbers behind the various online-job-ad-based measures, it can't be.
Here is a selection of estimates for January, 2014, based on information provided by Finance and the Conference Board of Canada (which also produces online-ad-based vacancy reports):
- Conference Board’s revised Help Wanted Index “new” job ad count: 246,000
- Conference Board’s revised Labour Market Tightness Indicator “available” job ad count: 468,000
- Finance’s unrevised “available” job ad count: 775,000
- Statscan’s revised JVS “available” payroll job count: 193,000
A rate or index is only as reliable as the data and methodology used to produce it. Neither Finance nor the Conference Board provide that key information in their reports. Both declined requests for the information, citing contractual restrictions prohibiting them from sharing it.
Wanted Analytics, the data provider, considers its data and methodology proprietary. The company would neither disclose information on the job-postings websites it indexes nor the quality control measures it takes – information that would shed light on its data's consistency and reliability.
A few weeks ago, the Parliamentary Budget Office (PBO) published its Labour Market Assessment 2014. It reviewed the Conference Board data, which had just been revised to remove Kijiji from its online job posting count. While not cited by the PBO, the impact Kijiji had on the underlying job ad numbers was remarkable, reducing the January figure from 803,000 to 468,000 (42 per cent).
But even if Finance were to revise its job vacancy estimate accordingly, its online-ad-based measure would still remarkably overstate actual labour demand.
First, Finance included self-employment in its definition of labour demand, whereas Statscan's JVS definition only includes job vacancies with employers.
Second, in explaining its data source selection, Finance noted: "Wanted Analytics uses an algorithm that identifies unique advertised postings on the basis of the combination of company name, job title/description, and city or province."
Combining those details goes a long way in explaining the problem, best illustrated with an anecdote from Mr. Kenney's home province of Alberta. Shortly before Finance released its Jobs Report, Wanted Analytics reported that Alberta's top-advertising employer was Wal-Mart, but its top advertised job title was babysitter.
Finance's definition of labour demand, in line with that of Wanted Analytics, its selected data source, doesn't restrict the number of online job ads counted to only those posted by employers. Rarely do online ads looking for babysitters, for example, list a company name, let alone the same job description. Such ads also tend to be re-posted quite often.
The number of job ads included in a given measure differs wildly based on the specifications for the data set – whether one counts only "new" or all "available" ads, as well as how one defines "new" and/or "available." And while the Conference Board removed Kijiji from its online-job-ad sources, other similar websites – such as Craigslist, on which Kijiji was modelled – remain in its revised Wanted Analytics data set.
In short, different measures can produce wildly different estimates, depending on which websites are included and how job ads are counted. That speaks to a problem that goes far beyond "technical concerns."
While Finance's "Kijiji" Jobs Report helps explain Statscan's rationale for cancelling its Help Wanted Index back in 2003, it's worth noting that Statscan had dropped its original Job Vacancy Survey in the 1970s for the more "cost-effective" HWI. The resulting data vacuum – JVS was only re-introduced in 2011 – precipitated this whole mess.
This episode also highlights a couple of broader concerns.
One is government (ab)use of proprietary, third-party data. If a government can play keep-away with questionable data it has used to (mis)measure a key economic indicator, this raises red flags in terms of transparency and accountability. And the concern is not unique to this one measure, to Finance nor to the federal government.
There's also something to be said for the short-sighted, oft-repeated "cost-effectiveness" mantra. How much is the absence of a reliable key labour market indicator now costing the Canadian economy? Economists can't reasonably forecast labour demand at the national level, let alone by region, industry and/or occupation, without a reliable measure of what the trends were pre– and post-recession.
It was recently reported that the current federal government has spent half a billion dollars on private lawyers and hundreds of millions more on advertising. By contrast, government contract disclosures suggest, Finance spent $17,000 on unreliable data to produce a faulty job vacancy rate.
One can't help but question where the priorities lie.
Sam Boshra is an independent Montreal-based economist and editor for EconomicJustice.ca.