Over the past couple of years, more than a few economy write-ups (including our editor's) have questioned the disparity between the numbers reported by Statistics Canada's two regular jobs reports, the Labour Force Survey (LFS) and the Survey of Employment Payroll and Hours (SEPH, which is scheduled for release Thursday morning). The surveys are different in nature, so there are many variables that could account for a disparity.
While the LFS surveys household members, the SEPH surveys their employers, making it a more reliable source of information specific to employers, such as the number of Canadians employed in each industry. Unfortunately, in recent years the SEPH has failed to classify many employers in the survey by industry, resulting in a spike in "unclassified" jobs – where it is unclear what industry these workers are employed in. The problem stems from a change in survey methodology, and has significant consequences.
Astute readers may notice that the headline SEPH employment figure, bold-titled "sector aggregate" – showing the number of employees by North American Industry Classification (NAICS) each month – isn't actually a sector aggregate. For the latest (October, 2013) SEPH release, the reported sector aggregate is 425,400 jobs greater than the sum of individual industry sectors, the difference down slightly from a peak of 438,200 in July, 2012. These discrepancies are the so-called "unclassified" business payroll jobs. Unclassified jobs in the October SEPH release accounted for almost as many jobs as four broad industry sectors (forestry and logging; mining, quarrying, oil and gas; utilities; and management) combined.
In a cost-cutting move following the early-1990s recession, Statscan significantly reduced the sample size of its Business Payroll Survey (BPS) – the sample survey used for SEPH – from 70,000 to 10,000. (It has since been increased to 15,000.) That sample today represents one million actual employers. Administrative (payroll) records provided by the Canada Revenue Agency (CRA) are used to make up for the reduced BPS sample size, transforming the SEPH from sample survey to part sample, part administrative data.
Sample surveys usually require at least one direct contact with a survey respondent before they are included in the sample. Not so with administrative data surveys. While the CRA records provide information on each business's total employment and gross payroll, the information provided about its primary activity isn't always sufficient for NAICS industry classification, according to Statscan. During an economic downturn, it becomes more difficult to reach some businesses (dissolutions, restructures and new formations) to follow up. The reduced sample size magnifies each unclassified business's weight in the SEPH results.
This swelling number of unclassified businesses likely contributed to increases in both the frequency and magnitude of SEPH data revisions. Our editor noted a decline of 30,200 jobs in the September, 2013, SEPH report, following a two-month (July-August) gain of 137,000 jobs. Both those figures have been subject to substantial revisions. The July-August gains were originally reported as 111,600. September's decline was subsequently revised down to 10,400. And those revisions pale in comparison to March, 2013, where an initial loss of 22,100 jobs became a gain of 29,826 – the latest revision cutting 8,865 unclassified jobs while adding 38,691 industry-classified jobs. The initially reported monthly job gains for 2013 to date (January to October) add up to 117,400 jobs; the revised gains add up to 174,310. Another major revision is scheduled in March.
The recent SEPH data revisions pose a few minor challenges. Major revisions can render the analysis provided on the release date meaningless. They also require updates to public, private and/or academic research and analysis using the data, with major revisions possibly altering certain findings.
The accumulated lump of nearly half a million unclassified jobs over a short period of time poses greater challenges. As troublesome as the unclassified jobs spike looks nationally, it's magnified at the provincial level, where more unclassified than industry-classified jobs were created in all but three provinces (Saskatchewan, Alberta and Newfoundland) between October, 2008, and October, 2012.
If those unclassified jobs were real and should have been classified into one or a few industries and/or provinces, any trend analysis using the SEPH data to forecast employment growth relative to the recession would be unreliable.
Since Statscan can't collect data on individual employee pay and hours from unclassified businesses – and jobs created during a recession tend to be more precarious – any trend analysis forecasting average pay or hours would, likewise, be unreliable.
Statscan has taken steps to suppress the growth of unclassified jobs in the SEPH. An electronic BPS questionnaire was introduced in December, 2012. This and other undisclosed remedial measures effectively change the survey methodology, and users should be informed of the break in the series. Likewise, users should also be informed if/when Statscan eventually undertakes a major revision to industry-classify the hundreds of thousands of unclassified jobs the SEPH has accumulated in recent years.
Longer term, Statscan needs to significantly increase the BPS sample size. As noted last month, the BPS is also used for Statscan's Job Vacancy Survey, which also suffers data quality issues stemming from the limited survey sample size.
The SEPH unclassified jobs boom, like the 2011 National Household Survey, serves as an example of overreliance on administrative data. As the federal government and Statscan examine the potential savings of switching more sample surveys to administrative data, hopefully the impact on data quality and accuracy is given greater consideration in the cost-benefit analysis.
Sam Boshra is an independent Montreal-based economist, and editor for EconomicJustice.ca.