Government study reveals significant errors in voluntary census

Share

A study conducted by Statistics Canada weeks before Ottawa revealed its plan to scrap the mandatory long-form census found that significant errors can creep into survey results gathered on a voluntary basis.

The Harper government's early summer announcement that it would make the 40-page long-form census optional has caused a major rift between Ottawa and researchers, businesses and health officials who warn it will undermine the rich trove of data upon which they rely.

A June, 2010, internal study obtained by The Globe and Mail under the access-to-information law offers an inside look at how new census-taking rules could skew data in a range of areas from housing to demographics.

Statistics experts warn its findings demonstrate how minorities and groups such as renters could be measurably underrepresented or miscounted in the coming 2011 census.

The Statscan study, Potential Impact of Voluntary Survey on Selected Variables, was prepared with full knowledge of Ottawa's census change plans. It attempts to re-create what the 2006 long-form census questionnaire would have yielded had respondents not been compelled to answer it. These simulated results were contrasted with real 2006 census data.

Statscan researchers found the voluntary approach produced less accurate results - a problem that was especially significant in small population groups, according to outside statistics advisers who reviewed the report for The Globe.

"If the [long-form]questionnaire had been a voluntary survey in 2006, the picture of the population of Canada that would have emerged seems to be different for sub groups of the population based on citizenship, visible minority, language and education," the report said.

For instance, the real 2006 census long-form found that renting households as a percentage of the population in Canada had dropped by 3.08 percentage points from the 2001 census.

But when the Statscan study simulated the results of a voluntary 2006 long-form - which reflect the lower response rates expected in optional surveys - it got a markedly different answer. Calculations instead indicated that rented dwellings in Canada as a share of the population declined by 8.07 percentage points from 2001.

The difference - nearly five percentage points - suggests a voluntary survey in 2006 would have massively undercounted renting households.

"They would be gravely underestimating the renters," says Ian McKinnon, chair of the government-appointed National Statistics Council, which advises Canada's chief statistician.

"That's hundreds of thousands of rental units," he said.

Federal census-takers can manipulate numbers to compensate for underreporting if they know how census results differ from optional surveys, but Ottawa has conducted relatively few benchmark tests that compare the findings of voluntary and mandatory approaches.

Mr. McKinnon, who said he was speaking for himself only and not the council he chairs, said a more distorted picture of Canada will hurt governments' ability to act on issues.

"If you're the city of Toronto and you're very concerned about access to housing, you want to know how many rental units are out there. What if there's such a big error in your gold, benchmark estimate?"

In another example from the report, the real 2006 long-form census found that visible minorities as a share of the population increased by 2.77 percentage points between 2006 and 2001. The simulated voluntary approach would have reported an increase of only 0.74 percentage points.

Mr. McKinnon said this second example also demonstrates how a voluntary approach in 2006 could have miscounted visible minorities by hundreds of thousands.

The Statscan report's authors cautioned that their work was theoretical and didn't include all census data. They simulated the lower response rates expected in a voluntary survey by including only 2006 census responses gathered as of May, 2006, when the response rate for the mandatory long-form was 74 per cent. Other internal Statscan projections have estimated the best response rate from optional surveys is about 70 per cent.

Don Drummond, former chief economist at the Toronto-Dominion Bank and a member of the National Statistics Council, said the report "provides some validation to the concerns that have been expressed about the conversion to a voluntary [approach]- that the results would be heavily skewed toward certain groups and away from other groups."

Neither Industry Minister Tony Clement's office nor Statscan immediately responded to requests for comment.