Wilf Falk knew instantly that the population estimates he was looking at meant trouble for Manitoba.
It was April, 2013, nearly two years after census day, and Statistics Canada had gathered the country’s top provincial statisticians to review the revised, adjusted and final population figures. Mr. Falk could see at a glance that the revised results were too low. In his 34 years as Manitoba’s chief statistician, he’d never seen anything like it.
Over the next six months, he asked Statistics Canada to check and recheck its figures. They did, but the result didn’t improve. He tried to keep the fight “within the statistical family,” but Statscan wouldn’t budge. The stakes were high. The dispute centred around just 18,000 people, but losing those people would cost Manitoba nearly half-a-billion dollars over five years – no small amount in a province where the annual budget is $14-billion. In the arcane world of official statistics, head counts equal currency.
“The buck stops with me on the statistical side,” Mr. Falk said. “It was my duty to say, ‘This is not right.’… Numbers have consequences.”
As the weeks passed it was clear the two sides were at an impasse. Time was running out. Statscan maintained it had applied the method correctly.
In late autumn, the final version of a document called the certificate of population landed on the desk of Wayne Smith, Canada’s chief statistician. That single table, Mr. Smith said, has the power to move millions and billions of dollars. The document showed that Manitoba’s population would remain at its new, lower level. Mr. Smith signed his name at the bottom and sent it to the Minister of Finance. Manitoba was instantly $100-million poorer.
The census of population is meant to be an exact, or nearly exact, reflection of Canada’s population on one given day, in this case May 10, 2011. The census also determines the baseline for all future population estimates. But the census can’t reach everyone.
To determine how many people were missed, and how many people were counted twice, Statscan developed coverage studies. After every census it sends a team into the field to track down a sample of Canadians in every province to see if they were counted. This is called the reverse record check (RRC) and it’s done by a team of 12 full-time employees, supplemented by about 20 part-timers. In some cases, they act like private eyes, using phone books, Facebook and conversations with neighbours to try to find people. Manitoba’s coverage sample involved 6,000 people. The rate at which these people are found, or not found, determines the size of the statistical adjustment to the provincial population.
In past years, many people in Manitoba were missed. It has a large aboriginal population and aboriginal people tend to be missed at higher rates. Immigrants tend to get missed, and Manitoba had its highest levels of immigration in decades between 2006 and 2011. In 2011, the province also faced massive flooding that forced many people from their homes. Yet once the results of the reverse record check were complete, Statscan concluded that the adjusted population was only 1,233,728. A year earlier, it was thought to be 1,251,690.
But when they looked more closely at that sample, they examined something called the T-statistic, which acts as a test of statistical accuracy. Manitoba’s T-statistic was extremely high, “way out of bounds,” Mr. Falk said. (Manitoba’s was 3.35. Next highest was Alberta at 1.61). It points to a bad sample in the reverse record check, he said.
“It’s the canary in the coal mine,” he said. “The probability of getting a more extreme result than we observed in 2011 … is nearly non-existent.”
Statscan agreed there was something unusual. “We took a rigorous look at this,” Mr. Smith said. “We found nothing, and we went over it with a fine tooth comb.”
Manitoba’s Finance Minister Jennifer Howard said she has raised the issue personally with Finance Minister Jim Flaherty, but so far has received no indication of a remedy. Every year, Manitoba will lose $90-million in equalization money and $10-million in health and education transfers. Ms. Howard is proposing a joint panel examine the census result.
Doug Norris, a former director general at Statscan and one of Canada’s leading experts on the census, said there is a red flag in the numbers, but it’s not clear what Statscan could do about it. Any revision would be outside the already agreed-upon methodology.
“What do you do?” Mr. Norris said. “Here the problem is there’s a lot of dollars at stake. That’s what makes what’s essentially a small error so important.”
Mr. Smith said Statscan stands by its work. “We’re satisfied there’s no evidence that would justify us modifying in any way the population estimates we published,” he said.