Statistics Canada recently released the last batch of data from its National Household Survey that replaced the cancelled long form census in 2011.
The reaction to the data, particularly the income data just released, has been universally negative. Statistics Canada itself cautions using it and has not linked these data to the past censuses. Others have used somewhat stronger language calling it “garbage”, “dangerous” and “useless”. The question arises as to why a government would waste so much of tax-payers money to get something of so little value. Even The Globe decided to write an editorial, concerned about the message these data give about poverty.
Important as all these concerns are, the quality of the NHS data just released is the least of the problems caused by the cancellation of the long form census.
There are two much bigger issues to worry about. The first relates to the loss of the anchor that a census provides in adjusting other survey data. That means the loss of the census would affect data quality of many surveys to come in the future. Second, and in my mind the most serious consequence of canceling the census, is the loss of trust in Statistics Canada to be independent of government interference. It does not, in fact, matter whether interference is actually happening or not. Perception is all that matters.
The role of a census as an anchor is hugely important. The census is like a doctor who is alone looking after the health of citizens in a community. When someone is not well you go see the doctor to get treated. Similarly, biases in many voluntary surveys (like patients) that Statistics Canada undertakes need to be dealt with through adjustment with the help of the census (like a doctor) to make their results representative of the phenomena they are trying to capture. Take away the doctor from the community and patients have to help each other out when they get sick. God help us.
On the question of trust in the agency producing data, the vast majority of data users, even the sophisticated ones, are not in a position to determine for themselves how good or bad the data are. They use them and base their decisions on them, but only to the extent that they trust the organization that produces them. Trust in the agency depends on two critical factors: how good the Agency is in doing its technical work – and here Statistics Canada is second to none in the world; and do citizens believe that the government is not telling the agency what numbers to produce. Take away the independence of and trust in the agency, and even good data become less useful.
Todd Hirsch wrote of his experience of someone questioning him on the accuracy of Canadian inflation-rate data produced by Statistics Canada and argued that these data could not be trusted. As proof, she cited the government’s behaviour in canceling the census. He went on to say it wasn’t the first time he’d been questioned on the reliability of official data. I can say I have the experience of being told that many times as well by some of the most highly educated Canadians since the cancellation of the census.
This is serious enough that we need to fix the flaws in the laws governing Statistics Canada which leave decision making on technical statistical issues in the hands of a Minister. We have a model in the Canadian government we can follow. Ask yourself: are data as important for the functioning of a civilized democratic society as the setting of interest and exchange rates and the collection of your tax dollars? If the answer is yes, and it should be, Statistics Canada should become like the Canada Revenue Agency.
The role of a Minister then would be to ensure that the Agency is accountable to Parliament, but on all technical and statistical issues, the Chief Statistician would be the boss. It will be he who would decide on issue such as the size of the sample and the techniques to be used to deal with sampling and estimation errors and the non-response bias. It will be he who would decide which questions to ask in the census following consultations with data users. It will be he who would have to answer questions on the accuracy and reliability of data. And, of course, it will be he, rather than a Minister, who would need to explain whether or not a voluntary survey is as good as the census.
Munir Sheikh, a former Chief Statistician of Canada, is an Executive Fellow at the School of Public Policy, University of Calgary.
Follow us on Twitter: