Polls pollute our democracy? NonsenseBrian Gable
A few pollsters caused a stir recently when they suggested political polls are polluting Canadian democracy. The ensuing debate, which has bizarrely pitted pro-polling pollsters against anti-polling pollster-pundits, has largely been a tempest in a teapot.
The main concern critics have raised is that polls are becoming less accurate. They posit two reasons for this inaccuracy. The first is that publicity-seeking research firms give away free data to the media but cut corners on methodology because they're not being paid. The logic here seems shaky. Why would a firm trying to advertise its expertise by collecting and interpreting data offer inaccurate work? This would be like a winery promoting its wares by giving away samples made with grapes inferior to those used in the wines that are for sale. For firms that trade in accuracy and insight, making mistakes in public is a problem - whether you were paid for your efforts or not.
The second reason suggested for the supposedly increasing inaccuracy of polls is that the industry is not keeping pace with changes in technology, especially the growing number of cellphone users and the Internet's centrality to the way people communicate. Like every industry, the research industry is working to keep pace with technological change. (Recall that door-to-door surveys were once the standard.)
Online research is being steadily incorporated into major firms' toolboxes. The firm that most accurately predicted the outcome of the last federal election used online polling. Researchers also use cellphone supplement samples and other means of ensuring that they're accurately reflecting the population at large. The industry is continually developing not only the tools it uses but the means by which it measures the accuracy of those tools.
In any case, the proof is in the pudding. If polls had really become much less accurate in the past few decades, then we would expect them to be less successful at matching election outcomes. They haven't.
In the 2008 election, the Conservatives won a minority government with 37.6 per cent of the popular vote. Seven firms released measures of party support immediately before the election. All correctly predicted a minority Conservative government, and most were within the margin of error on the popular vote for most parties. The industry's average measurement of Conservative support was 34.7 per cent, 2.9 points off the vote. Industry averages were off by smaller margins for the other parties. Not exactly an embarrassment.
It's true that surveys can vary in their quality. Ultimately, polls are just one more source of information. In a democracy, those with a public platform have a duty to be truthful, and all of us have a responsibility to navigate our public discourse critically. Just as we know which columnists, bloggers and even restaurant and movie critics we trust - because of their record over time - we can learn to trust and distrust pollsters according to the accuracy of the data they produce and the wisdom with which they interpret it.
Response rates, cellphones and online panels are not the issue. Any serious research firm is capable of providing an accurate, representative snapshot of the attitudes and opinions of ordinary Canadians. Election numbers repeatedly affirm that pollsters can produce good numbers. Considering those numbers with a reasonable degree of literacy - an understanding of the capabilities and limitations of the science - is not just a job for the number crunchers. It's a job for all of us.
Michael Adams is president of the Environics group of companies, founder of the Environics Institute, and a fellow of the Marketing Research and Intelligence Association.