You can expect much more federal election campaign coverage in the coming weeks and with it, regular opinion polling, which is all done to help you decide. But guidelines for reporting on polls are needed so the importance of these surveys isn’t inflated or misinterpreted.
Some of you have told me you view polls with suspicion, especially since a few have been misleading. The 2012 Alberta election saw a pollster predicting the Wildrose Party would win. It didn’t and the Progressive Conservatives won easily. A similar result happened next door in British Columbia the next year when New Democrat Adrian Dix was well ahead until voting day when Liberal Christy Clark won. The U.S. presidential election was another case that seemed to surprise at least some pollsters.
The Globe and Mail will rely on Nik Nanos, chief data scientist and founder of Nanos Research, whose track record has been very strong. He uses a rolling poll that he prefers to one-off polls because they “give a sense of direction. It is important to understand the movement of voters and that direction is more important than today’s numbers.”
He argues that politicians are right when they say the only poll that really matters is the one on election day, but opinion polling allows voters to put a spotlight on key policies and demonstrate who is gaining traction. It allows you to see which candidates have a chance at governing and might need greater scrutiny.
When polling misses the mark, as in cases mentioned above, Mr. Nanos said one of the problems is that pollsters don’t survey right up to the last minute. You need to capture the people who decide at the very last minute, like those who do their shopping on Christmas Eve.
He argues that the strength of his polling comes from random sampling, whether online or on land and cellphone lines, keeping the survey short, asking open-ended questions on policies and starting with the most important question.
Even with this care and attention to the data and facts, it is important for Globe journalists to take care with reporting and analyzing polling data and to apply the same skepticism to polling as they do to political party platforms.
So here are my seven suggestions for covering political polls.
- Cover only trustworthy sources. This is true for any news coverage. Ignore the dubiously funded, the tiny sample size, the wild outliers, especially the online polls where anyone can vote because they are often taken over by trolls, bots or party workers.
- Include key details in every first news story about a poll. Who funded the poll. What was the question and, where possible, the order of questions. How many people were sampled and over what time period.
- Include the margin of error and remind readers that it is plus or minus. Right now, the polling is showing a tight race, so don’t say one party or another has jumped out ahead if the variation is well within the margin of error. There is nothing wrong with repeating that it remains close.
- Show the trend and explain why. Talk to independent analysts as well as the pollster about why this might be happening, what policies or statements are gaining traction with voters.
- Don’t focus on the sub-groups – regional, age etc. – unless the sample size is large enough that the pollster has confidence in breaking that out. Then take extra time to explain those regional or other variations without focusing too much on the numbers.
- If there appears to be a newsworthy outlier poll, find out what other pollsters are saying and whether this one has any credibility. If there is no value to it, other than surprise, then ignore it.
- Spend more time explaining the policies, the leader’s views and temperament and their qualifications to be prime minister, both in terms of running our country and in foreign relations.
You may still be skeptical about polling, but remember it is a service to help you understand where the campaign is moving and why. And it will be one key piece of the overall coverage.
Sylvia Stead