Skip to main content

Among the most notable losers in Ontario's election were the legions of pollsters who failed to foresee the Liberal romp. But at least the Canadian prognosticators have company in their embarrassment.

Eric Cantor's pollster is reported to have told him only days before the recent Virginia Republican primary that he had a 34-percentage-point lead. In fact, the House majority leader wound up losing by 11 points.

Then there's Mitt Romney's team, which oozed confidence right up to election day 2012 that their candidate was headed for the White House. The polls, they claimed, showed wide support.

Story continues below advertisement

Not so much, it turned out.

Whatever side of the ideological divide you're on, it's clear that something is awry in the state of political polling – something that extends beyond any single election, politician or pollster. The surveying techniques that worked so well in the past are stumbling, and there's no shortage of explanations as to why.

Frank Lutz, a Republican pollster, writes in a New York Times op-ed that Mr. Cantor's pollster was guilty of "quantitative malpractice" but makes the surprising assertion that "polls can't predict elections." The real problem, he asserts, is the "voracious media beast" that demands to know an election's winner ahead of time.

Mr. Lutz's unusual modesty about polls is actually a bridge to his larger point – that polling is both art and science. Public opinion is too subtle and nuanced to be gauged by simple "yes" or "no" questions, he asserts.

That is no doubt true, but it sidesteps the question of why polls have had so many high-profile pratfalls over the past couple of years. Public opinion has been sophisticated and subtle for decades. It's only recently, though, that pollsters seem to be messing up with regularity.

One possibility is that polling itself has become more difficult. As more and more people become cell phone-only users, it is harder for pollsters to get a representative sample of the population through the time-honoured practice of calling up people's homes. Add in call-blocking technology and caller-display options, and the number of people who are willing to offer up their opinions to a survey taker shrinks even further.

Stuart Soroka, a professor of political science at McGill University, says the cost of traditional phone polling is now too expensive for many purposes. "Four out of five phone calls don't work" in a typical survey, he estimates, "but each one of those non-responders involves a cost" to the polling company in the form of staff time and salary.

Story continues below advertisement

Pollsters are responding by turning to online surveys. The polling company recruits people by promising money or points if they agree to be part of a continuing panel of respondents. It can then conduct endless online surveys among those panel members at next to no incremental cost.

The potential problem with this approach is that the pollster still has to take the results from the panel and adjust them, using sophisticated statistical techniques, to reflect trends in a wider population. That is no simple matter.

There are other challenges, too, especially when it comes to politics. "It's also about how people are approaching elections," says Prof. Soroka. He believes there has been a rise in what he terms "flexible partisanship," as fewer people identify themselves with a political party.

"In the 1970s and '80s, people often made their voting decision well before the election," he said. These days, people are more likely to be "campaign deciders" who don't make up their mind until the last moment.

Despite those rapidly shifting currents, polls in Canada have done a relatively good job of predicting the share of voters that a party will attract, Prof. Soroka says. The challenge has been in converting those vote totals into an accurate forecast of seats, because the answer hinges on precisely how votes are distributed among ridings. That's more detail than a relatively small sampling of respondents can offer – as all of Ontario recently discovered.

Report an error Editorial code of conduct Licensing Options
As of December 20, 2017, we have temporarily removed commenting from our articles. We hope to have this resolved by the end of January 2018. Thank you for your patience. If you are looking to give feedback on our new site, please send it along to feedback@globeandmail.com. If you want to write a letter to the editor, please forward to letters@globeandmail.com.