Canada’s pollsters woke up Wednesday with what is becoming an eerily familiar feeling – egg on their face, so to speak, after the B.C. election results. How could they have so badly underestimated Christy Clark’s late surge?
The result comes after similar ones in Quebec and Alberta: The polls didn’t predict election night. Alberta Premier Alison Redford, once a polling underdog herself, couldn’t help but congratulate Ms. Clark’s Liberals for “proving the pollsters, pundits [and] political scientists so spectacularly wrong.”
Some pollsters were rolling out mea culpas on Wednesday. “Clearly we missed on some of it pretty badly,” Ekos Research Associates pollster Frank Graves said. Others said it’s not that simple.
Polls have become a pillar of election campaigns – with newspapers and other media relying on them in their coverage of how a race is evolving. Pollsters say they are searching for answers about what went wrong and are refining their methodologies, and some warn their credibility is at stake. But they also say the industry is changing along with Canada, with shifting demographics and forms of communication, and the evolving nature of low-turnout, last-minute-decision campaigns.
And pollsters weren’t all entirely wrong. A Forum Research poll last week did forecast a slim majority government for Ms. Clark, and many polls showed a narrowing result and a horse race emerging. Some also saw large swaths of undecided voters.
As such, not everyone thinks pollsters failed. Pollster Angus Reid called this race “polling at its finest,” suggesting the NDP were, in fact, ahead and that Liberal voters were motivated by polls showing just that.
Regardless, the polling industry is one in transition, struggling to refine methods to cope with several factors.
Not all polls are created equal. There are three general strategies: live phone interviews, automated phone polls and online panels that use pre-assembled lists of participants.
In B.C., phone polls fared much better than online polls.
The poll that best predicted Monday’s results was an automated phone poll, known as “IVR,” or Interactive Voice Response. Published five days before the election, Forum Research found the Liberals were just two points behind, but predicted the vote was spread out in such a way that they’d actually win 43 seats, or a one-seat majority. The Liberals ended up winning 50. “It wasn’t bad. It could have been better, of course,” Forum President Lorne Bozinoff said.
IVR polls are quick. However, some other pollsters dismiss IVR as too randomized and unscientific.
“It’s robocalling, basically, and most pollsters wouldn’t give it the time of day. I think [Forum] just happened to luck out on that one,” said Steve Mossop, president of Insights West. Its most recent poll, on May 2, was done online and found the gap closing, but had the NDP still up eight per cent.
“I think there is some serious question or doubt [about pollsters’ credibility]. Pollsters have always hung their hat on this – if we can predict elections, we can predict other things. This is a blow to the industry. It’s not insurmountable, but it’s a blow for sure,” Mr. Mossop said.
Some phone polls can call cellphones and, therefore, reach people without land lines. But not all do, and Forum’s did not. Live interview telephone polling also takes longer and is more costly, but many see it as more reliable.
“Even with declining response rates, it is still the best way to collect data, by far and bar none,” said Paul Seccaspina, president of Oraclepoll Research, which used live interviews to publish a poll, last week, showing Ms. Clark catching up but down four points. She won by five.
Online panels did, on average, the worst. Critics say they tend to skew towards younger voters and have to be adjusted. They also say there’s no proven way to estimate a margin of error. Online polls failed, said Dimitri Pantazopoulos, Ms. Clark’s former principal secretary and a Liberal pollster. “I think [most pollsters] had some kind of flaw, and they called it wrong. ... The numbers were solid, they were trending in the right direction,” he said.
Pollsters, in defending themselves, have a reminder: Campaigns matter.
In Alberta’s election last year, most polls showed a large lead for the Wildrose Party, but they were conducted before a series of controversies led the party to stumble down the stretch. Ms. Redford won.
The late swings are due, in part, to undecided voters. Mr. Seccaspina’s poll found 24 per cent of respondents hadn’t decided, while Mr. Mossop found 15 per cent had not, but were leaning Liberal. An Ipsos Reid exit poll found 11 per cent of voters decided in the voting both.