The results of Monday's provincial election in Quebec had some surprises, but the popular vote tally of the major parties was not one of them. For the second consecutive provincial election, the polls made the right call.
When the polls were wrong
Has the demon of the provincial elections in Alberta and British Columbia been exorcised? Certainly some of the lessons from those campaigns were learned, as the positive results in Nova Scotia last October had suggested. Multiple pollsters in the Quebec campaign, for instance, reported support among those most likely to cast a ballot, rather than only the support of the entire (partially non-voting) population.
Four polls were conducted in the last week of the campaign, with their surveys ending on either April 3 or April 4. Those four polls averaged 40.3 per cent for the Liberals, 26.6 per cent for the Parti Québécois (PQ), 23 per cent for the Coalition Avenir Québec (CAQ), and 7.9 per cent for Québec Solidaire (QS). The actual preliminary results of yesterday's vote were 41.5 per cent for the Liberals, 25.4 per cent for the PQ, 23.2 per cent for the CAQ, and 7.6 per cent for QS. The aggregate performance of the polls was thus exceptional.
How the pollsters did
Of the four final polls, those of Forum Research, Angus Reid, and EKOS Research performed the best, with a total error of between 5.7 and 6.6 points across the four main parties.
In terms of total error, the final poll by Forum conducted via interactive voice response (read more about how the so-called robocall polling works) on April 3 and surveying 1,251 Quebeckers was the closest. But at 44 per cent, the poll over-estimated the Liberals by 2.5 percentage points, suggesting an even larger majority victory than the one Philippe Couillard secured Monday night.
The EKOS poll, also conducted via IVR between March 27 and April 3 and surveying 1,422 Quebecers, had all four parties within the narrowest error bands. The estimation of likely voters was within 1.6 and 2.1 points for all four parties (the tally of eligible voters was within 0.9 and 2.2 points for all parties). But in under-estimating the CAQ at 21 per cent it did not hint at the seat gains the party could potentially make.
The Angus Reid poll conducted April 2 to 4 and surveying 1,410 Quebeckers online (learn how polling with online panels works) over-estimated the PQ (27 per cent) and the CAQ (25 per cent) to the detriment of the Liberals (39 per cent), though was closest to the mark for QS (7 per cent).
Ironically, the poll that finished at the back of the pack of the final four was the only one conducted by a Quebec-based pollster. Léger's final survey, conducted April 2 and 3 and surveying 1,220 Quebeckers online, had a total error of 8.7 points. It showed a closer race between the Liberals and the PQ at 38 to 29 per cent, putting a majority government in doubt. Léger was nevertheless very close to the final result for the CAQ.
But these four polls only varied from the final result by relatively small margins, and all provided an accurate picture of the race: a large Liberal lead, a PQ faltering and in second, and a CAQ making gains but stuck in third. In other words, the polls mislead no one.
Which methodology was most accurate
No particular methodology stood out above the others, with both IVR and online polling performing equally well. No live-caller telephone polls were conducted during the campaign, however, perhaps an indication of the changing face of the polling industry. The final stages of the election were also deprived of a last poll by CROP, the other major Quebec-based pollster. But after conducting telephone polls during the 2012 election, that firm had also moved over to online in the first few weeks of this campaign.
Which parties out- or under-performed?
The popular vote results of the election may have matched the expectations set out by the polls very closely, but the Parti Québécois won fewer seats than expected. The CAQ, meanwhile, managed to win more seats than it did in 2012 despite dropping by almost four points. How did this happen?
It would seem that the CAQ managed to out-perform expectations north and south of Montreal, where many of its seats were won by very slim margins. In the 12 seats won by the CAQ around the city, the average margin of victory was under 1,600 votes. That compares to margins of an average of more than 5,000 votes in the five ridings won in central Quebec and more than 6,000 votes in the five ridings won by the CAQ around Quebec City. The CAQ was able to get its vote out in the ridings it needed to, making its vote more efficient than in 2012, when 27 per cent of the vote won them 15 per cent of the seats. Last night, 23 per cent of the vote netted them 18 per cent of the seats.
These entrails will be gone over in great detail in the coming weeks and months, but the fact remains that the pollsters in Quebec did remarkably well. This is especially the case considering that vote intentions shifted so greatly throughout the last five weeks (support for the PQ and CAQ swung by at least 10 points apiece) and that no pollster was in the field in the final two days of the campaign. Maybe the stigma that has been attached to political polls since the debacles in Alberta and British Columbia can be laid to rest – until the next election, at least.
Éric Grenier writes about politics and polls at ThreeHundredEight.com.