Liberal leadership hopeful Justin Trudeau delivers a keynote address to Reviving the Islamic Spirit conference in Toronto on Dec. 22, 2012.Chris Young/The Canadian Press
The ongoing federal Liberal leadership race has the potential to shake-up the political landscape in Canada considerably – and it might already be destabilizing the party's numbers in the polls.
Since Justin Trudeau launched his leadership bid on Oct. 2, the Liberals have been polling all over the map. In the 10 polls released between then and the end of last week, the Liberals have registered as high as 30 per cent, as low as 18 per cent, and everything in between. (Read the infographic)
This cannot be chalked up entirely to the vagaries of polling. While the Liberals have been jumping around, the Conservatives and New Democrats have been steady. In those 10 surveys, the Conservatives have only polled between 31 and 36 per cent support, while the NDP has registered between 26 and 32 per cent. The standard deviation from the mean is only 1.6 points for the Tories and 1.9 points for the NDP. By comparison, the standard deviation of Liberal support has been 3.6 points since the beginning of October.
This is unusual. Comparing polls from different firms can be an apples-to-oranges exercise, but all polls are striving towards reflecting an accurate picture of the situation. While some differences should be expected, there should not be this degree of variation.
Greatest polling roller coaster since 2011 election
If the previous 72 polls are divided into comparably sized groups, this standard deviation of 3.6 points for the Liberal Party is greater than any party has experienced since at least the May 2011 general election. The only comparable periods of large fluctuations in the polls were in the aftermath of the last election, the months following former NDP leader Jack Layton's death, and just before and after the NDP leadership convention at the end of March. But even these tumultuous periods saw smaller degrees of deviation, and the current volatility in the Conservative and NDP numbers since October rank among the lowest since the last vote.
Before jumping to the conclusion that the Liberal leadership race is entirely to blame, other possibilities need to be ruled out. The most obvious is the usual statistical variations that polls inevitably have. Assuming all polls had random samples (the online ones do not, and even traditional telephone polling is having difficulty building a truly random sample these days), the margin of error cannot account for all of the variation. In only four of the 10 polls does the average polling result for the Liberals since October (25.4 per cent) fall within the margin of error. The odds that the six other polls can be entirely explained by them falling outside of the polls' standard 95-per-cent confidence are too low to take seriously.
Telephone polls show higher Liberal support
But the different methodologies employed in polls need be taken into account as well. Three methods are used by national polling firms: telephone surveys using live-callers, telephone surveys using interactive voice response, and Internet surveys using online panels. Each of these methodologies has been used since the beginning of October, and if we divide the Liberals' results up accordingly we do see some patterns.
Polls using live-callers have averaged 28.4 per cent for the Liberals and have two of the highest results for the party. IVR surveys averaged 26.6 per cent, while online panels returned an average result of 22 per cent for the Liberals. It is possible that online surveys are introducing a methodological bias into the equation, and past performance backs that up.
In the first nine months of 2012, divided into blocks of three, we also see that online polls were always averaging less than those conducted by IVR and those employing live-callers. On average, polls with live-callers have had the Liberals 1.6 points higher than polls using IVR, and 3.3 points higher than polls relying on online panels.
This could be one explanation for much of the variance between polls, but the difference between telephone and online polls that has been recorded since Oct. 2 is almost twice the average for the rest of the year. And it still does not account for the recent period of far greater standard deviation in the Liberals' polling than any other period since the last election.
Leadership race is boosting Liberal numbers
That the polls were taken on different field dates could be another factor, especially considering that some of the best and worst polling results are clumped together. That means that political events, such as the leadership race, become the most plausible explanation. There is a strong case to be made here.
Justin Trudeau's campaign launch has to be considered one of the driving forces in the Liberals' improved numbers: the party averaged 21.8 per cent in national polls in the months preceding his launch, compared to the 25.4 per cent since. More importantly, every polling firm that was in the field before and after Oct. 2 has indicated growth in Liberal support.
But recent events might have also played a role. The worst results that the Liberals have put up since Oct. 2 were in polls that were taken after the controversies surrounding comments made by David McGuinty and Justin Trudeau about Alberta, as well as Mr. Trudeau's stance on the long-gun registry.
We cannot be fully certain, however, if that dip in support was real or just part of the volatility that has occurred in the last two or three months. Mr. Trudeau's candidacy has certainly mixed things up quite a bit, as polls have shown support for a Trudeau-led Liberal Party as low as 31 per cent and as high as 39 per cent.
What we know about where the parties stand
Though we have a good idea that the current volatility in Liberal polling numbers are unusual and are likely influenced by methodological bias, standard statistical variation, and politics itself, it does not tell us much about the current level of actual Liberal support. But there are some things from the polls that we can "know" with more certainty: the Conservatives are leading, the New Democrats are second, and the Liberals are still in third. Probably.
There is strong evidence that this is the case. The Conservatives have led or have been tied for the lead in 15 of the last 17 polls since the beginning of August, and have averaged about 34 per cent over that time. The New Democrats, meanwhile, have averaged 30 per cent in that period and have been placed second, or tied for first or second, in 14 of those 17 polls. The case for a third-place Liberal Party is even stronger: they have been third or tied for third in 76 of the 82 national polls conducted since May 2011, and have averaged 24 per cent since early August.
The probability that the polls are simply wrong that the Conservatives lead the NDP is only about 14 per cent, based on past polling error. The chances that they are wrong about the margin between the NDP and the Liberals are only about 4 per cent. That makes for some pretty good odds that the general picture the polls are painting is correct. The expectation that Liberal polling numbers will continue to fluctuate wildly in the coming months is also a safe bet.
Éric Grenier writes about politics and polls at ThreeHundredEight.com.