Go to the Globe and Mail homepage

Jump to main navigationJump to main content

Report on Business

Economy Lab

Delving into the forces that shape our living standards
Best Business Blog, EPPY awards, 2011 and 2012

Entry archive:

Economy Lab has moved

Only Globe Unlimited members will now have access to a wide range of insightful commentary
and analysis on the economy and markets previously offered on this page.


Globe Unlimited subscribers will be able to read these columns,
written by some of Canada’s most deeply respected economists,
such as Christopher Ragan, Sheryl King, Andrew Jackson, and Clement Gignac,
as part of our ROB INSIGHT section.


All of our readers will still be able to browse the Economy Lab archives and read our
broader coverage of economic data and news by accessing their 10 free articles a month.


Learn more about Globe Unlimited and how to subscribe.

(Paul Chiasson/Paul Chiasson/The Canadian Press)
(Paul Chiasson/Paul Chiasson/The Canadian Press)

Economy Lab

Take poll changes with two grains of salt Add to ...

Many federal election news stories have been generated by one party or the other going up or down in the polls by one or two per cent. The stories almost always note that polls have margins of error.



For example the recent Nanos poll is based on about 1000 respondents, giving a statistical margin of error of 3.1, percentage points, nineteen times out of twenty.

What is not mentioned in the reporting is that changes between polls have about a 40 per cent higher margin of error than individual polls themselves.

So even if a party is up say 3.5 percentage points comparing a new poll with a previous poll, if each poll had a margin of error of 3.1 percentage points, the 3.5 percentage point increase should be compared to a margin for error of about 4.4 percentage points. There is a reasonable chance that the party's vote intention share in the total population did not change at all: all that happened was that the pollsters randomly happened to choose more of the party's supporters in the second poll.

The margins of errors for changes in leads can be twice as large again. If a party is leading by 5 percentage points in one poll and then by 9 percentage points in the next poll, the margin of error around that 4 percentage point gain could be over 8 percentage points. While probably the lead increased, there is still a significant chance that the lead decreased.

Moreover, the official error calculation assumes that the sample of respondents is random and that responses are accurate: it only takes into account the error associated with trying to make inference from a sample of 1000 people as to what 15 million voters might be thinking.



But there are probably additional errors, especially when comparing polls from different pollsters. Has anyone else wondered why the Greens consistently score better on the Ekos poll (9.3 per cent over the last two weeks) than the Nanos poll (3.75 per cent over the same period)? At first I wondered whether it was just that respondents hear "Ekos" which puts "Eco" in their minds, but compared to the other polls, Nanos is just as much on the low side as Ekos is on the high side. I don't know the answer, but it is a reminder that polling techniques likely matter.

Most everyone takes polls with a grain of salt. When considering changes from an individual poll to the next, I take extra grains.



Michael Veall is an economics professor at McMaster University



Follow Economy Lab on twitter

Follow us on Twitter: @GlobeBusiness

 

In the know

Most popular video »

Highlights

More from The Globe and Mail

Most Popular Stories