Skip to main content
The Globe and Mail
Support Quality Journalism.
The Globe and Mail
First Access to Latest
Investment News
Collection of curated
e-books and guides
Inform your decisions via
Globe Investor Tools
Just$1.99
per week
for first 24 weeks

Enjoy unlimited digital access
Enjoy Unlimited Digital Access
Get full access to globeandmail.com
Just $1.99 per week for the first 24 weeks
Just $1.99 per week for the first 24 weeks
var select={root:".js-sub-pencil",control:".js-sub-pencil-control",open:"o-sub-pencil--open",closed:"o-sub-pencil--closed"},dom={},allowExpand=!0;function pencilInit(o){var e=arguments.length>1&&void 0!==arguments[1]&&arguments[1];select.root=o,dom.root=document.querySelector(select.root),dom.root&&(dom.control=document.querySelector(select.control),dom.control.addEventListener("click",onToggleClicked),setPanelState(e),window.addEventListener("scroll",onWindowScroll),dom.root.removeAttribute("hidden"))}function isPanelOpen(){return dom.root.classList.contains(select.open)}function setPanelState(o){dom.root.classList[o?"add":"remove"](select.open),dom.root.classList[o?"remove":"add"](select.closed),dom.control.setAttribute("aria-expanded",o)}function onToggleClicked(){var l=!isPanelOpen();setPanelState(l)}function onWindowScroll(){window.requestAnimationFrame(function() {var l=isPanelOpen(),n=0===(document.body.scrollTop||document.documentElement.scrollTop);n||l||!allowExpand?n&&l&&(allowExpand=!0,setPanelState(!1)):(allowExpand=!1,setPanelState(!0))});}pencilInit(".js-sub-pencil",!1); // via darwin-bg var slideIndex = 0; carousel(); function carousel() { var i; var x = document.getElementsByClassName("subs_valueprop"); for (i = 0; i < x.length; i++) { x[i].style.display = "none"; } slideIndex++; if (slideIndex> x.length) { slideIndex = 1; } x[slideIndex - 1].style.display = "block"; setTimeout(carousel, 2500); } //

Jeffrey S. Rosenthal is a professor of statistics at the University of Toronto whose books include Knock on Wood: Luck, Chance, and the Meaning of Everything.

The U.S. presidential election gripped us all. Razor-thin margins. Four days of uncertainty about the outcome.

It wasn’t supposed to be this way. Joe Biden was expected to defeat Donald Trump easily, in a rout. How did we know? The polls told us so!

Story continues below advertisement

Ah yes, the polls. As a statistics professor in the public eye, I am asked about public opinion polls all the time. Can we trust them? Are they accurate? How did they go so wrong? Why should we ever bother with them again?

I even hear this from other statisticians. After Mr. Trump’s surprise 2016 victory, one statistician told me that he would like to talk to pollsters “after a few drinks” to learn what was really going on. This week, another statistician asked what I could possibly say to the media about polls, whose recent inaccuracies had completely shaken her confidence.

So, have opinion polls been revealed, once and for all, to be shams, unworthy of our attention? No!

For one thing, the polls weren’t as far off as many people think. Early partial counts showed Mr. Trump massively outperforming expectations. But the final counts, which is what polls try to predict, will have Mr. Biden up by several percentage points in the popular vote. This is not as high as the eight to 10 points the polls predicted, but not so far off either.

Furthermore, he will end up winning most of the states he was expected to – though still missing a few (such as Florida), and nearly missing some supposedly “easy” ones (such as Wisconsin).

Also, emotions can cloud our judgment. Most Canadians, and most statisticians, strongly preferred Mr. Biden to Mr. Trump, and were very upset that he underperformed. If Mr. Biden had, instead, overperformed and won the election twice as easily as predicted, they probably wouldn’t have complained nearly as much.

But the real issue is that polling is much harder than most people realize. I am often asked, “How can we predict millions of votes, based on just a few thousand random samples?” Actually, that part is easy – random samples are a kind of repeated experiment, for which simple statistical analysis can derive estimates, forecasts, probabilities and margins of error with ease.

Story continues below advertisement

The problem is, poll samples are not random. Oh, the pollsters try. They phone people randomly, but most people do not answer. The dirty little secret of polling is that response rates are very small: fewer than one person in 10 will actually pick up the phone, agree to talk to the pollster and reveal voting preferences. And the rest? Why do they not answer, and how will they vote? We’re not sure – they never speak to us.

The pollsters are left with the challenge of forecasting how everyone will vote, based on responses from the small, non-representative minority they can reach. This task is essentially impossible – if it were given as a question on a statistics exam, the students would all complain that it was unfair.

Pollsters soldier on, reweighting their samples using demographic information such as race and age and education to try to mimic the full population as best as they can. Sometimes they succeed very well – polls accurately predicted the U.S. presidential elections of 2008 and 2012, the U.S. midterm elections of 2018, the Canadian federal election of 2019, and several recent Canadian provincial elections.

Other times the polls miss out, like underestimating Mr. Trump’s support in the 2016 and 2020 U.S. presidential elections (though still not as badly as many people think). Why? Perhaps some of Mr. Trump’s supporters were too “shy” to admit their preference, and others were less likely to talk to “elite” pollsters, and some were harder to reach owing to work and other responsibilities – nobody knows for sure.

Some pollsters have given up on random samples entirely. They use alternative methods such as online panels, computer analysis of posted tweets (the Ottawa-based company Advanced Symbolics had great success with this in the 2019 Canadian federal election, but similarly missed the 2020 U.S. presidential election), or even intercepting and surveying people while they browse the web (Toronto-based RIWI used this to correctly predict a tighter 2020 race). They are all chasing the same elusive dream, to figure out how everyone is thinking based on samples neither random nor complete.

So, should we give up on polls forever? Hardly. Without poll results, election commentators would have few insights beyond whatever tidbits they overheard in bars and gatherings. Candidates would have no idea how to prioritize their campaign plans. Voters would have no clue about which races were close, and how to vote strategically to achieve their desired outcome.

Story continues below advertisement

Polls are here to stay, flaws and all. And really, is that such a terrible thing? If pollsters could predict every election with perfect accuracy, campaigns would be irrelevant, citizens would feel superfluous and there would be no reason to hold the vote. The limitations of polls make them sometimes very accurate and sometimes less so. We should trust them somewhat, but not completely, and they can never replace the actual election that follows.

In the end, election polls are like movie trailers. They give you a pretty good sense of what to expect. But if you want to know what really happens, you still have to watch the movie.

Keep your Opinions sharp and informed. Get the Opinion newsletter. Sign up today.

Your Globe

Build your personal news feed

  1. Follow topics and authors relevant to your reading interests.
  2. Check your Following feed daily, and never miss an article. Access your Following feed from your account menu at the top right corner of every page.

Follow topics related to this article:

View more suggestions in Following Read more about following topics and authors
Report an error Editorial code of conduct
Due to technical reasons, we have temporarily removed commenting from our articles. We hope to have this fixed soon. Thank you for your patience. If you are looking to give feedback on our new site, please send it along to feedback@globeandmail.com. If you want to write a letter to the editor, please forward to letters@globeandmail.com.

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff.

We aim to create a safe and valuable space for discussion and debate. That means:

  • Treat others as you wish to be treated
  • Criticize ideas, not people
  • Stay on topic
  • Avoid the use of toxic and offensive language
  • Flag bad behaviour

Comments that violate our community guidelines will be removed.

Read our community guidelines here

Discussion loading ...

To view this site properly, enable cookies in your browser. Read our privacy policy to learn more.
How to enable cookies