Susan Pinker is a psychologist and author of The Village Effect: How Face-to-Face Contact Can Make us Healthier and Happier.
It’s notoriously hard to predict the future. The Jetsons, a futuristic cartoon that first aired in the early 1960s, showed the average third-millennium family scooting around in flying cars. Men worked three hours a day, women didn’t work at all and kids went on Cub Scout outings to the moon.
True, this was children’s television. But even today’s pundits routinely get things wrong. Few predicted the Brexit outcome. The 2008 financial crisis and the 9/11 attacks also came as a surprise. Indeed, false bets have been the theme of this American primary season, with New York Times columnist Ross Douthat tweeting that “the entire commentariat is going to feel a little silly when Marco Rubio wins every Republican primary” and FiveThirtyEight’s Harry Enten writing that Donald Trump had a better chance of playing in the NBA finals than winning the Republican nomination. If economists and experts of various political persuasions can’t predict what’s ahead, can anyone?
The answer is a qualified yes, say University of Pennsylvania psychology professors Philip Tetlock and Barbara Mellers. In a recent study in Perspectives on Psychological Science, the married couple and their colleagues show that they can identify and cultivate an elite group of predicters, called superforecasters. This select group is not clairvoyant. No one is. But under certain conditions a small fraction of the public is more likely than anyone else to think clearly about the future.
To find a pool of prognosticators, the researchers launched the Good Judgment Project, which recruits volunteer forecasters from professional societies, universities, research centres, columnists and bloggers. From 2011 to 2015, about 25,000 participated in forecasting tournaments that posed about 500 questions, such as “Will North Korea detonate another nuclear weapon within the next three months?” “Will Greece leave the eurozone?” and “How many refugees will flee Syria next year?”
Each question had to pass what the researchers called the clairvoyance test, meaning there could be no dispute about the answer after the fact. Still, the forecasters’ challenge was not to nail a prediction 100 per cent but to estimate, as closely as possible, the probability of an event occurring. On a question about Greece leaving the eurozone in 2015, Prof. Tetlock said that “many experts and markets were putting probabilities on Grexit as high as 60 per cent, whereas superforecasters were in the range of 20 per cent at the peak of the crisis.” Greece stayed in the eurozone last summer, so the superforecasters – among them mathematicians and scientists, but also a pilates instructor and a Canadian underwater hockey coach – turned out to be better seers than the pundits.
What makes the superforecasters so good? The “supers,” or the top 2 per cent of all forecasters, had above average levels of fluid intelligence on intelligence tests that emphasize abstract thinking.
“They were faster information processors and better pattern detectors,” Prof. Tetlock explained. They also had a superb grasp of geopolitical facts, an appetite for intellectual challenges, and were highly competitive – they enjoyed getting closer to the truth than anyone else.
Still, the researchers wanted to find not just the sharpest minds but the best environments for making predictions. Volunteers were randomly assigned to experimental groups in which forecasters either worked solo or in virtual teams. Working “together” took place via a software platform that allowed forecasters to present new facts, rationales or critiques to their team members online. These team forecasters were also shown how to avoid the common pitfalls of team work. “We warned them about the dangers of group-think, and gave them guidance about how to disagree without being disagreeable,” Prof. Tetlock told me.
Given the competitive streak that typified superforecasters, I was surprised to learn that forecasters working in teams beat the solo predictors by a long shot. But it was not only the working conditions that allowed predictors to thrive. More than anything else, it was the mindset.
The “supers” had a willingness to update their beliefs constantly as new data rolled in. That openness was the strongest ingredient in accurate predictions – which makes these superforecasters not like pundits at all.Report Typo/Error
Follow us on Twitter: