Skip to main content
The Globe and Mail
Get full access to globeandmail.com
Support quality journalism
Just $1.99 per week for the first 24 weeks
Just $1.99 per week for the first 24 weeks
The Globe and Mail
Support quality journalism
Get full access to globeandmail.com
Globe and Mail website displayed on various devices
Just$1.99
per week
for the first 24 weeks

var select={root:".js-sub-pencil",control:".js-sub-pencil-control",open:"o-sub-pencil--open",closed:"o-sub-pencil--closed"},dom={},allowExpand=!0;function pencilInit(o){var e=arguments.length>1&&void 0!==arguments[1]&&arguments[1];select.root=o,dom.root=document.querySelector(select.root),dom.root&&(dom.control=document.querySelector(select.control),dom.control.addEventListener("click",onToggleClicked),setPanelState(e),window.addEventListener("scroll",onWindowScroll),dom.root.removeAttribute("hidden"))}function isPanelOpen(){return dom.root.classList.contains(select.open)}function setPanelState(o){dom.root.classList[o?"add":"remove"](select.open),dom.root.classList[o?"remove":"add"](select.closed),dom.control.setAttribute("aria-expanded",o)}function onToggleClicked(){var l=!isPanelOpen();setPanelState(l)}function onWindowScroll(){console.log("scroll");var l=isPanelOpen(),n=0===(document.body.scrollTop||document.documentElement.scrollTop);n||l||!allowExpand?n&&l&&(allowExpand=!0,setPanelState(!1)):(allowExpand=!1,setPanelState(!0))}pencilInit(".js-sub-pencil",!1);

Scientists say new approaches to analysis of online data can provide new insights into human nature.

violetkaipa/Getty Images/iStockphoto

When Eric Horvitz was a medical student, he learned an anecdotal rule of thumb: Expect more cases of congestive heart failure around the holidays. Years later, as a computer scientist and co-director of Microsoft Research, he was able to put that rule to the test.

Working with anonymous patient records from a large urban hospital in Washington, he and colleagues were able to show how a concentration of Internet searches for recipes with ingredients high in salt and cholesterol in the neighbourhoods around the hospital seemed to closely correspond to the heart cases that showed up soon after.

For Dr. Horvitz, it was another example of the startling ways in which the Internet is revealing information about human behaviour that could be used to prevent disease and improve public health.

Story continues below advertisement

By combing through reams of anonymous search logs to spot hidden relationships in the data, "we get a sense for what's really going on," Dr. Horvitz said Monday at the annual meeting of the American Association for the Advancement of Science.

In another example, Dr. Horvitz and his collaborators were recently able to identify a previously unpublished side effect of combining Provacol, a commonly prescribed statin, with paroxetine, an antidepressant, by noticing the way people searched online for information about the two drugs. He also presented work showing how women diagnosed with breast cancer searched for information online around crucial decisions regarding treatment, or how subtle language changes in the Twitter feeds of new mothers could be used to track the onset of postpartum depression – data that can potentially be used going forward to design a mobile app that could tell women if they are at risk based on their communication patterns.

"We are looking at the Web as a sensor for various challenges in health and diagnosis," he added.

The work offers a striking contrast to public concerns about online surveillance, which erupted in the aftermath of the Edward Snowden scandal. The difference, researchers say, is that instead of sifting through online data, e-mail logs and other sources to identify and track individuals, researchers are looking at our collective digital records in hopes of pulling out new and useful information.

"The question is, can we – through this – understand our behaviours better?" said Jure Leskovec, a computer scientist at Stanford University.

Dr. Leskovec studies the evolution of online networks, watching how information flows and how individuals participate – or don't – depending on their use of language. He has used the data to improve how students interact as a community when they enroll in large-scale open online courses offered by the university.

Despite voicing cautions and caveats about some of the limitations of online data – including the fact that it leaves out vast groups of people who have little or no contact through the Internet – researchers at the Chicago meeting shared their growing sense of excitement about using big data to explore ideas in the social sciences that previously existed only in the realm of theory.

Story continues below advertisement

Michael Macy, director of the Social Dynamics Laboratory at Cornell University, said he and colleagues had been using Twitter and anonymized e-mail logs to test whether geopolitical conflicts follow the "clash of civilizations" – a controversial theory advanced two decades ago by Harvard political scientist Samuel Huntington which depicts a world aligned in mutually antagonistic cultural groups.

"We can actually revisit Huntington's thesis by looking at interpersonal communications all around the globe," Dr. Macy said. He said his analysis found a pattern in the data similar to Prof. Huntington's prediction. Although he cautioned against over-interpreting the result, he added: "I think we can still learn something from it."

Dr. Macy said that, for social scientists, the arrival of big data was having an impact not unlike that of the Hubble Space Telescope on astronomy, or the advent of functional magnetic resonance imaging (fMRI) on brain science. "We have an opportunity," he said, "to find patterns that were never visible before."

Report an error Editorial code of conduct
Tickers mentioned in this story
Due to technical reasons, we have temporarily removed commenting from our articles. We hope to have this fixed soon. Thank you for your patience. If you are looking to give feedback on our new site, please send it along to feedback@globeandmail.com. If you want to write a letter to the editor, please forward to letters@globeandmail.com.

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff.

We aim to create a safe and valuable space for discussion and debate. That means:

  • Treat others as you wish to be treated
  • Criticize ideas, not people
  • Stay on topic
  • Avoid the use of toxic and offensive language
  • Flag bad behaviour

Comments that violate our community guidelines will be removed.

Read our community guidelines here

Discussion loading ...

To view this site properly, enable cookies in your browser. Read our privacy policy to learn more.
How to enable cookies