Skip to main content

Psychologists use murky research methods, survey finds

Alexander Raths/Thinkstock

Murky research methods, which essentially distort the truth, may be very common in the field of psychology, a study suggests. While subtle, these questionable research practices pose a great threat to scientific integrity.

The study is based on a survey e-mailed to 5,964 psychologists at major U.S. universities. They were able to provide their answers anonymously.

Of the 2,155 who responded, half admitted to selectively reporting only those experiments that delivered the results they wanted.

Story continues below advertisement

The findings, published in the journal Psychological Science, also revealed:

  • 43 per cent admitted to excluding data after looking at the potential impact on the results.
  • 22 per cent stopped collecting data earlier than planned because they got the result they were looking for.
  • 35 per cent reported an unexpected finding as if it had been expected all along.

"These practices are very damaging … because they vastly increase the likelihood of finding a positive result when there isn't anything really going on," said George Loewenstein, one of the authors of the study and a professor of economic and psychology at Carnegie Mellon University in Pittsburgh.

If these results are the ones most likely to get published, it creates a skewed, or distorted, body of research over time. "The literature becomes populated with false results – with results that are not true," Prof. Loewenstein added.

He suspects that many researchers do not appreciate that what they are doing is wrong. And if some act this way, others will feel compelled to follow suit just to stay competitive in the "publish-or-perish" world of academic research. "It becomes a race to the bottom," he added.

The consequences can be far-reaching. Published research is often used as a basis for public policy and determining the best practices for patient care. Furthermore, previous studies often shape the direction of future research.

"Scientists may try to build on the initial findings, but they can't replicate them. So a lot of time is wasted," Prof. Loewenstein said.

"Some psychologists will be unhappy with us publishing this [survey] because they will feel we are airing their dirty linen," he said. "The solution, in our view, is not to sweep this under the rug, but we need to clean up our collective act."

Story continues below advertisement

Leslie John, a study co-author at Harvard Business School, suggested that psychologists should consider instituting a system in which journals only accept articles for publication if the study was registered before it began with details about how it would be carried out. This safeguard would prevent last minute changes to a study's design or the disappearance of a failed study from the research record.

Report an error Editorial code of conduct
Due to technical reasons, we have temporarily removed commenting from our articles. We hope to have this fixed soon. Thank you for your patience. If you are looking to give feedback on our new site, please send it along to feedback@globeandmail.com. If you want to write a letter to the editor, please forward to letters@globeandmail.com.

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff.

We aim to create a safe and valuable space for discussion and debate. That means:

  • Treat others as you wish to be treated
  • Criticize ideas, not people
  • Stay on topic
  • Avoid the use of toxic and offensive language
  • Flag bad behaviour

Comments that violate our community guidelines will be removed.

Read our community guidelines here

Discussion loading ...

Cannabis pro newsletter