Peer-reviewed journals play a pivotal role in modern medicine.
When a journal publishes a study from a drug trial, it conveys the impression that the research is sound. In fact, drug companies will buy study reprints, at great cost, and distribute them to doctors who are often too busy to sort through the medical literature on their own.
A published study is often shopped around as though it represents the seal of approval from a prestigious journal. But not every journal is prestigious, the peer-review process is sometimes faulty, and some journals derive a significant chunk of their revenues from drug-company advertising and study-reprint sales.
"Medical journals, in many ways, are an extension of the marketing arms of pharmaceutical companies," said Richard Smith, the former editor of the British Medical Journal and author of a revealing book, The Trouble with Medical Journals.
Medical journals can be ranked in a hierarchy. The top tier includes: the New England Journal of Medicine, the Journal of the American Medical Association, The Lancet, Annals of Internal Medicine and the British Medical Journal.
There are countless other
biomedical journals - between 10,000 and 20,000 worldwide - specializing in narrow branches of medicine, certain diseases or specific body parts. "Nobody knows the exact number," Dr. Smith said.
But the number of journals that contain clinical studies relevant to patient care can be narrowed down to about 160, said Brian Haynes, chief of the health information unit at McMaster University in Hamilton.
The leading journals have ample staff and a well-established network of expert reviewers who can assess a study and request changes before it goes to print. Most smaller journals have far fewer resources at their disposal.
Yet even with experienced eyes reviewing the manuscripts, there is no guarantee all mistakes will be caught, Dr. Smith said.
When he was at the BMJ, the journal did an experiment to test the veracity of its own reviewing process. A study, in which eight mistakes were deliberately written into the text, was sent to a group of reviewers.
"The median number of errors spotted was two. Twenty per cent of people didn't spot any and nobody spotted more than five," Dr. Smith recalled in an interview.
Dr. Haynes estimates that only a tiny fraction of studies published each year - about 0.1 per cent - are both scientifically sound and potentially relevant to doctors and patients.
That doesn't mean most biomedical research is worthless, Dr. Haynes stressed. Science is an incremental process - two steps forward and one step back - and new findings, he said, must be confirmed by subsequent studies.
Meanwhile, medical publishing is being transformed by the Internet. Now "paperless journals" release studies online, and interested parties can post their own ratings of the work.
John Hoey, former editor of the Canadian Medical Association Journal, has been involved in the startup of a Canadian-based Web journal called Open Medicine.
"What they [the staff] are trying to do is see if they can run a journal that has very little need for resources and, therefore, very few financial conflicts of interests," said Dr. Hoey, who is an adjunct professor at Queen's University in Kingston.
Join the Discussion: