Reviewing peer review

A glance at the February issue of The Scientist: Reviewing peer review

It has become harder than ever for researchers to get their work published in top-tier, peer-reviewed journals, says Alison McCook, the magazine’s news editor. As the number of rejection letters to authors has piled up, she adds, so too has the number of questions regarding the efficacy of peer review.

A major problem facing peer-reviewed journals is the sheer number of articles that are submitted to them every year, says Ms. McCook. Reviewers are overburdened, she writes, and their jobs have been made only tougher by scientists who try to distinguish themselves by exaggerating the importance of their research.

“Indeed, sorting through the hype can make a reviewer’s job at a top journal even more difficult than it already is,” writes Ms. McCook.

Yet peer review’s problems may be deeper than just overworked reviewers, she says, as there is still little evidence to show that the process actually improves papers. Rather, she writes, many signs point to the contrary. As an example, she cites a study published in the Annals of Emergency Medicine, which found that reviewers failed to detect two-thirds of the major errors in a fake manuscript. Peer review has also been hindered by allegations of bias and conflict of interest, she says, adding that reviewers are less likely to reject papers that cite their own work.

Several publishers have taken steps to improve peer review, but some of those measures have aroused controversy. The British Medical Journal, for instance, has broken with tradition by requiring its reviewers to sign their reviews. While editors there say that signed reviews tend to produce comments that are more constructive and helpful , other editors argue that the policy could inhibit reviewers from being entirely candid.

Ms. McCook quotes Bernd Pulverer, the editor of Nature Cell Biology, as saying, “I would find it unlikely that a junior person would write a terse, critical review for a Nobel-prize-winning author.”

The article, “Is Peer Review Broken?,” is available at http://www.the-scientist.com/2006/2/1/26/1/

Background articles from The Chronicle:

• Peer-Review Researchers Explore Hyped Conclusions, Open Access, and Bias (9/19/2005)
• Scholar Sends Sham Papers to Social-Work Journals to Show Weakness of Peer Review (10/29/2004)

Opinion:

• We Need to Rethink the Editorial Role of Peer Reviewers (10/25/1996)
• It Is Time to Create an Open System of Peer Review (6/23/1995)