Incompetence by their reviewers was the most common problem reported by scientists who submitted manuscripts to scholarly journals. Almost two-thirds voiced that beef in a survey administered to scientists employed by the National Institute of Environmental Health Sciences.
The supposedly expert reviewers, scientists complained, had not carefully read articles, were unfamiliar with the subject matter, or made mistakes of fact or reasoning. The survey results, the first of their kind, were reported in the September issue of the journal Science and Engineering Ethics.
Only a small percentage of scientists reported experiencing two of the most serious violations of peer-review ethics, breach of confidentiality (7 percent) or theft of ideas (5 percent). That finding appears at odds with anecdotal reports that those two problems are pervasive and that the peer-review system is a corrupt, old-boys’ network.
Perception Gap
Journal editors have been working for years to improve the quality of peer reviews and police misconduct among reviewers, but the new study indicates they have not completely succeeded, said Nicholas H. Steneck, who directs a research-ethics program at the University of Michigan at Ann Arbor.
“There seems to be a gap between what the editors are trying to do to improve and what the authors are perceiving,” he said.
Respondents reported other kinds of unethical conduct, too. Nearly 18 percent said that reviewers’ comments had included personal attacks. Approximately 10 percent said reviewers had delayed publication of the paper so they could publish a paper on the same topic first.
The survey was sent to more than 500 scientists at the environmental-health institute, part of the National Institutes of Health, and just over half responded.
The findings were described in the journal by bioethicist David B. Resnik and two coworkers. They called their study “exploratory” because of its limitations: Their sample was not random, and they did ask how often the problems had occurred over the scientists’ careers.
Still, they contend the results are probably representative of the experiences of scientists outside of the environmental-health institute. And the findings, the coauthors wrote, suggested cause for worry about the health of peer review.
“A researcher who is concerned that his/her ideas will be stolen, for example, may not disclose all the information that is needed to repeat his/her experiments,” they wrote.
To lessen perceptions of incompetence, they said, journals should increase their efforts to train reviewers. And to deal with bias, the journals could do more to select impartial ones.
The paper’s findings should be taken with a grain of salt, but those recommendations are important, said Brian C. Martinson, a senior research investigator at HealthPartners Research Foundation, a nonprofit center in Minneapolis, who has studied scientific misconduct.
“Anyone who’s ever received a negative review” of a journal manuscript is likely to call the reviewers incompetent—"that’s just human nature,” he said.
But the paper’s estimate about the prevalence of egregious behavior may be low, Mr. Martinson added. About half of a random sample of scientists he surveyed in 2002 reported knowing of a colleague who had stolen ideas within the past three years (The Chronicle, June 9, 2005).
It’s difficult to know how many of such reports are accurate, Mr. Martinson said. Nevertheless, perceptions of unfairness by peer reviewers are dangerous, and should be dealt with, because they provide an excuse for others tempted to engage in the same kind of unethical behaviors. “They might say, If it’s not fair, why should I play fair?” he said.
