For years, advocates of open access to research literature have claimed that making a paper freely available online will increase the number of times it is cited by other articles. But the data show otherwise, according to the first study that randomly assigned papers to either free or subscription access.
The study appeared online Thursday in BMJ, formerly known as the British Medical Journal, and will be in the August 9 print edition.
Numerous studies have found that open-access articles receive twice as many citations as subscription-access papers, or more, but critics have noted that access may not be the cause of the increase. It is also possible that better articles were more likely to appear free online than were weaker ones. Thus the increase in citation rate may simply be because the paper being cited was of higher quality.
To find out the real cause of the citation advantage, Philip M. Davis, a graduate student in communication at Cornell University, randomly selected to appear online free of charge 15 percent of the papers published between January and April 2007 in 11 journals of the American Physiological Society. The remaining papers remained accessible only to subscribers. (The open papers appeared with an icon of an open lock on the journal’s Web site.) In this way, both free and subscription-only papers should have been equal in quality.
In January 2008, Mr. Davis and colleagues at Cornell collected and analyzed the citations. They found that the papers that were freely available were actually cited slightly less often than those that were not, though the difference was not statistically significant. More people did, however, read the openly accessible papers than the ones protected by subscription barriers, as measured by full-text and PDF downloads, as well as number of unique visitors to those Web pages.
Mr. Davis said his results suggest that open access increases browsing by scientists or reading by lay people. But it does not increase the direct use of those articles by other scholars in their own work. Most scientists who are writing research papers, he argued, already have free access to all sorts of journals, even those that are subscription-only. They are at institutions with excellent libraries and online collections, and so, he said, “for all intents and purposes, scientists have free access to the literature.”
Some other experts agree. Michael J. Kurtz, an astronomer and computer scientist at the Harvard-Smithsonian Center for Astrophysics, has studied (and promoted) open access in his own field. He said he finds Mr. Davis’s paper “totally convincing.”
But some researchers and open-access advocates are unswayed. Gunther Eysenbach, an associate professor of health policy, management, and evaluation at the University of Toronto, said Mr. Davis did not allow enough time to pass between publication of the randomized papers and his search for citation rates.
After an article is published, he said, any manuscript that would cite it must be written, sent to a journal, peer reviewed, published, and then indexed by a citation collector like the one Mr. Davis used, ISI Web of Science. “This process takes longer than nine to 12 months,” he said. “It really doesn’t make any sense to expect a difference” so soon, he added.
Stevan Harnad, a professor of cognitive science at the University of Southampton and a prominent promoter of open-access archives, agreed that timing was a flaw in the BMJ study. He added that previous research had found that an increase in readership, which Mr. Davis noted among the open-access papers, indeed was followed by an increase in citations 18 months later.
The controversy continues and awaits more results. Mr. Davis will look at the physiology journals’ citations for another four years and is expanding the study to include journals in other fields. Dr. Eysenbach, Mr. Harnad, and at least one other scholar, Mark J. McCabe, of the University of Michigan at Ann Arbor, are each also running studies that try to discover how much of the apparent citation advantage for open-access articles is due to a bias in quality rather than access itself.