How to Bring Prestige to Open Access — and Make Science More Reliable

Beginning next year, a coalition of European research-funding agencies will require funding recipients to publish grant-supported work in open-access journals. While this is a positive step in making research more widely accessible, many researchers are worried because the plan, known as Plan S, also severely restricts their ability to publish in high-profile subscription journals, which are typically associated with high impact factors.

This is not a trivial concern. Regardless of the evidence that a journal’s impact factor is not associated with the quality of its papers, researchers continue to be evaluated by the journals they publish in, rather than by the work itself.

Grant-making agencies can address this prestige problem while still exerting control over publication practices. Moreover, in doing so they can help solve the replicability crisis that has plagued scientific research in recent years. How? By establishing journals themselves that adhere to publication practices that promote replicability.

Too many studies in the social and medical sciences can be neither reproduced nor replicated, which has done serious harm to science-research integrity. Efforts made by individuals, work groups, and scientific journals to solve the crisis have moved the needle some, but there needs to be a critical mass of researchers participating. Individual scholars will be hesitant to embrace reform if they believe it might hamper their careers.

That is why publishing articles in a funding-agency journal would be both required of and, more important, restricted to grant beneficiaries. Publication in such a journal would be its own mark of prestige. Submissions to these journals would become publicly available before peer review, much like a preprint, so that others could evaluate the work as soon as possible.

Hypothesis-driven studies would be evaluated and published as Registered Reports, an emerging model in which reviewers evaluate the quality of the rationale and methods of the studies rather than the results. A successful grant application would be pre-accepted before a single data point had been collected. This would ensure publication regardless of result, helping to eliminate publication bias, the preference for positive results. Because hypotheses, sample-size estimations, and analytical plans would have to be specified within the grant, accountability by researchers would increase.

Exploratory research would also be welcome in these journals, but it would be labeled as such. Doing so would prevent researchers from capitalizing on chance, by presenting interesting results they discover serendipitously (or otherwise) as the outcome of a directed research plan.

Those improvements in research quality would not be the only benefit to funding agencies: They would also save money. That’s because article-processing charges are now a common request in grant-application budgets, and those rates would almost certainly be reduced with an in-house journal. A 2016 report revealed that the average cost across publishers for processing charges was more than $2,000. Publishers like PeerJ have shown it’s possible to maintain journals with processing charges of around half that.

The idea for a grant-funding-agency journal is not entirely new. For instance, the Wellcome Trust, the Irish Health Research Board, and the Bill & Melinda Gates Foundation (one of the backers of Plan S) have each established their own open-access journals. However, these groups do not require beneficiaries to publish in them.

There are several potential criticisms of this proposal. First, an appreciable volume of research is not supported by any agency grants, so researchers with other sources of funding may feel disadvantaged. However, this concept isn’t meant to promote a single journal to the top of the prestige pile, but rather add an open and transparent journal to those ranks. Most fields don’t have just one prestigious journal, but a collection of them.

Another possible concern is that reviewing grant applications with detailed protocols would take more time. While this might be true, any lost time would be saved on the back end because the people approving applications would be the same ones reviewing resulting papers. Reviewers would just need to confirm that authors followed protocols and appropriately interpreted results. The money saved from article-processing charges could also go back into compensating grant reviewers for their time, a practice that many agencies already adopt.

Finally, there’s the risk of the Matthew effect — that advantage begets further advantage. Women, members of minority groups, and early-career researchers are underrepresented among grant winners, so this proposal might exacerbate the imbalance. Some agencies have taken steps to redress these imbalances by creating awards targeting underrepresented groups. If this practice were more widespread, those groups would have a greater opportunity to publish in a prestigious venue.

Of course, several details would need to be clarified, such as how these journals would reject exploratory research. One possible solution is to use, as other open-access platforms have done, the term “not approved” rather than “rejected,” indicating that the paper requires further work to resolve serious flaws. In theory, authors could submit low-quality exploratory work that could indefinitely linger in “not approved” limbo, but that strategy would obviously be against their best interests.

Funding agencies, out of all participants in the research process, hold remarkable influence on how research is conducted. It is time that they take full advantage of that influence to make science more reliable.