Responding to sharp criticism from scholars and editors at home and abroad, the Australian government has decided to abandon a controversial journal-ranking scheme that was a key piece of its attempt to assess the quality of research it helps support. The program had drawn attention from officials in the United States and Europe who are also testing new ways to measure quality.
The rankings, a centerpiece of the Excellence in Research for Australia framework, assigned a grade of A*, A, B, or C to scholarly journals in all fields. The first full round of assessment took place in 2010, with results published early this year. Many researchers in the humanities and social sciences objected to the results, which they said did not reflect their fields’ own standards. Scholars in Australia also complained that the rankings were being used inappropriately by university administrators to evaluate individual researchers.
Sen. Kim Carr, Australia’s minister for innovation, industry, science, and research, announced on Monday that the rankings would be jettisoned. “There is clear and consistent evidence that the rankings were being deployed inappropriately within some quarters of the sector, in ways that could produce harmful outcomes, and based on a poor understanding of the actual role of the rankings,” Mr. Carr said in a written statement. Instead of rankings, he said, the Australian system will incorporate “journal quality profiles.” Mr. Carr added that “the removal of the ranks and the provision of the publication profile will ensure they will be used descriptively rather than prescriptively.”
It wasn’t clear from Mr. Carr’s statement what the journal profiles would include, although he mentioned frequency of publication as one element. He also said that he had asked the Australian Research Council, which administers the assessment system, “to continue investigating strategies to strengthen the peer-review process” involved in the exercise. Mr. Carr described the changes to the plan as “enhancements” and “improvements” rather than an undoing of fundamental principles.
Anna Poletti, a lecturer in English at Monash University, has been one of the most public critics of the journal rankings. She has drawn attention to what she sees as a troubling mismatch between the government rankings and how scholars assess journals in their fields. In her view, the rankings have also created pressure, especially on younger researchers, to publish in the highest-rated journals rather than those their colleagues respect.
In an e-mail, Ms. Poletti said she was glad the government had backed away from the rankings. “It is pleasing that the end result puts more emphasis on the peer-review process rather than the use of a blunt instrument,” she said. “The decision will (hopefully) go some way to making the assessment process less of a barrier to international collaboration, as Australian academics no longer have to factor in something as parochial as the journal rankings when collaborating with overseas colleagues.”
Still, Ms. Poletti wanted more details about what will take the place of the rankings. “I am a bit cautious about what the ‘journal quality profile’ will look like and am looking forward to seeing more information on that from the ARC,” she said. She wonders whether the quality profiles will put a lot of weight on who’s on a journal’s editorial board, as the old rankings did.
For instance, the journal Biography: An Interdisciplinary Quarterly, a prestigious journal in Ms. Poletti’s field, went from a B rank to an A* to a C over the course of a year or so. Ms. Poletti and the journal’s editors believe that its lack of an editorial board contributed to its low rank.
Craig Howes, an editor of Biography and a professor of English at the University of Hawaii-Manoa, said he welcomed the decision to drop the rankings. “I hope that this decision leads to a greater focus on evaluating quality through peer review, and through a good hard look at the individual venues for publication by those responsible for evaluation,” he told The Chronicle in an e-mail.
Mr. Howes emphasized that it was important to maintain standards of scholarly quality, even though the Australian rankings weren’t the best way to measure a journal’s worth. “The goal should not be to abandon standards,” he said. “Any journal should be ready to explain and justify its status within its field, and within the profession in general.”
