Szymon Machajewski has a host of artificial-intelligence tools in his arsenal when he’s called on to review papers that are being considered for publication in academic journals.
There’s Explainpaper, where one can upload a paper, highlight a confusing portion of the text, and get a more reader-friendly synopsis. There’s jenni, which can help discern if a paper is missing relevant existing research. There’s Quivr, where the user can upload a paper and pose queries like: What are the gaps in this study?
Machajewski, associate director of academic technology and learning innovation at the University of Illinois at Chicago, is still the one ultimately writing the peer review. But these tools, he said, have become an essential aid to his and some other peer reviewers’ work.
“I am sure there are still people who are printing papers and reading them by the fireplace on a Saturday,” he said, but AI tools are helpful for those who need to be efficient.
Using AI as an assistant is a growing trend among academic editors, as journals field more submissions while tapping a depleting well of peer reviewers. In this reality, an AI tool that can quickly identify whether a paper’s subject matter falls within a journal’s scope, or can expeditiously find potential peer reviewers with relevant expertise, can be valuable.
It’s also a trend that — at least for now — hasn’t spurred the same level of editorial policymaking and calls for transparency as authors’ and researchers’ use of such tools.
In the year since ChatGPT’s rollout, fear that papers created by generative AI might be submitted as scholars’ work has prompted many publishers and journals to post policies fencing in authors’ use of AI. Springer Nature, one of the world’s largest academic publishers, prohibits crediting AI as an author on a paper and requires authors to disclose if and how AI was used, for example.
Yet, for the most part, AI-specific policies for journal editors and reviewers haven’t followed.
The Chronicle contacted 15 major publishers for this story. The five who responded — accounting for upward of 7,700 journals — emphasized that AI tools, if used at all, are never the sole decision-makers, and that editors remain responsible and accountable for the editorial process and final calls.
Many AI tools on the market are capable of functioning autonomously, though — opening up the potential for misuse in the absence of clear and publicized guidelines. The occasional social-media post from academics decrying “automatic” rejections are just one sign of how inconsistent or opaque practices can breed confusion and distrust.
How Publishers Are Using AI
With publication in peer-reviewed journals being essential for most academics working toward promotion and tenure, journal editors must sift through a sometimes-overwhelming number of papers.
The publishing giant Elsevier received some 2.7 million submissions across its more than 2,800 journals in 2022 and published over 600,000 papers. University presses are sought-after homes for academics’ research, too; the University of Chicago Press, for example, sees over 10,000 submissions annually across the more than 90 journals in its portfolio. Another, Cambridge University Press, published 18,600 papers across more than 400 journals last year.
We’re relying on these editors’ expertise to judge whether these manuscripts are suitable for our journals. I just don’t see AI replacing that judgment and that expertise.
Perhaps it’s not surprising, then, that high-level discussions about the role of artificial intelligence in editorial decision-making began well before ChatGPT. In the fall of 2021, the Committee on Publication Ethics, a nonprofit membership organization that sets global standards, published a “discussion document” on the subject, inspired by questions raised at a 2019 forum. The document acknowledged that “opportunities for AI use in publishing are increasing at a high rate” and urged publishers to be more transparent, as well as cautious, about how heavily they lean on AI in their decision-making.
One of the intended messages was: “We’re all using it, so just be open about it,” said Marie Soulière, who serves on COPE’s council and is a senior manager with the Switzerland-based publishing company, Frontiers. That way “people could learn and start to trust” these tools.
Indeed, many publishers and journal editors who spoke with The Chronicle said they’re using one or more AI tools to inform decisions. Frontiers, for one, has an in-house “Artificial Intelligence Review Assistant,” or AIRA, which its journals’ editors, reviewers, and research-integrity team can use to conduct dozens of different checks during the review process. In seconds, AIRA can scan a manuscript and — using a content-matching algorithm — produce a “suitability score” with a list of journals where that paper may fit. It can assess a paper’s grammar, spelling, and sentence structure to produce a language score, indicating how much copy editing would be required. When given the names of an author and a prospective peer reviewer, it can scour Frontiers’s databases and various public sources for undisclosed conflicts of interest, such as past co-authorships and shared grants.
Another, Taylor & Francis, says some journal editors use Crossref Similarity Check, powered by iThenticate, to conduct plagiarism checks — a commonly cited use. The publisher’s ethics and integrity team is also testing a new tool that flags papers that contain trademark signs of being produced by paper mills, like manipulated images. (Paper mills produce and sell fraudulent manuscripts, often based on fabricated data, under the guise of legitimate research.)
Spokespeople for Elsevier and Cambridge University Press & Assessment both declined to name specific tools for proprietary reasons, but they wrote in emails that their journals can use AI to assist with tasks such as identifying suitable peer reviewers for papers. Andrew Davis, vice president for communications at Elsevier, added that the company “employs over 2,500 technologists exploring the latest technologies and developments” in AI.
Dipping Toes in the Water
Not everyone is keen to lead the pack, though.
Dominic Klyve, a mathematics professor at Central Washington University and editor of The College Mathematics Journal, says he could see value in an AI tool that can scan a submission — he gets about 350 a year — and give him a quick recommendation on whether it is a good fit. For his journal, a good fit means “not too technical.” Well-written prose. A paper that tells a story.
But the journal’s subject matter, and unique style, make him hesitant to use one.
“Mathematics writing involves equations and odd formatting. … Some AI tools absolutely can read it, but it’s a barrier,” he said. And “I don’t think I trust it yet to differentiate between good writing and great writing, which is something we would need.”
Amy Drew, director of publications for the Association for Psychological Science, said the association’s flagship journal, Psychological Science, and the other five academic journals she oversees use AI tools on a limited basis; for example, editors may use the Web of Science Reviewer Locator to identify potential peer reviewers. (She estimated that the association’s journals receive between 2,500 and 3,000 paper submissions annually, and publish between 400 and 500 of them). But the journals’ emphasis on a “high touch” review process makes increasing reliance on AI tools less appealing.
“We’re relying on these editors’ expertise to judge whether these manuscripts are suitable for our journals,” she said. “I just don’t see AI replacing that judgment and that expertise.”
Megan Fritts, an assistant professor of philosophy at the University of Arkansas at Little Rock, says she’s experienced firsthand what can happen when AI supersedes human judgment.
Back in March, she and a co-author eagerly sent their paper on the intersections between science and philosophy to a journal run by a major academic publisher. After about a month of silence, they received an emailed rejection. “Our decision,” it read, “is based on our automated plagiarism report of your submission running at 42 percent.”
That reported plagiarism was of her doctoral dissertation, which she’d previously uploaded to an online scholarly database but never formally published anywhere. A human being, she believes, would have looked into, and recognized, that.
Fritts said she emailed the journal’s editors asking for a second look, but she never heard back. It left a bad taste in her mouth. “Getting a paper published in academic philosophy takes a very long time,” she said. “Wasted time sets you back.”
The Case for Skepticism
All sources The Chronicle spoke with agreed that healthy skepticism about the use of AI during the editorial-review process isn’t a bad thing.
For example, if a publisher or journal used an AI tool to assess the language in a paper and automatically rejected those that received a low score, non-native English speakers’ work could be disproportionately turned away, said Soulière, the COPE Council member. She added that AI tools can furnish biased recommendations if they are trained on biased data.
One theoretical scenario: If an AI tool is trained on a journal’s archives but that journal has historically published papers whose authors are predominantly from, say, the United Kingdom or Germany, the tool is going to view future papers from those countries more favorably, Soulière said. That’s because when AI tools ingest text, they’re capturing all kinds of information — names of colleges, keywords, even patterns in phrases and language style — that can give away a paper’s origins.
There’s also the reality that companies pushing out AI tools can be opaque about the ways their products collect and use data. Chhavi Chauhan, an AI ethicist, worries that if editors or reviewers feed parts of unpublished manuscripts into generative AI tools like ChatGPT, the personal and proprietary information contained in those manuscripts could become training material. The tool could then, essentially, regurgitate that material as an answer to another user’s query.
Journal editors must ensure that a paper submitted to them “stays novel until the time it’s disseminated,” said Chauhan, who serves as director of scientific outreach at the American Society for Investigative Pathology.
That particular risk has been explicitly addressed in a few places. Taylor & Francis’s peer-reviewer guidelines, for example, forbids uploading unpublished manuscripts or related data into tools “that do not guarantee confidentiality, are accessible by the public and/or may store or use this information for their own purposes (for example, generative AI tools like ChatGPT).”
Asked about the products he uses as a peer reviewer, Machajewski, of the University of Illinois at Chicago, wrote in an email that tools like Quivr can be installed on a personal computer or institutional server, preventing uploads to “third-party systems.” The risk is higher, he noted, when people use public, cloud-based tools and/or those that aren’t licensed through their institution. Still, Machajewski wrote that there is always some risk when using AI tools — just as there is risk with using now-ubiquitous tools like a personal Google Drive.
At this point, AI is “just another button on the keyboard” that can’t be removed, he said. The focus, then, should be on understanding when and how to use it, and within what parameters.
What Should Be Done?
Among the publishers who responded to The Chronicle, those that didn’t have AI-specific policies for editors and reviewers cited an openness to developing them.
A spokesperson for Cambridge University Press & Assessment wrote that the publisher would “consider issuing further policies and guidance dependent on need as AI develops further.” One for Wiley wrote that the company is “in the process of developing guidance on the use of AI in editorial decisions.” Fred Fenter, chief executive editor at Frontiers, wrote that while the company’s “high-level public policy” for peer review touches on AI, it isn’t ruling out “developing a more-detailed policy” in the future.
Soulière, too, noted that while COPE, the membership organization, has not yet crafted formal guidelines following its 2021 discussion document, “we want to do it.” (The organization released a formal statement in February relating to AI and authorship.)
On a more granular level, Fritts said she’d like to see journals provide clearer guidance to prospective authors — telling them which AI tools they may employ in the review process, and the implications, so authors can make informed adjustments before submission (in Fritts’s case, perhaps removing her dissertation from the online database so it wouldn’t trigger the plagiarism checker).
Chauhan added that dialogue boxes or disclosures could be appended to published manuscripts, letting readers know if and how AI was used during editorial review.
“I think that’s a very valid starting point,” she said. It would signal to readers that “the process was transparent and trustworthy throughout.”
Trust, after all, is the metaphorical Everest for artificial intelligence.
The Chronicle will continue to explore how different sectors touching higher ed are thinking about and adopting AI tools. Tips? Email taylor.swaak@chronicle.com.
