Skip to content
ADVERTISEMENT
Sign In
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Events
    • Virtual Events
    • Chronicle On-The-Road
    • Professional Development
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
  • More
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Events
    • Virtual Events
    • Chronicle On-The-Road
    • Professional Development
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
    Upcoming Events:
    Student Housing
    Serving Higher Ed
    Chronicle Festival 2025
Sign In
News

Leading Humanities Journal Debuts ‘Open’ Peer Review, and Likes It

By Jennifer Howard July 26, 2010

Getting published in a humanities journal usually works like this: Submit an article, then hope for the best as the editors send it to a few hand-picked specialists for critique. The reviewers and the authors are not supposed to know one another’s identity.

To continue reading for FREE, please sign in.

Sign In

Or subscribe now to read with unlimited access for as low as $10/month.

Don’t have an account? Sign up now.

A free account provides you access to a limited number of free articles each month, plus newsletters, job postings, salary data, and exclusive store discounts.

Sign Up

Getting published in a humanities journal usually works like this: Submit an article, then hope for the best as the editors send it to a few hand-picked specialists for critique. The reviewers and the authors are not supposed to know one another’s identity.

But now scholars are asking whether this double-blind peer-review system is still the best way to pass judgment. The Internet makes it possible to share work with many people openly and simultaneously, so why not tap the public wisdom of a crowd? One of the top journals in literary studies, Shakespeare Quarterly, decided to put that question to the test.

For this year’s fall issue, a special publication devoted to Shakespeare and new media, the journal offered contributors the chance to take part in a partially open peer-review process. Authors could opt to post drafts of their articles online, open them up for anyone to comment on, and then revise accordingly. The editors would make the final call about what to publish (hence the “partially open” label). As far as the editors know, it’s the first time a traditional humanities journal has tried out a version of crowd-sourcing in lieu of double-blind review.

The verdict from several scholars who took part: mostly a thumbs up, with a few cautionary notes and a dollop of “It’s about time” mixed in.

“It was on the whole a successful experiment,” said Martin Mueller, a professor of English and classics at Northwestern University, who took part as a reviewer.

Michael Witmore, a professor of English at the University of Wisconsin at Madison, co-authored an article on the use of statistical analysis and a text-tagging database to reveal linguistic patterns in Shakespeare. He and his co-author “got some terrific ideas and some citations” from the comments of the six or so people who actively responded to the article, he said. “It produced a more interesting paper.”

Scholars and editors in the sciences have been trying out open peer review for some time, with not entirely rosy results. The journal Nature did a test run in 2006. In a published overview, the editors concluded the venture had not been a success. Many authors expressed interest but few participated, and the quantity and quality of the comments were disappointing.

Interviews with participants in Shakespeare Quarterly‘s open peer-review trial, however, suggest this attempt went much better than Nature‘s did. At least one participant pointed out that the humanities’ subjective, conversational tendencies may make them well suited to open review—better suited, perhaps, than the sciences.

Katherine Rowe, chair of the English department at Bryn Mawr College, guest-edited the special issue. She and the editorial board decided that the issue’s new-media theme offered a chance to investigate how scholarly authority works in a networked environment.

“This was genuinely an experiment,” Ms. Rowe said. “We didn’t know what would result.”

ADVERTISEMENT

The journal’s publisher, the Johns Hopkins University Press, supported the idea. MediaCommons, a digital scholarly network set up to encourage such experiments and conversations, agreed to host the project. So Ms. Rowe and other editors put out a call for papers, culled submissions, then offered authors still in the running a chance to post drafts online. All accepted.

Rounding Up Experts

Ms. Rowe invited about 90 scholars, including Northwestern’s Mr. Mueller, to comment. Anybody willing to publish thoughts under his or her own name could join in, but the guest editor wanted recognized authorities as part of the field.

“‘What’s the nature of expertise?’ is one of the questions that really gets opened up by an open process,” Ms. Rowe said. “Everybody wanted to be sure that experts would be involved.” By her count, about 40 commenters, invited and self-selected, finally participated.

All told, four articles and three review essays were posted on MediaCommons during a two-month review period this past spring. (The articles and comments remain archived on the MediaCommons site.) As it turned out, all seven submissions will appear in the issue.

ADVERTISEMENT

Some of the authors acknowledged doubts going into the experiment. Would open review be rigorous enough? Was it risky to post work in progress? But “the results were terrific,” said Mr. Witmore, who wrote his paper on Shakespearean linguistic analysis with Jonathan Hope, a reader in the English department of the University of Strathclyde, in Glasgow. “It’s very different from getting a two-paragraph reader’s report from a journal,” Mr. Witmore said. “In this case, what you get is individual readers from a wide range of subspecialties zooming in on a particular paragraph, saying ‘Tell me more about this’ or ‘Why did you do this?’ It seemed more like a dialogue.”

Another scholar, Alan Galey, submitted an article about Shakespeare and the history of information. An assistant professor on the Faculty of Information at the University of Toronto, he worried that an article vetted this way might carry less professional weight—a matter of particular concern to a junior professor going for tenure. “It was very much going on faith in a way,” he said.

Mr. Galey’s dean told him to make sure the process would be rigorous and fair. The stature of the journal also helped reassure him on that point. So did Ms. Rowe’s willingness to answer questions and her decision to invite established scholars to join in. Many crowd-sourcing experiments depend on scale, Mr. Galey pointed out, but this relied “on relationships among scholars where you know you can trust somebody. It wasn’t a Wild West by any means. It was as controlled a process as traditional peer review. It was just controlled in a different way.”

Mr. Galey wound up feeling that the experiment paid off. “I got better feedback from this process than I’ve had from any other peer-review process,” he said.

ADVERTISEMENT

Another participant, Ayanna Thompson, an associate professor of English at Arizona State University, wrote an article focused on race in performance-based teaching of Shakespeare. She called the experiment “a fascinating process.” But she also found it stressful, and saw evidence that some old ways die hard.

Better Feedback

On the positive side, she got lots of feedback—"the equivalent of eight single-spaced pages of comments,” she told The Chronicle via e-mail. She felt that the lack of anonymity encouraged reviewers to engage with the material “in a much more thoughtful and thorough way than in blind reviews because their names were attached to the comments.” That helped her clarify and revise sections of her argument. But going through a public critique “was a stressful experience, and I kept saying to myself, ‘I’m glad I have tenure,’” she said.

She also discovered that academic status still played a role. “The people who were commenting online were very established, senior scholars. In fact, several junior scholars notified me offline that they had comments, but that they did not want to post them in case they contradicted the senior scholars,” she said. “So while the process was supposed to democratize the review process, it did keep some of the old hierarchies in place.”

Still, Ms. Thompson is glad she took part. “The revised article is much stronger for the feedback, and I feel lucky to have received so much deep engagement with my work,” she said.

ADVERTISEMENT

David Schalkwyk is editor of the journal and director of research at the Folger Shakespeare Library. He called the experiment a success and said that Shakespeare Quarterly plans to try it again for its future special issues, whose focused topics are most likely to attract a critical mass of knowledgeable reviewers.

The system did, however, come at a cost from an editor’s perspective, he said. For instance, editors and writers had to keep tabs on the evolving discussion and build in extra time for revisions; authors also tended to make their articles longer and longer in response to multiple comments. “I think we underestimated the amount of time that was required,” Mr. Schalkwyk said.

That’s what keeps him from adopting the approach for every issue. He said, “My reservation about applying open peer review to normal issues is entirely practical. It’s not philosophical. If I could do it, I would.”

For Mr. Mueller, the reviewer from Northwestern, the journal’s experiment was welcome—and past due. “I think it will take a decade for scholarly communication to find ways of fitting itself into this new technological environment,” he said. “It takes time. The human problems always take more time to manage and solve than the technological problems.”

We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Tags
Scholarship & Research
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
About the Author
Jennifer Howard
Jennifer Howard, who began writing for The Chronicle in 2005, covered publishing, scholarly communication, libraries, archives, digital humanities, humanities research, and technology.
ADVERTISEMENT
ADVERTISEMENT

More News

UCLA students, researchers and demonstrators rally during a "Kill the Cuts" protest against the Trump administration's funding cuts on research, health and higher education at the University of California Los Angeles (UCLA) in Los Angeles on April 8, 2025.
Scholarship & Research
Trump Proposed Slashing the National Science Foundation’s Budget. A Key Senate Committee Just Refused.
Illustration of a steamroller rolling over a colorful road and leaving gray asphalt in its wake.
Newly Updated
Oregon State U. Will End a Renowned Program That Aimed to Reduce Bias in Hiring
Dr. Gregory Washington, president of George Mason University.
Another probe
George Mason President Discriminated Against White People After George Floyd Protests, Justice Dept. Says
Protesters gather outside the Department of Education headquarters in Washington, D.C., on Feb. 14, 2025 to protest the Trump administrations cuts at the agency.
An Uncertain Future
The Education Dept. Got a Green Light to Shrink. Here Are 3 Questions About What’s Next.

From The Review

Photo-based illustration with repeated images of a student walking, in the pattern of a graph trending down, then up.
The Review | Opinion
7 Ways Community Colleges Can Boost Enrollment
By Bob Levey
Illustration of an ocean tide shaped like Donald Trump about to wash away sandcastles shaped like a college campus.
The Review | Essay
Why Universities Are So Powerless in Their Fight Against Trump
By Jason Owen-Smith
Photo-based illustration of a closeup of a pencil meshed with a circuit bosrd
The Review | Essay
How Are Students Really Using AI?
By Derek O'Connell

Upcoming Events

07-31-Turbulent-Workday_assets v2_Plain.png
Keeping Your Institution Moving Forward in Turbulent Times
Ascendium_Housing_Plain.png
What It Really Takes to Serve Students’ Basic Needs: Housing
Lead With Insight
  • Explore Content
    • Latest News
    • Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Events
    • Chronicle Store
    • Chronicle Intelligence
    • Jobs in Higher Education
    • Post a Job
  • Know The Chronicle
    • About Us
    • Vision, Mission, Values
    • DEI at The Chronicle
    • Write for Us
    • Work at The Chronicle
    • Our Reporting Process
    • Advertise With Us
    • Brand Studio
    • Accessibility Statement
  • Account and Access
    • Manage Your Account
    • Manage Newsletters
    • Individual Subscriptions
    • Group and Institutional Access
    • Subscription & Account FAQ
  • Get Support
    • Contact Us
    • Reprints & Permissions
    • User Agreement
    • Terms and Conditions
    • Privacy Policy
    • California Privacy Policy
    • Do Not Sell My Personal Information
1255 23rd Street, N.W. Washington, D.C. 20037
© 2025 The Chronicle of Higher Education
The Chronicle of Higher Education is academe’s most trusted resource for independent journalism, career development, and forward-looking intelligence. Our readers lead, teach, learn, and innovate with insights from The Chronicle.
Follow Us
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin