Skip to content
ADVERTISEMENT
Sign In
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Virtual Events
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
  • More
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Virtual Events
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
    Upcoming Events:
    An AI-Driven Work Force
    AI and Microcredentials
Sign In
News

Leading Humanities Journal Debuts ‘Open’ Peer Review, and Likes It

By Jennifer Howard July 26, 2010

Getting published in a humanities journal usually works like this: Submit an article, then hope for the best as the editors send it to a few hand-picked specialists for critique. The reviewers and the authors are not supposed to know one another’s identity.

To continue reading for FREE, please sign in.

Sign In

Or subscribe now to read with unlimited access for as low as $10/month.

Don’t have an account? Sign up now.

A free account provides you access to a limited number of free articles each month, plus newsletters, job postings, salary data, and exclusive store discounts.

Sign Up

Getting published in a humanities journal usually works like this: Submit an article, then hope for the best as the editors send it to a few hand-picked specialists for critique. The reviewers and the authors are not supposed to know one another’s identity.

But now scholars are asking whether this double-blind peer-review system is still the best way to pass judgment. The Internet makes it possible to share work with many people openly and simultaneously, so why not tap the public wisdom of a crowd? One of the top journals in literary studies, Shakespeare Quarterly, decided to put that question to the test.

For this year’s fall issue, a special publication devoted to Shakespeare and new media, the journal offered contributors the chance to take part in a partially open peer-review process. Authors could opt to post drafts of their articles online, open them up for anyone to comment on, and then revise accordingly. The editors would make the final call about what to publish (hence the “partially open” label). As far as the editors know, it’s the first time a traditional humanities journal has tried out a version of crowd-sourcing in lieu of double-blind review.

The verdict from several scholars who took part: mostly a thumbs up, with a few cautionary notes and a dollop of “It’s about time” mixed in.

“It was on the whole a successful experiment,” said Martin Mueller, a professor of English and classics at Northwestern University, who took part as a reviewer.

Michael Witmore, a professor of English at the University of Wisconsin at Madison, co-authored an article on the use of statistical analysis and a text-tagging database to reveal linguistic patterns in Shakespeare. He and his co-author “got some terrific ideas and some citations” from the comments of the six or so people who actively responded to the article, he said. “It produced a more interesting paper.”

Scholars and editors in the sciences have been trying out open peer review for some time, with not entirely rosy results. The journal Nature did a test run in 2006. In a published overview, the editors concluded the venture had not been a success. Many authors expressed interest but few participated, and the quantity and quality of the comments were disappointing.

Interviews with participants in Shakespeare Quarterly‘s open peer-review trial, however, suggest this attempt went much better than Nature‘s did. At least one participant pointed out that the humanities’ subjective, conversational tendencies may make them well suited to open review—better suited, perhaps, than the sciences.

Katherine Rowe, chair of the English department at Bryn Mawr College, guest-edited the special issue. She and the editorial board decided that the issue’s new-media theme offered a chance to investigate how scholarly authority works in a networked environment.

“This was genuinely an experiment,” Ms. Rowe said. “We didn’t know what would result.”

ADVERTISEMENT

The journal’s publisher, the Johns Hopkins University Press, supported the idea. MediaCommons, a digital scholarly network set up to encourage such experiments and conversations, agreed to host the project. So Ms. Rowe and other editors put out a call for papers, culled submissions, then offered authors still in the running a chance to post drafts online. All accepted.

Rounding Up Experts

Ms. Rowe invited about 90 scholars, including Northwestern’s Mr. Mueller, to comment. Anybody willing to publish thoughts under his or her own name could join in, but the guest editor wanted recognized authorities as part of the field.

“‘What’s the nature of expertise?’ is one of the questions that really gets opened up by an open process,” Ms. Rowe said. “Everybody wanted to be sure that experts would be involved.” By her count, about 40 commenters, invited and self-selected, finally participated.

All told, four articles and three review essays were posted on MediaCommons during a two-month review period this past spring. (The articles and comments remain archived on the MediaCommons site.) As it turned out, all seven submissions will appear in the issue.

ADVERTISEMENT

Some of the authors acknowledged doubts going into the experiment. Would open review be rigorous enough? Was it risky to post work in progress? But “the results were terrific,” said Mr. Witmore, who wrote his paper on Shakespearean linguistic analysis with Jonathan Hope, a reader in the English department of the University of Strathclyde, in Glasgow. “It’s very different from getting a two-paragraph reader’s report from a journal,” Mr. Witmore said. “In this case, what you get is individual readers from a wide range of subspecialties zooming in on a particular paragraph, saying ‘Tell me more about this’ or ‘Why did you do this?’ It seemed more like a dialogue.”

Another scholar, Alan Galey, submitted an article about Shakespeare and the history of information. An assistant professor on the Faculty of Information at the University of Toronto, he worried that an article vetted this way might carry less professional weight—a matter of particular concern to a junior professor going for tenure. “It was very much going on faith in a way,” he said.

Mr. Galey’s dean told him to make sure the process would be rigorous and fair. The stature of the journal also helped reassure him on that point. So did Ms. Rowe’s willingness to answer questions and her decision to invite established scholars to join in. Many crowd-sourcing experiments depend on scale, Mr. Galey pointed out, but this relied “on relationships among scholars where you know you can trust somebody. It wasn’t a Wild West by any means. It was as controlled a process as traditional peer review. It was just controlled in a different way.”

Mr. Galey wound up feeling that the experiment paid off. “I got better feedback from this process than I’ve had from any other peer-review process,” he said.

ADVERTISEMENT

Another participant, Ayanna Thompson, an associate professor of English at Arizona State University, wrote an article focused on race in performance-based teaching of Shakespeare. She called the experiment “a fascinating process.” But she also found it stressful, and saw evidence that some old ways die hard.

Better Feedback

On the positive side, she got lots of feedback—"the equivalent of eight single-spaced pages of comments,” she told The Chronicle via e-mail. She felt that the lack of anonymity encouraged reviewers to engage with the material “in a much more thoughtful and thorough way than in blind reviews because their names were attached to the comments.” That helped her clarify and revise sections of her argument. But going through a public critique “was a stressful experience, and I kept saying to myself, ‘I’m glad I have tenure,’” she said.

She also discovered that academic status still played a role. “The people who were commenting online were very established, senior scholars. In fact, several junior scholars notified me offline that they had comments, but that they did not want to post them in case they contradicted the senior scholars,” she said. “So while the process was supposed to democratize the review process, it did keep some of the old hierarchies in place.”

Still, Ms. Thompson is glad she took part. “The revised article is much stronger for the feedback, and I feel lucky to have received so much deep engagement with my work,” she said.

ADVERTISEMENT

David Schalkwyk is editor of the journal and director of research at the Folger Shakespeare Library. He called the experiment a success and said that Shakespeare Quarterly plans to try it again for its future special issues, whose focused topics are most likely to attract a critical mass of knowledgeable reviewers.

The system did, however, come at a cost from an editor’s perspective, he said. For instance, editors and writers had to keep tabs on the evolving discussion and build in extra time for revisions; authors also tended to make their articles longer and longer in response to multiple comments. “I think we underestimated the amount of time that was required,” Mr. Schalkwyk said.

That’s what keeps him from adopting the approach for every issue. He said, “My reservation about applying open peer review to normal issues is entirely practical. It’s not philosophical. If I could do it, I would.”

For Mr. Mueller, the reviewer from Northwestern, the journal’s experiment was welcome—and past due. “I think it will take a decade for scholarly communication to find ways of fitting itself into this new technological environment,” he said. “It takes time. The human problems always take more time to manage and solve than the technological problems.”

We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Tags
Scholarship & Research
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
About the Author
Jennifer Howard
Jennifer Howard, who began writing for The Chronicle in 2005, covered publishing, scholarly communication, libraries, archives, digital humanities, humanities research, and technology.
ADVERTISEMENT
ADVERTISEMENT

More News

Photo illustration showing Santa Ono seated, places small in the corner of a dark space
'Unrelentingly Sad'
Santa Ono Wanted a Presidency. He Became a Pariah.
Illustration of a rushing crowd carrying HSI letters
Seeking precedent
Funding for Hispanic-Serving Institutions Is Discriminatory and Unconstitutional, Lawsuit Argues
Photo-based illustration of scissors cutting through paper that is a photo of an idyllic liberal arts college campus on one side and money on the other
Finance
Small Colleges Are Banding Together Against a Higher Endowment Tax. This Is Why.
Pano Kanelos, founding president of the U. of Austin.
Q&A
One Year In, What Has ‘the Anti-Harvard’ University Accomplished?

From The Review

Photo- and type-based illustration depicting the acronym AAUP with the second A as the arrow of a compass and facing not north but southeast.
The Review | Essay
The Unraveling of the AAUP
By Matthew W. Finkin
Photo-based illustration of the Capitol building dome propped on a stick attached to a string, like a trap.
The Review | Opinion
Colleges Can’t Trust the Federal Government. What Now?
By Brian Rosenberg
Illustration of an unequal sign in black on a white background
The Review | Essay
What Is Replacing DEI? Racism.
By Richard Amesbury

Upcoming Events

Plain_Acuity_DurableSkills_VF.png
Why Employers Value ‘Durable’ Skills
Warwick_Leadership_Javi.png
University Transformation: a Global Leadership Perspective
  • Explore Content
    • Latest News
    • Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Virtual Events
    • Chronicle Store
    • Chronicle Intelligence
    • Jobs in Higher Education
    • Post a Job
  • Know The Chronicle
    • About Us
    • Vision, Mission, Values
    • DEI at The Chronicle
    • Write for Us
    • Work at The Chronicle
    • Our Reporting Process
    • Advertise With Us
    • Brand Studio
    • Accessibility Statement
  • Account and Access
    • Manage Your Account
    • Manage Newsletters
    • Individual Subscriptions
    • Group and Institutional Access
    • Subscription & Account FAQ
  • Get Support
    • Contact Us
    • Reprints & Permissions
    • User Agreement
    • Terms and Conditions
    • Privacy Policy
    • California Privacy Policy
    • Do Not Sell My Personal Information
1255 23rd Street, N.W. Washington, D.C. 20037
© 2025 The Chronicle of Higher Education
The Chronicle of Higher Education is academe’s most trusted resource for independent journalism, career development, and forward-looking intelligence. Our readers lead, teach, learn, and innovate with insights from The Chronicle.
Follow Us
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin