Writing anonymous peer reviews is an academic “black art.” Such assessments are vital to scholarly publishing but we receive no formal training in how to write one.
Drawing on my own experiences as an author and editor, I would like to offer guidance about this for young scholars. Some of my observations are particularly germane to the social sciences, but many recommendations pertain to publishing across all disciplines.
The process. If a manuscript meets a minimum level of scholarly quality, the journal editor compiles a list of academics who are asked to review the submission. Reviewers are chosen because an editor knows them personally or because they are recommended by colleagues. Sometimes an editor identifies reviewers by scrutinizing the relevant literature or by asking authors to recommend some names.
As a prospective reviewer, you would receive an e-mail inviting you to review a manuscript and an abstract of the submission. You can then accept or decline via e-mail or, increasingly, a journal’s online submission-management system.
If asked, decide quickly if you will do the review. A scholar’s career can hang on the fate of one significant publication, and if that piece stalls interminably at the review stage it can be a professional disaster. If you decline, inform the editor quickly, as it is unfair to expect an editor to wait weeks for a response, only to then have the invitation declined (or never answered at all). Editors always appreciate it when, if you decline a request, you recommend other reviewers.
After agreeing to do the review, you will be sent the manuscript or details about how to access it online. The papers are confidential, so do not reference the submission and absolutely do not draw upon its findings or data for your own work. You will also be given a date by which to complete your review—usually one to three months. Immediately mark that deadline on your calendar and make sure you finish on time. If your situation changes and you have to cancel or delay your review, let the editor know immediately in case a replacement must be found.
Why write a peer review of a manuscript? Because it is part of our scholarly responsibilities. You will not be paid and it will take time away from your own work. But academic publishing depends on peer reviewers volunteering their time. You have undoubtedly benefited, or will benefit in the future, from this arrangement.
Given that most (but not all) journals aim to secure three anonymous reviews, you should aim to review a minimum of three manuscripts for every article you publish. Unfortunately, intensifying professional demands means that editors spend an inordinate amount of time and energy trying to find willing, competent, and prompt peer reviewers.
A second reason to evaluate manuscripts is that doing so helps keep you abreast of new developments in your field. Faculty members may find it difficult to maintain a reading schedule in light of their own pressing research, teaching, and publishing demands. A consistent pattern of writing peer reviews can keep you on top of the literature.
Finally, reviewing manuscripts allows you to shape the discipline. You undoubtedly have strong, reasoned opinions about what constitutes sound scholarship, and you want to ensure that your own subdiscipline is represented by the best works. Serving as a peer reviewer makes you a gatekeeper, as you put your own small stamp on the types of works that are recognized and rewarded.
Should you review this manuscript? There are good reasons why you might decline to evaluate a specific manuscript. If you are a graduate student, for example, make your status known to the editor, as some journals do not want graduate-student reviewers. Likewise, be honest about your own expertise. Is the paper in an area (theoretically, methodologically, substantively) in which you have a solid grounding? Even if you are not an expert in the area in which the paper makes its main contribution, the editor might still want you to review the piece, as editors sometimes want a manuscript assessed by someone with general knowledge about the discipline. But be sure to clarify your situation.
Be honest about whether you can provide a fair assessment. If, for example, you are strongly opposed to the entire field of sociobiology, you should simply decline to be a peer reviewer for manuscripts in that area.
Likewise, do you know the author? While the double-blind process means that authors and reviewers are not identified, it is sometimes easy to determine the author. That is particularly true in a small field, and as you develop personal connections with more and more colleagues.
Inform the editor if you know the author, but also decide whether you can, nonetheless, offer a fair assessment of the manuscript. Some people believe you should never review a manuscript by an author whom you know, but such an absolute ban would preclude many of the most senior and knowledgeable people in a subdiscipline from reviewing a lot of papers.
Certainly, do not review the manuscripts of your supervisors or close friends. But beyond such intimate relationships is a world of collegial gray. Here again, transparency is the best policy. Inform the editor of your situation and whether you think you can be objective. The journal staff will decide if they still want you to review the submission.
Writing a peer review. Your review, combined with the editor’s own assessment of the paper, will serve as the basis for the publishing decision. Sometimes journals give reviewers formal guidelines. They might include explicitly asking a reviewer to assess the paper’s methodology, theoretical contribution, interest to a wide readership, and the like. Such directions are increasingly laid out in forms on the journal’s submission-management Web page.
In terms of length, most reviews are one to two single-spaced pages. Think of the task as involving three sections. The first is a brief paragraph that summarizes the manuscript. This section is a way to remind the editor of exactly what the manuscript is about and what it presents as its contributions. Do not assume that the editor has read the submission in the same fine-grained detail as you have. You have two audiences for your review: the author and the editor. Keep both readers in mind when preparing your comments.
The second section is the most important, as it is here where you provide your opinions on the quality of the manuscript. Almost anything is fair game in making such an assessment.
At the most basic level, check to ensure that the submission meets the journal’s formal requirements—that it is not too long, that its topic or methodology is appropriate for the journal. Does the piece have a logical structure, including introduction, conclusion, and some kind of thesis statement? Has the author identified a “problem” the paper is trying to deal with? Such problems could be very different depending on the discipline, ranging from a long-standing issue of logic, a gap in the empirical literature, or a political paradox. Does the manuscript have an argument? Is the writing reasonably clear and coherent? Is there anything new here, or is it just a demonstration of a well-known fact? Does it engage with the relevant current literatures? If it is an empirical paper, what are your views on the quality of the data or the appropriateness of the methodology? Are the data presented clearly? Do all of the sections belong or does it read like two or more papers have been cobbled together?
Any of these considerations (plus a litany of others) can be the basis for fair commentary and criticism. However, be sure that you are engaging with the main substance of the analysis and not fixating on smaller matters that can be dealt with in revisions or copy-editing. Also, evaluate the manuscript on its own terms and do not criticize it for failing to live up to your views on the type of paper the author should have written.
The final section of your review can tackle smaller, more particular issues. This is the place to identify page-specific points, such as where the argument gets murky, terminology is incorrect, better referencing is required, and the like. Authors can find such detailed guidance extremely useful, but do not fixate on the minutiae of formatting. Remember that editors are primarily concerned about whether a paper makes a contribution to the field. They often have staff to deal with formatting issues. At the same time, if errors in composition, editing, and presentation are so egregious as to raise questions about whether the manuscript should be published, then you should draw those weaknesses to the editor’s attention.
It is imperative that you remain civil and provide constructive comments. Anonymity can tempt people to write needlessly mean-spirited remarks. Authors can find such evaluations devastating, particularly if they are just starting their publishing careers. If you absolutely hate a manuscript, make sure you still find a way to say something positive, even if you do no more than compliment the author for studying an important topic.
And while civility is paramount, it is equally important to tell the truth. If there are problems with a manuscript, outline them in some detail. Authors are justifiably annoyed if a reviewer says only positive things and then recommends rejection.
If you find that you cannot understand the paper’s argument, do not assume that is a problem with you. There is sometimes an inclination among junior scholars to think that if a piece is convoluted to the point of indecipherability, you are simply too pedestrian a thinker to grasp the brilliance of the submission. Remember, as a mature, educated reviewer you are also part of the journal’s desired readership. If you cannot understand the piece (at least after a couple of careful readings), it is a problem with the manuscript, not with you.
Remain anonymous when writing your comments. Most academics recognize that criticism and rejection are an inherent part of our enterprise. A small number, however, will take even minor criticism as an opportunity to nurture a lifelong vendetta.
You do not want such individuals to know that you reviewed their manuscript. Remove identifying information from your evaluation, and if you are submitting your review as an electronic file be sure to delete your name and institution from the “preferences” section of your word-processing program. On your CV or annual report you might list which journals you have reviewed for, but do not identify the manuscripts you evaluated.
Also, be extremely careful about referencing your own works in a peer review. There is no easier way to identify the author of a review than by the fact that it recommends (or insists) that the author engage with some esoteric piece that appears to have only a tangential relationship to the manuscript.
In addition to your written assessment of the submission, you will be asked to offer a succinct recommendation as to whether it should be published. Typically your options are:
- Accept: The manuscript is of sufficient importance and quality that it can be published essentially as is, with only minor copy-editing changes.
- Revisions required: Changes are necessary, but these are not major. Here the editor might personally determine whether the revisions are likely to be sufficient enough that paper doesn’t need to be sent out for another round of peer reviews.
- Revise and resubmit: The manuscript has promise, but it needs to make some significant changes before it might be publishable. Typically an article in this category would undergo a second round of reviews. (Note: Just because you have recommended “revise and resubmit” does not mean you are obliged to rereview the manuscript).
- Reject: The manuscript is not publishable and could not be made publishable with a reasonable amount of revision.
- Not suitable for this journal: The piece is more appropriate for a journal that focuses, for example, on different topics, methodologies, or theoretical orientations.
Do not put your recommendation on whether to publish in the written part of your review that is shared with the author. Instead, click off the appropriate box in the online system, or communicate your decision confidentially to the editor. Your “decision” is ultimately a recommendation; it is the editor’s job to decide whether the piece should be published.
Editors can find themselves in a delicate situation if, after reading all the reviews, they decide that the piece should not be published, but one or more of the written reviews to the author have said that the piece could be revised or accepted.
You have the opportunity to provide directly to the editor comments that are not shared with the author. Here you can be more succinct and also satisfy any overwhelming need to say nasty things about the manuscript. Do not be shy about using that option, as editors appreciate candid feedback.
Also, ask to see the other reviews and the editor’s letter informing the author of the decision about the manuscript. Reviewing is a practical art, so it can be invaluable to see how other reviewers approached the same submission and how an editor responds to different reviews. Some journals will not share that information while others do so as a matter of course. No one will be offended if you ask.
There are more nuances to peer review, but hopefully these comments will help guide new reviewers through the process. Attentive readers will recognize that many of the lessons I have outlined here could also double as advice for how to prepare you own manuscripts.
