Advice

Choosing an External Reviewer

iStock

June 06, 2017

S ay your department gets a green light to bring in an outside expert — typically, an academic in the discipline — to do a comprehensive evaluation of your program. How does the department decide who will best fill those shoes?

A lot is at stake here. Academic program reviews — or "APRs" as they’re known — are common in higher education and represent a form of internal quality control. They are also that rare opportunity when the administration may well invest in your program’s improvement, based on the results of the evaluation.

That is why choosing an appropriate external reviewer (and sometimes more than one) is a key aspect of any program review. This is our third column on the APR process — based on our experience as external reviewers for more than 70 departments in our field (psychology). In Part 1, we looked at ways to make these periodic assessments useful. In Part 2, we offered a field guide to the faculty characters that an external reviewer can expect to meet in evaluating a department.

Now we switch back to the department’s perspective and offer some basic guidelines on finding the right reviewer. Sometimes a department brings in one reviewer, sometimes two or three. To be most effective, it’s best to do an APR every five to 10 years and to invite a different guest appraiser each time.

How reviewers are selected varies from campus to campus. Sometimes the department chair picks the name. Sometimes the department puts forth a list of potential reviewers and the provost or the dean makes the choice. Whoever makes the final decision should take the following tips into consideration.

Don’t pick your old chum from graduate school. Picking someone you know well — like an old pal from your academic salad days — is unlikely to lead to constructive suggestions for the department. It will be too tempting for everyone to spend time revisiting old "war stories" instead of focusing on the matter at hand. Instead, select an acquaintance or a stranger with a reputation as fair and objective — if only to avoid leaving the impression that the review is fixed.

Consider your neighbor carefully. Inviting a colleague from a neighboring institution to do the evaluation is fine so long as your two institutions are not competitors on the admissions front. Although your department’s self-study document is unlikely to contain any operational secrets, there may well be some information — things like grade distributions, salary data, or internal admissions statistics — that you might not want to share with other colleges in your region. On the other hand, if an academic across town is known to be a critical but helpful voice, then by all means invite that professor to be an external reviewer.

Conducting an APR is an excellent way to introduce quality control to your department and to gain a fresh perspective through the eyes of a qualified outsider.
Select someone whose skills "fit." It’s a very good idea to choose someone who has particular expertise in a key aspect of your program. For example, if your department offers multiple, large sections of an introductory service course (in our field that would be introductory psychology), then consider hosting a reviewer who has deep experience as a teacher or as a manager of that enrollment magnet. Or, if you and your colleagues want to introduce a capstone experience for your majors, then bring in a reviewer who designed such a course.

Similarly, if your institution is considering a reorganization or changing its assessment methods, experts who have had success in those areas may be more broadly helpful than someone who hasn’t. The latter can only evaluate your program but can’t talk about what’s going on elsewhere in those realms.

Besides sharing your program’s self-study with your reviewer in advance, be sure to share what you hope to learn from the APR process. Be direct: If you want advice on a specific matter — on, say, writing-intensive courses — then be sure to raise that issue before the visit or shortly after the reviewer arrives on the campus. Specific questions will help the external reviewer shape the process and the final report.

Avoid the untrained rookie. In general, first-time reviewers tend to be overreliant on what they’ve experienced at their own institution. They may be tempted to make recommendations that would, in effect, recreate their home campus. If you’re thinking about inviting newbie reviewers, ask them questions that will reveal the degree to which they are attuned to national trends in curriculum and assessment. Try to establish the scope of the experience from which their observations will be drawn.

Consider institutional size as a qualifier. It’s not a hard-and-fast rule but it may make the most sense to recruit a reviewer who operates in a similar setting to your own. For example, a reviewer whose experience has been strictly in a nonunion environment might be less helpful if union dynamics play a key role in campus culture. If your environments are comparable, that usually means the reviewer will more readily understand the specific complexities of your operation.

Ask peers for reviewer recommendations. APRs are routine at many colleges and universities. Contact your peers at institutions similar to your own or rely on your own network to learn the names and affiliations of academics who are known as excellent external reviewers in those contexts.

Consult professional organizations. If your own contacts don’t yield a serviceable list of names, remember that some academic disciplines maintain databases of colleagues who are ready, willing, and able to serve as external reviewers. Typically, reviewers can be matched to your program’s particular goals, such as starting an internship program or migrating face-to-face course offerings to an online platform.

Match calendars. Once you have identified your evaluator, make sure you convey how soon after the campus visit you expect to receive the final APR report. Sooner is always better, as it is in everyone’s interest to finish the review process expediently.

In our experience, two weeks is the outer limit for receiving a review — any time beyond that is problematic unless there are special circumstances involved. We know of one departmental review that involved three external reviewers, at least one of whom did not submit his portion of the document even after three months. That’s not only bad form and disruptive to the review process, it’s unprofessional.

Conducting an APR is an excellent way to introduce quality control to your department and to gain a fresh perspective through the eyes of a qualified outsider. If the external reviewers have done their job well, the review document will serve as your department’s marching orders until the time rolls around for the next one.

Dana S. Dunn is a professor and former chair of psychology at Moravian College. Jane S. Halonen is professor of psychology and former dean of arts and sciences at the University of West Florida.