Once you accept the role of academic program reviewer — hired by another department to evaluate its inner workings — you face three stages of work: the prep work, the campus visit, and the write-up. We focused on the first stage in Part 1 of this series on program review. Now we will discuss how to make the best use of your time on the ground.
As longtime program reviewers, we’ve been asked a lot of questions about how to do APRs well and efficiently. These reviews are a well-established accountability practice and a valuable way to give back to your discipline, not to mention being one of the few forms of paid faculty service.
You may do the campus visit in person or, if the review is exclusively remote, via Zoom or some other platform. Either way, it’s the most vital stage of your information gathering. Here’s what to expect.
How long will the visit take? Most APR visits take no more than two days, especially if you do enough advance legwork (such as reading the department’s self-study and scrutinizing various campus websites). Some visits may require a bit more of your time if a program has asked for special recommendations — for example, if the department is seeking advice on updating labs or planning a new track in the major.
Expect virtually every minute of your time to be scheduled. You will feel as though you have run a marathon when it’s over, if you do it right.
If the campus is remote from an airport, program officials may ask you to rent a car. However, most departments will send a faculty member to pick you up. Your data collection should begin with this meetup. The informality of taking you to your hotel often opens the way to important disclosures. Ask how your driver got the assignment. Drafted or volunteered? In our experience volunteers may be eager for a private audience to share their perspective outside the earshot of their colleagues. Both of us have endured our fair share of woeful meetup monologues — about “the way things used to be” and “how bad they are now” — en route to the hotel. Draftees are more likely to be circumspect about bringing up department secrets, especially if they do not have tenure.
Make a lot of notes. Information is going to come at you like water from a firehose. It is a good idea to inform all participants that you will be taking copious notes as you go. We advise against tape recording the process because that may make people nervous and suppress candid conversations. Instead, listen carefully, take detailed notes, and follow up on any unclear points.
Size up their commitment to critique. Programs differ in their enthusiasm for the APR process. Some have embraced the idea of steady improvement and are eager for feedback. Others, however, see APRs as a relatively empty ritual — one that will be forgotten once you are dropped back at the airport.
For example, one of us was invited (and declined) to do a review in a remote locale where the department host promised excellent meals and few meetings since the “bulk of the external review is already written for you.” Why decline such invitations? You don’t want to lose your personal integrity or develop a reputation in your field as someone who accepts pointless consulting work for the money.
Throughout your visit, ask random people whether they believe the outcome of the APR is likely to have any effect on future directions for the program.
You won’t control the schedule, but you can influence it. The department typically sets the schedule, not the reviewer(s). An APR day tends to involve back-to-back meetings in 30- and 60-minute blocks, with the time allocated by status (i.e., administrators and department chairs get large blocks while solo and group meetings with faculty members get shorter blocks). Working lunches are likely to be part of the mix.
Ask for the schedule to include breaks as well as open times when you can consolidate your notes and think through the implications of what you are hearing.
APRs usually begin early in the morning with an orientation, either with the department chair or with a campus administrator in assessment/accountability. Clarify any questions you have about the program’s expectations, timelines, or preferred report format. Ask about any important, recent developments that might influence what you hear. For example, visits that take place in the wake of enrollment shortfalls, unexpected budget woes, or campus lockdowns may affect how people respond to your questions.
Meet the chair. A private, candid meeting with the department chair should happen both at the beginning and end of the review process. To get oriented, ask the chair some version of, “What would be the best outcome of the APR from your vantage point?” How the chair answers can provide a good indication of whether a candid review will be welcomed or ignored. In your final meeting, talk about whether or not the chair’s preferred outcome is feasible.
Sometimes faculty critics will raise questions about the chair’s leadership during your visit. For example, a chair may be seeking a second term and your conversations with professors reveal clear opposition to such a plan. It will be uncomfortable, but a frank discussion with the chair about this lack of support can facilitate some face-saving decisions — particularly if the focus of the conversation can be shifted to succession planning and, perhaps, a sabbatical leave.
Meet the faculty. How you do so will depend on the size of the department. If it’s a small program, expect a lot of one-on-one, 15-minute meetings. If it’s a large department, expect hour-long meetings with groups of faculty members. Sometimes the department organizes those groups around shared faculty research interests and responsibilities. Or it might group people by rank (assistant, associate, full, nontenure track) to give you a better understanding of the department’s history as well as allow for an airing of potential intergenerational conflicts (if, say, senior professors are irritated by what they perceive as disrespect for department traditions and practices from new hires, who, for their part, may view older professors as resistant to new ideas).
On your agenda for these sessions:
- Start by asking if participants know the purpose of the APR. Do they have any questions or concerns about the process?
- New, untenured faculty members may be understandably anxious about the risks of sharing their views on the department. Reassure people that their comments will remain confidential. Make clear that you are looking for emerging trends or, on sensitive or controversial issues, for where the “weight of evidence” lies. Emphasize that comments will not be linked to individual names or even to particular groups.
- Among the standard questions to ask faculty members: Is the pathway to tenure and promotion clearly defined and achievable? Do you think the department is well regarded by the administration? What aspects of delivering the major are most frustrating? How attentive is the program to issues of diversity and inclusion?
- You’re not just documenting problems; you’re seeking solutions, too. Ask people to identify what might improve the quality of the academic experience for themselves and their students. To encourage creative responses, you could say something like, “If you had a magic wand with the power to change things, how would you use it to improve learning and working conditions?” The responses will help you zero in on what’s missing in the department culture.
- A group meeting may inhibit some people from commenting at all. Invite participants to contact you by email to share anything else that will help you to be fully informed as you write the review. Give them a deadline; you don’t want emails rolling in after you’ve already submitted the report. Also, some people may ask to meet you after hours (say, in the hotel bar). That’s not a good idea — tell them email will do.
Don’t forget to meet with adjunct faculty members, whose efforts are never recognized enough. In our experience, most part-time instructors appreciate being included in the APR process.
Also on the schedule should be meetings with small groups of undergraduate majors, graduate students, librarians, and professors from other departments. For example, librarians can comment on the rigor of the program’s research assignments. Faculty colleagues can comment on the department’s reputation on the campus for quality and collegiality.
Meet the senior administrators. Toward the end of the visit, you are likely to meet with a designated academic overlord (e.g., dean, provost, or president). Such meetings typically signify that the APR report will be taken seriously. Overlords are looking to improve efficiency and quality, so they appreciate creative suggestions — especially inexpensive ones.
Take the lead during these meetings and share some highlights that you anticipate including in your final report. Just keep in mind: The leaders you are interviewing may have an agenda.
Occasionally, these meetings can uncover factors you need to be aware of in writing your report (“we’re facing major budget cuts, so please don’t recommend creating four new faculty lines”). In addition, a private audience with a senior administrator is the time to raise serious concerns — such as ethically questionable behavior by a faculty member or chair — that need attention but might be problematic to put in writing.
You’ll have one or more exit interviews. Your final meeting with the department chair can serve as an official exit interview. However, it is often useful to conduct an exit interview with the program’s faculty as a whole to promote buy-in. Start the discussion by mentioning the program’s positive features and accomplishments before turning to any lingering concerns that may require greater delicacy.
Tour the campus. At some point, you should visit one of the department’s typical classrooms (check out the technology therein). Also tour faculty offices, as well as the department’s main office, lounge areas, relevant research labs, and any similar specialized spaces. Make sure to include in your report any deferred maintenance you spot, particularly if it impedes student learning, program efficiency, or safety.
A visit to the campus library is a good idea, as is a general tour of the campus. Administrators generally like to show off their newest shiny toy (e.g., an arboretum, a new academic building, a state-of-the-art student union), even when it has little relevance to the program being scrutinized.
What to get out of lunch and dinner meetings. Informal conversations over a meal can provide helpful context about the program climate. Do colleagues seem to enjoy one another’s company? Is the camaraderie (or the tension) palpable?
Some dinner companions let down their guard and provide valuable insights. You’re not looking for gossip for gossip’s sake here — the point is to understand the department’s internal dynamic and how that might be bolstering or harming its mission.
How to handle the “prickly pears.” Every department has a few difficult personalities (and, with luck, only a few). These faculty members (or, in a worst-case scenario, program leaders) harbor grievances — sometimes with very good reason — about their career trajectory, changes in the curriculum, the “quality of the students that admissions brings us now,” or some other idiosyncratic complaint tied to the institution and their place in it.
It’s not your responsibility as a reviewer to placate such folks, but hearing them out is essential. Try to be evenhanded in the time you spend with various faculty members so as to avoid contributing to the friction between opposing camps. Avoid compromising requests that may not be part of the formal schedule, such as selectively sharing draft portions of the APR report via email before the official submission.
How do you get paid and reimbursed for your expenses? Some host departments will expect you to turn over receipts before you leave town. Comply only if you anticipate no further bills. In traveling home, you may have some additional expenses for meals, parking fees, and tips that should not be omitted from your final tally. Some colleges send your expense reimbursements along with your honorarium, while others send the former well before the latter. Host institutions may also combine expenses and honorarium in one check, which will require an explanation to the IRS since only the honorarium portion is taxable.
Most institutions set a deadline for the report’s submission — your honorarium should arrive shortly afterward. However, we’ve also experienced host programs that “forget.” If you have not received payment after a month or so, send a polite inquiry as to “whether anything else may be needed for the processing of my payment.”
The amount varies. For two days of your time, you should expect to receive $1,000 to $1,500, plus reimbursement for expenses. The honorarium for virtual APRs may pay less (around $500), since you are spared the inconvenience of travel.
Usually you can negotiate your consulting fee. Reviewers can propose higher fees for programs facing severe challenges (e.g., the faculty members have stopped speaking with one another). We have also been successful in upping the honorarium by promising speedy turnaround and in-depth suggestions to help the program move forward on its strategic plan.
Next up is our final entry of this series, a look at the third stage of the APR process: how to construct a high-impact report and how to handle difficult situations that can tax even the most experienced program reviewers.
