“Academic program review” is accountability done by academics for academics. It’s standard practice to bring in outside reviewers to critique how well departments are functioning, but that doesn’t mean it’s well understood.
Between the two of us we’ve done about 90 APRs, and we get asked a lot of questions about this type of faculty consulting work — hence this series. So far, we’ve covered the basics of how to get started doing program reviews, and what to expect when you visit the department. Now in this third and final essay, we turn to the real work: condensing hours of notes from your campus visit into a report that offers a department real insights and recommendations.
Most reviews go smoothly. When an APR goes off the rails, it may be because of a fractured department, a clueless chair, a hostile administrator, or other problems. Our advice here will first focus on writing a high-impact review and then on what to do when things go awry.
Writing a Persuasive Report
The first rule of APR writing is simple. Get it done and submitted to the host institution as soon as possible. Many institutions will provide a desired deadline that is usually three to four weeks after your visit. It’s in your interest to write while the experience is still fresh in your mind, which is why we typically start drafting a review during the campus visit.
Some general tips on the writing and logistics:
- Focus the report on your observations and recommendations. Don’t just rehash the findings of the department’s self-study.
- If the institution uses a template form for APRs, then follow it to the letter. If not, a SWOT analysis — identifying strengths, weaknesses, opportunities, threats — can be a useful way to organize your report.
- Write as much as you think is needed. But remember: Pithier reports are more likely to be read. Adding a brief executive overview is a nice touch, and you can be sure it will be read.
- Touch on the key concerns expressed by the various stakeholders you interviewed.
- Resist the urge to tout your own department’s practices (“What we do to address this very same issue is …”) as a good solution for the host program.
- Make sure you send a tentative draft to your primary contact for fact checking before you release the final report. Inaccuracies will weaken enthusiasm for your recommendations.
- If you are part of a team of reviewers, divide the writing responsibilities. Create a Google Doc to get things rolling. It helps to identify a chief writer who can assume final responsibility for polishing the report. If a team member’s contribution is tardy, the chief writer can apply pressure directly — which may or may not bring the dawdler around. One of us worked on a two-person APR with a “co-writer” who never submitted a word or answered multiple entreaties to do so.
Don’t shy away from uncomfortable realities in your report. If the department has an outdated curriculum or far too few faculty members, say so. But in spotlighting problems, you are obligated to suggest solutions. You might, for example, recommend specific curricular updates or provide links to model curricula at similar institutions.
Any staffing recommendations you make should be mindful and modest. It’s not particularly helpful to recommend a hiring binge at a budget-strapped college. Yet you also need to be aware that administrators are very likely to resist even the most minimal proposal you make to create new tenure-track positions. They will claim financial stress due to enrollment woes (that old friend, the Demographic Cliff), pandemic fallout, or internal issues (e.g., a hiring freeze, a small endowment). You cannot change such facts, but you can be sympathetic to them while still documenting a need. If a department needs (but can’t afford) three new faculty hires, suggest a rationale for which of the positions to pursue first for the biggest impact.
Your role is to help a department see when it’s time to do things differently. In a psychology department, for example, rather than simply replacing a retiring cognitive psychologist, you might notice that the program would be better served by hiring a general-education specialist to teach in-demand introductory courses. Make the case in your report.
What if you witness bad behavior in the department, or hear about it from faculty members? Don’t put it in writing. Doing so will only direct attention away from the rest of your report (especially its important recommendations) and the bad actor will claim it was all a misunderstanding. Instead, raise the issue in private conversation with the chair or a senior administrator. Bear in mind: As a guest, you may be shocked by this professor’s conduct (we’ve both been treated shabbily or even insulted a time or two), but it may be an unfortunate but daily routine for others in the department.
You are a helpful observer, not a miracle worker or an advocate. As an external observer, it’s not your job — or within your power — to make changes in the department. Even if you identify what seems like a definitive solution to a particular problem, the program’s leaders or denizens could well disagree. Your role is to spark serious discussions within the department, and your report will be most useful if you suggest more than one possible way to proceed.
Always remember: You are an outsider who is trying to help. Don’t get pulled into departmental politics or side with one faculty camp over another. Easier said than done, right?
What to Do When Complications Arise
Not all APR experiences are happy ones. If you encounter any of the following challenges — or challenging people — how should you deal with those issues in your report?
The hamstrung reviewer. Early in the process, you usually meet with the institution’s assessment overlord. This is supposed to be an orientation but it can feel more like re-education if the official tries to dictate what you should recommend. Our favorite examples: “Do not recommend any new faculty lines” and “We are looking for creative recommendations that do not involve additional resources.”
As we’ve noted, tight economic times should make you sensitive to what’s possible. Still, the APR is your opportunity to document the true status of a program. Acknowledge the economic realities, but accurately identify what it would need to rise to the next level.
The hostile senior administrator. Sometimes a dean or other leader demonstrates a negative bias toward a department, characterizing it as “dysfunctional” or “toxic” when, by other accounts, it is thriving. The leader’s view could be tainted by personal animosities toward the chairperson or an outspoken professor.
Tread carefully. Probe for details to assess how seriously that view should be reflected (if at all) in what you write in the report. If you find a functioning department, include evidence of its effective performance. Without taking sides, you could subtly highlight that the administrator’s view doesn’t jibe with the data.
The clueless chair. We have both evaluated departments in which the chairs didn’t understand how they were perceived. Some of them had robust faculty and administrative support, in which case verifying their popularity and endorsement to continue in the position was a joyous experience. More often, however, we’ve encountered the opposite: chairs who were underperforming and didn’t seem to realize the extent of discontent. Many institutions compound the problem by not giving regular performance feedback to department heads.
The APR may be the only avenue department members have for conveying dissatisfaction to the administration in a way that doesn’t single out any specific complainant. And while it will be a difficult conversation, you should try to convey the faculty’s restive mood to the chair in your exit interview. Ask the chair: How are your actions contributing to a less-than-satisfied — and thus, less-than-effective — department?
The generational divide. Sometimes departments run aground because the values and priorities across faculty generations are out of sync. For example, younger academics may see senior professors as immune to change, having lost the drive to embrace the new; older members may view the rookies as disrespectful of department traditions and tacit folkways (i.e., the “children” are trying to overthrow the “parents”). Or there may be generational tensions over research productivity and tenure standards.
A worst-case version of this problem is the senior professor who tries to use the threat of tenure support to get junior colleagues to fall in line with some decision, and then passes off the attempt as a “joke.” It’s not funny to junior faculty members; it’s anxiety-provoking.
You can’t resolve such tensions as an outside reviewer. What you can do: Talk about misplaced humor. Make recommendations — such as better faculty-mentoring programs — to help bridge the divide. And dedicate a portion of the APR to identifying strategies to empower junior scholars.
The faculty member who has stayed too long. In some APR visits, we’ve had professors tell us, “I won’t quit until I’m carried out.” Sometimes they were vital and contributing faculty members. But in other cases they were the opposite: Their scholarly efforts were nonexistent, their student evaluations abysmal, and their course enrollments low. Suspicious of change, their interactions with colleagues were contentious. Yet they wouldn’t retire because their personal identity was so tied to being a professor.
Among the solutions you might recommend in such cases are phased-retirement options or post-tenure review procedures. The goal is either to limit potential damage from folks who really should move on, or help them to contribute effectively again.
The genuinely toxic department. Hints of this may emerge in a department’s flimsy self-study. Or you may hear faculty members repeatedly blaming some entity (an uncooperative dean, a hostile provost, a neglectful president) for why things aren’t going well. Your APR interviews will be filled with stories of personal slights in department interactions (“Would you believe it if I told you that she …?”) rather than legitimate complaints about, say, how effectively students are being served.
It’s hard to sift through all of that carping and even harder to characterize in an APR report. But your job here is to provide an honest evaluation of where the troubles lie. The fact that you could clearly identify the dysfunction during a brief visit highlights that serious damage is being done. Here’s how to discuss fractious relationships in your report:
- Identify the factions and propose ways to restore peaceful relations. It’s not unheard of to suggest bringing in a professional mediator or ombudsperson to intervene.
- Give concrete examples of how the playground infighting is interfering with the quality of the student experience.
- Discuss severe challenges with senior administrators. Recommend local mediation to help the department overcome its bad practices. In worst-case situations (this has happened only once in our APR experience), you may need to tell the administration it’s time to “start over” because there is little hope this collective of people will be able to work together and deliver a quality program.
What to expect once you file your report. We’ll end with a cautionary note: Identifying the ways in which a department can improve necessarily reveals all the things that it is not doing well enough, or at all. It’s human nature to pay more attention to negative than positive feedback, and some people may intentionally distort your findings to serve their own goals. Nonetheless, your aim here is to write a balanced and honest analysis.
Normally an APR consultant’s work is finished when you submit the report. But occasionally, departments decide to treat you as an on-call volunteer/therapist. That may not be a burden if you liked the department. However, you are no longer on the clock, and you may have to pull back to protect your time.
Finally, don’t take it personally if you don’t get much feedback on your feedback. That’s typical. Don’t expect any updates on whether your recommendations made a difference in the short or long term. Sometimes the program will seek your services for its next scheduled APR (we have both been invited back). That may be the only indication that your contributions were well-regarded.
Every APR represents a form of service to the profession. But it’s also a learning opportunity — for the department and you. It can help you appreciate your own institution (warts and all) and recognize innovations elsewhere that you may be able to borrow and adapt for use at home.
