“Academic program reviews” have become a standard best practice in higher education’s accountability world. Every five years or so, a department brings in outside professors to critique how well — or how poorly — a program is functioning. Quite a bit can be at stake in an APR. Good reviews for a department can open the institution’s wallet, while bad ones can be used to justify budget cuts or leadership changes. So it makes sense for the professors doing these reviews to be seasoned, objective, attuned to disciplinary trends, and interpersonally savvy (to deal with fractured departments).
We have specialized in serving as academic program reviewers for many years. Between us we are closing in on completing 90 APRs. Obviously, we enjoy the work, but we also find it is a valuable way to give back to our disciplines.
Program reviews are a way for departments to communicate with one another: As the reviewer, you are offering your disciplinary expertise and sharing what’s happening at your home institution with your hosts, but you’re also learning from their folkways. Beyond the compensation you earn for APRs (more on that in a bit) and the knowledge you build, there are other advantages. Sometimes you hear about innovative ideas that you can steal and bring to your home department. Even if the review is stressful — the department is in turmoil and faculty members are showing their worst sides — you can return to your own department feeling more appreciative of your own colleagues.
Over the years, we’ve heard a lot of questions from colleagues — especially those new to program review — about how best to do this type of service efficiently and professionally. We have assembled a set of tips for faculty members who wish to embark on the APR consulting life. It’s a big topic so we are planning several columns. In this first installment, we focus on the prep work you need to do before you visit a department for the actual review.
How to get started as a program reviewer. Spread the word that you’re open for APR business. Often these invitations result from your professional networks: Professors you know in the field mention that their department is looking for an external program reviewer. However, some disciplines offer a central clearinghouse with names of academics who do this type of consulting, typically highlighting any special qualifications that might offer a competitive advantage. If your discipline has such a clearinghouse, submit your areas of expertise (e.g., curriculum, hiring, undergraduate advising) and a current version of your curriculum vitae.
Prepare a short version of your CV for program review. When you are more experienced at APR, you may get direct invitations from departments to review their program. Usually, however, just like most work in academe, there will be a competitive process. The institution looking for reviewers might ask you to submit your full CV or a short version organized to highlight your competence as an external reviewer.
Build the short version around aspects of your background that show your knowledge of national best practices in higher education. If you have done APRs, make that clear on your vitae. You might, for example, tout your knowledge of undergraduate curricula in your discipline, your experience with academic-assessment issues, or how to introduce critical-thinking activities into capstone courses.
What about the money? Much of the service work that faculty members are expected to do for their department, institution, or field has no added compensation attached. APR work is an exception — although given how time-consuming it can be, it doesn’t pay especially well. So how much does it pay? That isn’t a simple question to answer.
APR tends to be regarded as a consulting service. Usually it’s only paid if you are hired from outside the institution; it’s treated as unpaid service if you are appointed to review another department on your own campus.
Pay for external reviewers can range between $500 and $2,000 (often paid by an office that oversees campus accountability). A typical two-day program review probably represents 20 to 24 hours of work on the ground — including meetings, initial drafting of sections of your report (the bulk will be written when you return home), and business meals where you continue to gather data. Depending on your persnicketiness, you could be looking at perhaps 30 hours of work. So if you are being paid $500 for a 30-hour gig, you’re making less than $20 an hour.
The real work begins once you’ve been invited to do an APR:
Clarify the scope of responsibilities. Some institutions hire just one external reviewer, some hire two, and some create teams with external and internal members. When the program-review team has both types of members, it is usually assumed that the external consultant will play the lead role and be the primary writer of the report. But some do assign the lead role to an internal reviewer to facilitate more controlled follow-up activity.
Find out up front what the expectations might be for leadership of the team. If you are part of a team, meet remotely with the other members before your campus visit to establish a positive working chemistry.
Ask for a copy of the department’s self-study before your visit. Institutions often require departments to do their own analysis of how they think they are doing and then share it with the APR team. The self-study answers such questions as: Does the program’s mission fit with the institution’s? What are the department’s enrollment and budget trends, internal pressures, assessment results, and relations with campus administrators?
As the program reviewer, you should do two things at this point:
- Request a copy of the self-study to arrive at least two weeks before your campus visit.
- Ask how the self-study came to be. A solid self-study can actually help a department unite and re-energize its faculty members around their common mission. In less-functional departments, the self-study may be assigned to one unlucky faculty member (regrettably, often an untenured assistant professor) and the campus visit may reveal that the remaining faculty members are clueless about the conclusions.
A good self-study can help the program reviewer understand quickly how the department sees itself. And if you confirm its conclusions during your campus visit, you can easily convert some of the department’s analysis to your own formal report.
Find out the preferred format for the report. Some institutions have a standard APR template that reflects their distinctive accountability needs; others don’t, so how you execute your report is wide open. Try to get a feel for length and format. One of us did a review this past spring that was limited to a few single-spaced pages, but our average is more like 10 to 15 pages. One of us wrote a substantially longer report for a department that was experiencing serious challenges.
A reasonable organization for these reports is the standard SWOT analysis — for strengths, weaknesses, opportunities, and threats. That style of organization usually captures the most important information for the department’s needs.
Scrutinize the department’s web page. Even a poorly designed website can tell you things about the program. As you click though, keep these questions in mind:
- Does the information support the facts and the feel of the program as presented in its self-study?
- How aligned are program goals with institutional mission?
- Can you navigate the website easily to find the information you need? Pretend you are a student: How student-friendly is it? Do dead links and outdated information show inattention to important communication needs?
To help you ask better questions during your campus visit, use the department’s website to get familiar with its faculty members and what their program contributions might be.
Keep in mind: Covid-19 pushed reviews into a remote format — which may remain the case for some institutions as a way to pare down reviewers’ travel costs.
Participate in designing the interview roster. Whether you visit the department in person or online, you should have access to all faculty members, either singly or in well-defined groups (by subdiscipline, by academic generation). Typically, programs also arrange for you to interview relevant overlords — deans, provosts, and even presidents, if the APR is a highly valued process on that particular campus.
Programs will often solicit student volunteers (both graduate students and undergraduates) to meet with you, too. Just be aware the volunteers tend to be high-end, satisfied performers so you may miss some important context about the departmental climate. For example, you may get a good idea of how the department trains future graduate students, but not how well program graduates do on the job market.
For a fuller picture of the department, here’s who else you should interview:
- Representatives of allied departments with insight on the program’s campus reputation.
- Librarians, who can affirm that the department’s courses and assignments are rigorous or, conversely, that the department seems academically disengaged.
- Administrative support personnel. They know where the bodies lie.
Negotiate compensation. Most academic programs won’t have a huge bankroll to compensate external APR consultants. You may feel reluctant to bring up the fee because it can be embarrassingly low, by corporate-consultant standards. You’re not looking to get rich here, but you’re also not looking to be taken advantage of, either. Just ask “What kind of compensation or honorarium did you have in mind?”
Treat the department’s proposal as a starting point. If the fee seems too low for the time required — preparation, campus visit, report-writing, follow-up questions — you can accept the consulting proposal conditionally but request that the program determine if the payment might be negotiated upward in view of the quality of work you intend to provide. Justify your request for a higher fee based on the promise of a speedy and comprehensive report. Ideally, you should submit your finished report within two weeks of the campus visit.
Understand the tax implications. You may be offered a flat fee designed to cover your work as well as travel and meal expenses. This arrangement creates a tax challenge since you should be taxed on the fee but not the expenses. Either ask for two checks — one for the taxable fee and the other for the untaxed expenses — or be prepared to justify not paying taxes on part of the payment if you get audited.
Spell out your personal and logistical needs. Consulting days can be long and draining. The department won’t know your needs unless you share them:
- Most hosts provide meals, water, and snacks. However, if you have specific nutritional accommodations or allergies, notify the APR organizers in advance.
- Find out where you will be staying. Consultants can end up in well-appointed hotel rooms or minimally furnished dorms (one of us once stayed in a seen-better-days room at the top of a decrepit Victorian manse that housed the admissions office). It helps to know in advance so you can pack appropriately and ensure Wi-Fi access.
- If travel poses special problems for you, ask to be a virtual reviewer, which can save the host institution substantial money at the expense of a more complete picture that would derive from a campus visit.
- See if the host department is buying the ticket — most prefer to do so. That means one less receipt to track when the process is over and could spare you an expense if the department cancels your visit or the APR entirely.
In our next installment, we consider strategies for gathering the most reliable and valid information during your campus visit.
