Question: As the end of the semester approaches, I start the usual worrying about course evaluations. I always wonder how students use the various categories they are offered to rate faculty performance. I’m particularly confused by own institution’s categories: unacceptable, needs improvement, meets expectations, exceeds expectations, and greatly exceeds expectations. With such descriptors, I feel like the deck is particularly stacked against instructors with good teaching reputations. If my students expect me to be a good professor — and I am — because they’ve heard as much from my previous students, then doesn’t that mean I am just meeting their expectations if I do a good job?
Likewise, if a professor has a bad reputation and somehow turns things around in a course — either by actually improving course content and student engagement or by resorting to grade-inflation practices (accepting late homework assignments without penalty, allowing multiple exam retakes) — then wouldn’t that professor “greatly exceed expectations”?
Karen Kelsky has advised countless graduate students and junior faculty members on how to navigate the perilous waters of the job search. Twice a month she answers questions from readers on all aspects of the hiring process. Here is a sampling of her recent columns.
That is a great question — and one that speaks to a larger issue, which is the problematic nature of student evaluations of teaching.
Studies show that they disadvantage women and people of color. One 2016 study, based on data sets from both the United States and France, found that “student evaluations of teaching (mostly) do not measure teaching effectiveness.” The study found that gender bias on course evaluations skewed “how students rate even putatively objective aspects of teaching, such as how promptly assignments are graded,” and caused “more effective instructors to get lower [ratings] than less effective instructors.” Perhaps most depressingly, the study said it was impossible “to adjust for the bias, because it depends on so many factors.” Various studies over the years have also shown the systemic bias against minority faculty members on these evaluations, with minority women penalized the most.
Your question has picked up on another problematic aspect of teaching evaluations: their imperfect design.
That flaw has occurred to plenty of academics and anyone else who thinks critically about research design and the fact that it does not take place in a laboratory vacuum. Student evaluations of teaching raise basic questions about their design, namely:
- What is the baseline for evaluation?
- How does it relate to pre-existing expectations of students or to the reputation of the professor?
- How does it factor into repeat experiences with the same professor, which is very common in teaching-centered institutions (where student evaluations are going to be that much more important!)? Quite often in those institutions there is an expectation that faculty members will teach all types of courses — introductory, 200-level, upper-level.
- And of course the factor your question raises: What about improvement over time?
There really isn’t an easy way to deal with those built-in weaknesses, and there isn’t a systemic way to fix them, either. Sure, it’s possible that increased advocacy by faculty governance — highlighting the empirical unreliability of these evaluations — might help reduce their relative weight in performance assessment. That is, in places where faculty governance still has some sway. But I wouldn’t hold my breath, in part, because status quo forms of assessment are hard to change substantively (and tweaks here and there will not mitigate the systemic bias).
What you can do, though — and some faculty members already do — is engage your students explicitly about teaching evaluations.
Depending on what you are teaching, you might even be able to connect the topic to something you are studying in class. That might work, for example, if you are teaching sociology, psychology, research design, or critical theory — subjects in which issues of biases in production of “objective” knowledge are part and parcel of the curriculum.
But even if your courses don’t engage with those topics per se, you can spend some time before you hand out the course evaluations discussing their function, and going over some of the points that you raised in your question to me.
By and large, students don’t think in great depth about teaching or performance assessment as they fill out an evaluation. Mostly they see it as a perfunctory task at the end of a semester, or as a way to stick it to an instructor they didn’t like.
While there are no guarantees, it can’t hurt to talk with students directly about what the evaluation is supposed to measure. A good discussion might make them more discerning and thoughtful respondents. At the very least, you can unpack a vague category like “meets expectations.” Challenge students to clarify for themselves: Are they thinking of their expectations for the course? For the subject? For the instructor? Are they assessing the expectations you outlined on the syllabus and on the first day of class? Are they basing their evaluation on what they were told to expect about you and your course by other students or by Ratemyprofessors.com? Or are they comparing your performance to their general expectations of what a professor on your campus is like?
You still might not be able to correct for all the structural issues that arise when students are asked to evaluate their teacher but, at the very least, you can help them engage with the evaluation more mindfully.
Karen Kelsky is founder and president of The Professor Is In , which offers advice and consulting services on the academic job search and on all aspects of the academic and postacademic career. She is a former tenured professor at two universities. Browse an archive of her previous advice columns here .
