> Skip to content
FEATURED:
  • The Evolution of Race in Admissions
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
ADVERTISEMENT
Advice
  • Twitter
  • LinkedIn
  • Show more sharing options
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
  • Copy Link URLCopied!
  • Print

The Semester’s Ending. Time to Worry About Our Flawed Course Evaluations.

By  Karen Kelsky
April 25, 2018
The Professor Is In - Poppins

Question: As the end of the semester approaches, I start the usual worrying about course evaluations. I always wonder how students use the various categories they are offered to rate faculty performance. I’m particularly confused by own institution’s categories: unacceptable, needs improvement, meets expectations, exceeds expectations, and greatly exceeds expectations. With such descriptors, I feel like the deck is particularly stacked against instructors with good teaching reputations. If my students expect me to be a good professor — and I am — because they’ve heard as much from my previous students, then doesn’t that mean I am just meeting their expectations if I do a good job?

We’re sorry. Something went wrong.

We are unable to fully display the content of this page.

The most likely cause of this is a content blocker on your computer or network. Please make sure your computer, VPN, or network allows javascript and allows content to be delivered from c950.chronicle.com and chronicle.blueconic.net.

Once javascript and access to those URLs are allowed, please refresh this page. You may then be asked to log in, create an account if you don't already have one, or subscribe.

If you continue to experience issues, contact us at 202-466-1032 or help@chronicle.com

Question: As the end of the semester approaches, I start the usual worrying about course evaluations. I always wonder how students use the various categories they are offered to rate faculty performance. I’m particularly confused by own institution’s categories: unacceptable, needs improvement, meets expectations, exceeds expectations, and greatly exceeds expectations. With such descriptors, I feel like the deck is particularly stacked against instructors with good teaching reputations. If my students expect me to be a good professor — and I am — because they’ve heard as much from my previous students, then doesn’t that mean I am just meeting their expectations if I do a good job?

Likewise, if a professor has a bad reputation and somehow turns things around in a course — either by actually improving course content and student engagement or by resorting to grade-inflation practices (accepting late homework assignments without penalty, allowing multiple exam retakes) — then wouldn’t that professor “greatly exceed expectations”?

The Professor Is In - Poppins
The Professor Is In
Karen Kelsky has advised countless graduate students and junior faculty members on how to navigate the perilous waters of the job search. Twice a month she answers questions from readers on all aspects of the hiring process. Here is a sampling of her recent columns.
  • The Professor Is In: Stranded on the Academic Job Market This Year?
  • The Professor Is In: What if Your Teaching Experience Is Not ‘on Brand’ for the Hiring College?
  • The Professor Is In: ‘Should I Mention I’m From the Area?’
  • The Professor Is In: It’s Hard to Be Ill in Academe

That is a great question — and one that speaks to a larger issue, which is the problematic nature of student evaluations of teaching.

Studies show that they disadvantage women and people of color. One 2016 study, based on data sets from both the United States and France, found that “student evaluations of teaching (mostly) do not measure teaching effectiveness.” The study found that gender bias on course evaluations skewed “how students rate even putatively objective aspects of teaching, such as how promptly assignments are graded,” and caused “more effective instructors to get lower [ratings] than less effective instructors.” Perhaps most depressingly, the study said it was impossible “to adjust for the bias, because it depends on so many factors.” Various studies over the years have also shown the systemic bias against minority faculty members on these evaluations, with minority women penalized the most.

Your question has picked up on another problematic aspect of teaching evaluations: their imperfect design.

ADVERTISEMENT

That flaw has occurred to plenty of academics and anyone else who thinks critically about research design and the fact that it does not take place in a laboratory vacuum. Student evaluations of teaching raise basic questions about their design, namely:

  • What is the baseline for evaluation?
  • How does it relate to pre-existing expectations of students or to the reputation of the professor?
  • How does it factor into repeat experiences with the same professor, which is very common in teaching-centered institutions (where student evaluations are going to be that much more important!)? Quite often in those institutions there is an expectation that faculty members will teach all types of courses — introductory, 200-level, upper-level.
  • And of course the factor your question raises: What about improvement over time?

There really isn’t an easy way to deal with those built-in weaknesses, and there isn’t a systemic way to fix them, either. Sure, it’s possible that increased advocacy by faculty governance — highlighting the empirical unreliability of these evaluations — might help reduce their relative weight in performance assessment. That is, in places where faculty governance still has some sway. But I wouldn’t hold my breath, in part, because status quo forms of assessment are hard to change substantively (and tweaks here and there will not mitigate the systemic bias).

What you can do, though — and some faculty members already do — is engage your students explicitly about teaching evaluations.

Depending on what you are teaching, you might even be able to connect the topic to something you are studying in class. That might work, for example, if you are teaching sociology, psychology, research design, or critical theory — subjects in which issues of biases in production of “objective” knowledge are part and parcel of the curriculum.

But even if your courses don’t engage with those topics per se, you can spend some time before you hand out the course evaluations discussing their function, and going over some of the points that you raised in your question to me.

ADVERTISEMENT

By and large, students don’t think in great depth about teaching or performance assessment as they fill out an evaluation. Mostly they see it as a perfunctory task at the end of a semester, or as a way to stick it to an instructor they didn’t like.

While there are no guarantees, it can’t hurt to talk with students directly about what the evaluation is supposed to measure. A good discussion might make them more discerning and thoughtful respondents. At the very least, you can unpack a vague category like “meets expectations.” Challenge students to clarify for themselves: Are they thinking of their expectations for the course? For the subject? For the instructor? Are they assessing the expectations you outlined on the syllabus and on the first day of class? Are they basing their evaluation on what they were told to expect about you and your course by other students or by Ratemyprofessors.com? Or are they comparing your performance to their general expectations of what a professor on your campus is like?

You still might not be able to correct for all the structural issues that arise when students are asked to evaluate their teacher but, at the very least, you can help them engage with the evaluation more mindfully.

Karen Kelsky is founder and president of The Professor Is In , which offers advice and consulting services on the academic job search and on all aspects of the academic and postacademic career. She is a former tenured professor at two universities. Browse an archive of her previous advice columns here .

Read other items in this The Professor Is In package.
We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Karen Kelsky
Karen Kelsky, an academic-career adviser, is founder and president of The Professor Is In, which offers advice and consulting services on the academic job search. She is a former tenured professor at two universities. Browse an archive of her previous Chronicle column here.
ADVERTISEMENT
ADVERTISEMENT
  • Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
    Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
  • The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
    The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
  • Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
    Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
  • Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
    Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
1255 23rd Street, N.W. Washington, D.C. 20037
© 2023 The Chronicle of Higher Education
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin