A University Overhauled Its Course Evaluation to Get Better Feedback. Here’s What Changed.

SamuelBrownNG

Until recently, the University of Southern California took a conventional approach to student course evaluations. At the end of the semester, students answered about a dozen questions, including ones asking them to rate the instructor and the course. Their feedback was used as the primary — or in some cases, sole — evidence of professors’ teaching effectiveness in formal performance reviews, including those to determine tenure and promotion.

For years, professors at USC had expressed concerns about how their course evaluations were designed and used, echoing similar worries across the country. Research has found that course evaluations are a poor measure of learning, prone to bias, and often interpreted in ways that make little statistical sense.

The research — particularly the evidence on gender bias — persuaded the university’s provost, Michael W. Quick, to end the use of course evaluations as a direct measure of teaching effectiveness this past spring. Students are still providing feedback, but now they’re using a new tool that asks them to weigh in on the learning experience more than on the instructor. Their feedback will be used differently, too. It will no longer serve as the main mechanism for evaluating teaching. Instead, it will help individual instructors improve, and help their schools observe larger patterns.

So how will the university assess teaching now? USC is taking a multifaceted approach, part of a broader effort to elevate teaching on the campus. Teaching, like research, will be evaluated by professors’ peers. Each of the university’s schools will be able to customize its system, using some combination of peer-evaluation tools like class observations and course-design reviews.

The university is also recommending that faculty members complete a reflection statement on their teaching each year. In those statements, instructors explain how they have used data to improve their teaching, and student feedback will continue to be a source of data.

The idea, says Ginger Clark, assistant vice provost for academic and faculty affairs and director of the Center for Excellence in Teaching, is to examine teaching from multiple vantage points to evaluate it in a more comprehensive way.

While university leaders are convinced that students are not the best arbiters of teaching quality, they still think student input can help professors improve — if it is asked for more thoughtfully. So now students complete online evaluations asking a new set of questions that were rolled out in the spring. Let’s take a closer look at what USC asked students before and after the change.

Here’s the old document:

And here’s the new one:

Further Reading

The problems with course evaluations:

Research has found that students judge professors who are women more harshly than those who are men.

If course evaluations are steeped in bias, and they’re used in personnel decisions, then colleges could face litigation.

The value of student feedback:

Students can be trained to give better feedback, as in this effort at the University of California at Merced.

Kevin Gannon offers some advice for interpreting course evaluations.

Models for evaluating teaching:

Use a better student course evaluation, says the IDEA Center.

Use an inventory of teaching practices, argues Carl Wieman, an advocate for science teaching.

Incorporate peer evaluations, suggests Philip B. Stark.

Beckie Supiano writes about teaching, learning, and the human interactions that shape them. Follow her on Twitter @becksup, or drop her a line at beckie.supiano@chronicle.com.