Did You Learn Anything?

As the seasons come and go, so too does faculty dissatisfaction with the biannual ritual of having students evaluate our teaching. This past winter break brought with it -- on academic blogs, in The Chronicle’s forums, over photocopiers and coffee makers -- the usual round of rants and complaints about the unfairness of having student opinions factor into decisions about faculty hiring, tenure, and promotion.

I saw a lot of the usual assertions -- about the alleged link between easy grading and good evaluations, about perceived student biases toward or against various teaching strategies or categories of faculty members. Some of the evidence for those assertions was anecdotal; occasionally debaters pointed to a study or publication that supported their point.

In very few of those conversations did anyone note that the vast majority of studies on student ratings of teaching have concluded that students are very accurate judges of the most important thing we can ask them about their experiences in a course: Did they learn anything? Students do answer that question accurately when their internal estimations are gauged against external measures of their learning (such as exams).

Like many faculty members, though, I sometimes find myself frustrated with student ratings of teaching, for three reasons:

• Many ratings forms ask questions that are irrelevant to identifying effective teaching, that push particular pedagogical agendas, or that may help create bias. A form that asks whether the instructor used small groups in class, for example, implies that good teachers use small groups, which is not necessarily true.

• Many administrators rely too heavily on the ratings. They take the lazy way out and let the forms do the work for them, rather than letting the forms serve as part of a package of methods for evaluating a teacher, including classroom observations and analyses of written instructional materials.

• The ratings forms come at the end of the semester, when it’s too late to use the results to improve the course. That means you get summative judgments rather than constructive criticism.

Until my plans for world domination come to fruition (soon, all too soon), I can’t revise the evaluation forms used by your institutions or compel your administrators to take a more comprehensive look at your teaching.

I can, however, help you with the third item on my list, and offer some suggestions for how you might talk to your students right now -- at the midpoint in the semester -- in an effort to improve the rest of the course for both you and them.

End-of-semester student evaluations are supposed to help you become a better teacher, but they are also supposed to give your superiors a basis to make professional judgments about you. Midterm student surveys, in contrast, are completely voluntary. You can ask whatever questions you want and nobody but you will see the results.

They also don’t require a ton of work on your part or much class time. Even five minutes of student feedback can prove useful in figuring out what you are doing well and not so well in a given course.

I recommend two possible methods for getting a midterm assessment from your students, the first of which comes from a book I have recommended before: Classroom Assessment Techniques: A Handbook for College Teachers, by Thomas A. Angelo and K. Patricia Cross (Jossey-Bass, 1993), offers a host of methods you can use to gather information from students. Their most well-known method is the “Minute Paper,” and it really does take just a minute or two to administer -- five at the outside.

“To use the Minute Paper,” Angelo and Cross write, “an instructor stops class two or three minutes early and asks students to respond briefly to some variation on the following two questions: ‘What was the most important thing you learned during this class?’ and ‘What important question remains unanswered?’ Students then write their responses on index cards or half-sheets of scrap paper . . . and hand them in.”

That technique, as the authors explain, provides “manageable amounts of timely and useful feedback for a minimal investment of time and energy.” If every student mentions some trivial but entertaining point from your lecture as the most important thing they learned, you know you need to revisit the main idea next time. If a significant number of students list as an unanswered question one that you covered during a lecture, it’s time to review.

Angelo and Cross describe the Minute Paper as an instrument designed to give feedback on a single course session, but you can use it to gauge student opinion on an entire unit or the course as a whole. In any case, the student responses take just a few minutes to read and will help you see whether the ideas, concepts, and skills you are teaching correspond with what the students are learning.

The second method I would suggest to gather feedback from students is to take 10 or 15 minutes at the end of a class and administer a survey with some version of these two questions: “What classroom activities or assignments have been most effective in helping you learn this semester, and why?” and “What classroom activities or assignments have been least effective in helping you learn this semester, and why?” Ask students to respond anonymously and write a paragraph for each question.

Make sure you phrase the questions so that students understand you are not asking them to make summative judgments about you or the course. You are asking them to comment on what has worked for them in the classroom. You want their responses to be a description of their own learning experiences rather than judgments about you. Reiterate that when you hand out the surveys.

So don’t ask, “What teaching methods have been the most effective this semester?” All you will get is a bunch of responses that say: “They’ve all been great! It’s a great class! Prof. Lang rocks! Let’s go out for coffee!” Students will usually jump on an easy opportunity to suck up, even anonymously, while you still have power over their grades.

Explain carefully, both in writing and in class, that you are interested in seeing what you can do that will help everyone learn as much as possible in the final weeks of the semester, and that the exercise will work only if they give honest responses.

When you get their responses, pay attention to similar comments that come from multiple students. At least one student will always hate anything you do. But if 20 students say they are not getting much out of the group work, you should probably take a closer look at how you are conducting those sessions.

As a slightly Machiavellian aside, you can use the opportunity of administering midterm surveys to give a little speech telling students that you care about their learning, that you want the course to be the best possible experience for them, and that you value their views on how the course is going.

That pep talk, combined with an actual effort to get their ideas, will give the students a more favorable impression of you for the remainder of the semester than they might have had otherwise and should contribute to better ratings in the formal evaluations -- even if you don’t actually make any changes to the course.

I do recommend that you discuss the results of your midterm survey with students, and find at least one or two ways to modify your course in order to show them that you actually listened to what they had to say.

All of which may raise the hackles of faculty members who see such tactics as pandering or as giving in to the customer-service model of education that has been debated in The Chronicle’s pages recently. But that’s not the case here.

Your job as a teacher is to help your students learn. Who is more qualified than those very same students to tell you whether or not you are succeeding? The students may not have the qualifications to tell you what you should do in the classroom, but they can certainly tell you whether what you have been doing is helping them learn.

With that information at your disposal, you are well equipped to determine whether you need to make changes to your course. Without that information, you’re like an expedition leader who has a clear map to her destination, but who never looks behind to see whether anyone’s following.