Multiple Choice Questions on Exams

[This is a guest post by Derek Bruff, an assistant director of the Center for Teaching at Vanderbilt University, where he is also a senior lecturer in Mathematics. His book, Teaching with Classroom Response Systems: Creating Active Learning Environments, about the technology known as clickers, was published in February, 2009 by Jossey-Bass. On Twitter, he is @derekbruff. -- JBJ]

A couple of years ago, I was teaching a statistics course for the second time. The first time I had taught the course, I only had 36 students. This time I had 56. Thinking ahead to how long it would take me to grade their exams, which had consisted entirely of free-response questions the year before, I started wondering if I should include some multiple-choice questions on the exams this time around. I felt a little ashamed. After all, instructors only gave multiple-choice exams in those really big classes where free-response exams weren’t possible, right? It’s not like they want to give multiple-choice exams—they’re forced to by the size of their classes.

Then it hit me—putting multiple-choice questions on my exams was exactly what I wanted to do.

Let me rewind a little. A few years prior to teaching this statistics course, I had started teaching with classroom response systems, often called “clickers.” When using a classroom response system, I can pose a multiple-choice question to my students during class and expect each and every one of my students to respond to the question—independently, in fact—by submitting their answers using handheld radio-frequency devices (“clickers”). A receiver attached to my computer collects their response, and the clicker software displays a bar chart showing the distribution of results.

Over the years, I had gotten pretty good at writing multiple-choice clicker questions. I find that the multiple-choice format is particularly useful for engaging and assessing students around the more conceptual material in my math courses. For instance, here’s a question about confidence intervals I’ve used:

Suppose you construct a 95% confidence interval from a random sample of size n=20 with sample mean 100 taken from a population with unknown mean μ and known standard deviation σ= 10, and the interval is fairly wide. Which of the following conditions would NOT lead to a narrower confidence interval?

A. If you decreased your confidence level

B. If you increased your sample size

C. If the sample mean was smaller [Correct]

D. If the population standard deviation was smaller

What makes this question useful is that it asks students to respond using their intuition about how confidence intervals work. Although there are a few numbers in the question to make the situation somewhat more concrete for the students, this is not a computational question. Instead, the question asks students about the relationship between the width of a confidence interval and several other variables.

I feel confident that my students, mostly engineering majors, could answer a computational question on this topic. I’m typically less confident that they have internalized the associated concepts. Asking this clicker question allows me to assess their conceptual understanding and, if the results show that students don’t understand the situation as well as I would like, I can help them think through the topic right then and there during class.

Now, back to the statistics exam. My students and I spent at least a third of every class period working through conceptually-oriented, multiple-choice clicker questions. If I thought this was such a good use of class time, then why didn’t I include similar questions on my exams? More to the point, if I wanted my students to develop conceptual understanding of the statistics in my course, why wasn’t I assessing that? Particularly since I knew exactly how to do so, by asking the same kinds of multiple-choice questions I had gotten good at writing over the years teaching with clickers!

Starting that semester, I started writing my exams so that multiple-choice, conceptual questions contribute about 40 percent of my students’ grades on those exams. Free-response, computational questions contribute the rest of their scores. Often the multiple-choice exam questions are refined or enhanced versions of clicker questions asked earlier in the semester. Writing these questions doesn’t take much longer than writing free-response questions, and, of course, they help me look forward to grading a little bit more!

Have you found that the multiple-choice format is sometimes exactly what you need to assess particular aspects of your students’ learning? How have you used multiple-choice questions to assess more than mere factual recall? Share your experiences in the comments.

Image by Flickr user Mars Hill Church Seattle / Creative Commons Licensed