5 Tips for Using Multiple-Choice Tests to Bolster Learning

Professors usually give tests to measure students’ learning. But the act of taking a test can also cause learning, helping students retain and understand material.

A new article in the Journal of Applied Research in Memory and Cognition reviews existing research and has some good news for professors: The practices that research shows to be effective for assessment also help lead to learning.

The article — by Andrew C. Butler, an associate professor of education and of psychology and brain sciences at Washington University in St. Louis — offers five best practices that further both goals. Let’s take a look:

Stick to Simple Questions …

Many instructors turn to complex multiple-choice questions in order to test higher-order thinking skills. Such components of exams, Butler writes, include a statement or question, a set of responses like a, b, c, d, and e, and secondary choices like “a and b.”

“The general idea is that the greater complexity in format and/or answering procedure enables the assessment of greater complexity of thinking,” Butler writes. But evidence suggests that is not, in fact, the case.

Complex questions turn out to be a poor choice for assessment purposes because they encourage students to engage in strategic guessing. And students’ guessing ability says nothing about their understanding of content.

When it comes to learning, the most-studied type of complex question is probably “answer until correct,” a procedure in which students can keep selecting answers until they land on the right one, and get partial credit for doing so. While such questions have the potential to enhance learning by letting students reason through a problem, students could still resort to guessing. Research suggests such questions offer no more benefit than asking traditional questions and giving immediate feedback.

One possible exception to the keep-it-simple rule: confidence-weighted testing, in which students provide a confidence rating for each potential answer while also selecting which one they think is correct. That approach has potential, Butler writes, and should be studied further.

… And That Means Avoid “None of the Above” and “All of the Above”

Including those phrases as answer choices turns simple questions into complex ones, creating the assessment challenges that come with them.

Such options can also hinder learning in some cases. It’s possible to correctly identify “none of the above” as the correct choice without knowing the right answer to the question. That allows students to circumvent the effort to retrieve the correct answer, and such retrieval is what makes tests a learning experience.

There is some evidence that including “all of the above” can be positive for learning when it is the right answer, since students are exposed only to correct information. On balance, though, Butler writes, both answer choices should be avoided because “they are clearly detrimental to assessment, and any possible benefits to learning are relatively small.”

Target Specific Cognitive Processes

The best test questions require students to engage in specific cognitive processes designed to fit an instructor’s learning objectives. That approach reduces the range of possible ways that students might land on the correct answer, which makes the test a better measure of what students understand.

A test question might target a cognitive process like retrieving a fact, Butler writes, as in “Which of the following buildings is the tallest in the world?” Or it might ask students to do something a bit more sophisticated, like “analyze a set of conditions to make a decision,” in a question like “Given that the patient shows symptoms X, Y, and Z, which of the following diagnoses is most likely?”

Designing test questions to require certain kinds of mental processes is also a way to engage students in higher-order thinking, which produces learning.

Give 3 Plausible Options

Research on assessment suggests that giving students three choices (including the correct one) is optimal. That’s enough to provide a good measure of what students know and still give them sufficient time to answer an adequate number of questions during a test. The exception: when there are not two plausible incorrect answers to offer. In that case, one reasonable wrong choice is better than adding a throwaway choice.

Offering students relatively few choices is also best for learning, since every incorrect option exposes students to false information.

Make the Test Hard — but Not Too Hard

A test question measures learning by revealing what students understand. And a question’s ability to do that is related to its difficulty. It turns out that there’s an ideal difficulty level to aim for: “a bit higher than the midpoint between chance and perfect performance,” Butler writes.

A moderately difficult test appears to be best for learning, too. That’s because learning is helped when students successfully retrieve information, and it’s harmed when they’re unable to do so. There’s a learning cost, in other words, for getting questions wrong.

At the same time, a test shouldn’t be too easy. When it is, students can score well without having a deep understanding of the material.

And not all ways of making a test harder benefit learning, Butler adds. Trick questions, for instance, don’t do that. The key is designing questions that require students to engage in the kinds of thinking that match the instructor’s educational goals.

Beckie Supiano writes about teaching, learning, and the human interactions that shape them. Follow her on Twitter @becksup, or drop her a line at beckie.supiano@chronicle.com.