A while back, Jeremy L. Hsu had a water-cooler conversation in his department that revealed his colleagues varied in how they framed their exam questions. There were arguments in favor of each approach, but there didn’t seem to be much specific evidence to support them. So Hsu, an assistant professor of biology education at Chapman University, and some colleagues decided to test out their approaches.
A paper detailing their findings, “Investigating the Influence of Assessment Question Framing on Undergraduate Biology Student Preference and Affect,” is published in the December issue of CBE–Life Sciences Education, already available online.
Hsu supposed that students might benefit from using the examples of a diverse set of real scientists. That would let students both encounter authentic instances of what they were learning and remind them that science is done by people of different genders and races. That last point, he thought, would fit in with the broader movement to create a more inclusive environment for students who have been marginalized.
Another model — using the first names of students’ actual classmates in problems — connects to research on secondary-school classrooms showing that students respond to problems that are personalized around their interests.
The “you” framing, finally, suggests to students that they’re the ones doing the science.
To test the approaches, the professors used one of the three formats for otherwise-identical test questions in each of three sections of the same biology course. They also surveyed students to learn about their sense of belonging and science identity (whether they feel like a scientist and part of the scientific community) and, at the end of the term, their views on how the test questions had been posed.
The researchers didn’t find any difference in academic performance or students’ sense of belonging or science identity based on which format of questions was used. But most students preferred the “you” format, explaining that it was the cleanest because it eliminated any extraneous information. And there were suggestions that it also helped students think about work they’d done in the lab and insert themselves into the problem.
Before the experiment, Hsu had favored using real scientists’ names in part because “I thought students would find it cool and interesting” to tie their learning to authentic examples. But that was not how students took it. Some said that seeing a scientist’s name in the problem underscored the difference in status between themselves — still undergraduates — and the Ph.D.s doing the work described.
“It was the opposite of what we expected,” Hsu said. Seeing those names didn’t make students feel inspired or included. It made them feel intimidated.
The researchers are following up on their initial findings, analyzing student interviews and a second version of the experiment in which students in the same class encounter different question framing on different assessments. For now, they make three recommendations for instructors:
- Reduce cognitive load in assessment questions by removing unnecessary details.
- Consider students’ preferences when writing assessment questions.
- Maintain consistency with how assessment questions are framed, and align in-class examples.
Based on what the team has found so far, Hsu says he is primarily using the “you” framing in his own teaching. But given the size of the sample — and the fact that this approach was not favored by a subset of students — there wasn’t enough evidence to make a blanket recommendation.
Have you experimented with how you word test questions? If so, what have you discovered? Let me know at beckie.supiano@chronicle.com, and your example may be featured in a future issue of the newsletter.
Difficult discussions online
Recently I heard from Laura Le, a lecturer in biostatistics at the University of Minnesota-Twin Cities, who identified a hole in guidance on facilitating difficult classroom conversations. Le writes:
“There’s been lots of talk (and I’ve been to a workshop) on how to have difficult conversations in the classroom. However, the focus seems to be on in-person classes, and less so about asynchronous, online ones.
“So I have been pondering as of late how to talk about sensitive topics in an asynchronous course when you can’t read the room. I know I should do it, but have been hesitant to do so because I don’t want students to feel targeted or isolated, so much so that it impacts their experience or learning in my classroom.
“Your latest newsletter brought up something that I hadn’t initially considered, and that was ‘setting ground rules.’
“Do you know if others have thought about how to do this in an asynchronous setting in a respectful and productive/effective way?”
I told Le that newsletter readers might be able to answer that question, and she was eager to hear any ideas you have to share. Send your approach or resource to me at beckie.supiano@chronicle.com, and I’ll compile any leads for Le and share them here as well.
Why instructors need community
Last week I moderated a Chronicle panel on meaningful class engagement (if you click here to register for the event, you can watch a recording of it). I was struck by a comment one panelist, Usha Rao, made when I asked how institutions can better support instructors in this work.
“I personally believe that, just as therapists are expected to go to therapy themselves to unpack their experiences, we should expect teachers to have a community where they can unpack their experiences, learn what they can do, and what is beyond the scope of their job on a given day,” said Rao, a professor of chemistry and biochemistry at Saint Joseph’s University, in Pennsylvania, where she was the founding director of the teaching center. “This is a profession at very high risk of burnout. We have burnout among educators on par with the health-care professions, all the giving professions. We need to teach faculty — and this is especially true for our adjunct faculty, for our untenured faculty, or tenure-track faculty who haven’t gotten tenure yet — we need to talk about what does burnout look like, how do you stop burnout if you catch it soon enough.”
“As teachers,” Rao added, “sometimes we need to recognize that not every student is ready right now to take your help. Sometimes, all you can do is prop open that door just a little bit for the next teacher, or the next class, the next time they can kind of think about internal motivation to walk through that door.”
Thanks for reading Teaching. If you have suggestions or ideas, please feel free to email us, at beckie.supiano@chronicle.com or beth.mcmurtrie@chronicle.com.
As always, nonsubscribers who register for a free Chronicle account can read two articles a month. Your readership supports our journalism.
—Beckie
Learn more about our Teaching newsletter, including how to contact us, at the Teaching newsletter archive page.
