That reporting culminated in this 2019 story, which tried to make sense of some disappointing findings from several high-profile efforts to expand on promising early studies.
One concern that seems to pop up in every discussion of nudging is that it could make college too easy for students. Shouldn’t they be able to navigate their way without text-message reminders? Most observers, I’ve found, can be convinced that helping students apply for financial aid is in-bounds: Wealthy students don’t have to file, and it’s hard to argue that a process that begins with finding your parents’ tax returns has much to do with someone’s ability to succeed academically.
The idea of bringing these interventions into the classroom, though, is more fraught.
So when I saw preliminary results from an academic-nudging experiment at Georgia State University, my interest was piqued. Georgia State is well known for its use of data analytics to support student success, and a chatbot it created to send students reminders and answer their questions was among the more-promising directions for nudging mentioned in my 2019 article.
The university is now testing chatbots in a classroom setting: an online section of a large, required government course. The first wave of results, based on 500 students who took the course in the fall, are outlined in a new working paper led by Katharine Meyer, a postdoctoral student at Brown University. The research team found that the chatbots had significant effects on student performance — especially among students from historically underrepresented groups.
Sixty percent of students in the experiment’s control group earned B’s or better in the course. The assistance of the chatbots increased the chances students earned such grades by eight percentage points. The researchers also looked at the impact on first-generation students, 45 percent of whom earned B’s or higher in the control group. Receiving the messages increased the chances first-gen students earned such grades by 16 percentage points.
And what drove that increase? First-generation students who got the chatbot reminders subsequently took the recommended actions — like completing assignments — at higher rates.
I was curious how the course’s instructor, Michael C. Evans, a senior lecturer in political science, viewed the experiment. Evans told me he’s always eager to work with the university’s student-success efforts. And the chatbots aren’t all that different from other forms of support built into the course: Evans sends batches of customized emails through the learning-management system already. Still, the reminder messages provided another layer of support, and in a different medium . The chatbots also let students pose questions, which are fielded by a combination of the system’s AI-enabled chatbot technology and a graduate teaching assistant. The teaching assistant will respond to questions that the AI does not yet recognize and also may monitor and follow up on AI responses to provide additional information.
Evans had several observations about giving students this kind of support in a course. First, he noted, the initial findings are from a fall semester — meaning many of the students in this intro course were in their first term of college. “We all remember what that was like,” he said. “A lot of your mind space is devoted to figuring out how to be a college student, but also, where do you go to eat? How do you find your classroom?” There’s a lot to keep track of. “Anybody in that state of mind is going to be prone to missing due dates; forgetting little logistical things.”
It stands to reason, he thinks, reminders would help.
Some professors view meeting deadlines as an important skill. If they do, Evans thinks, they should make that explicit to students, and support them in developing it.
And Evans disagrees with the interpretation that students’ ability to remember and meet deadlines demonstrates organizational aptitude. Instead, he thinks, “a lot of the students meeting the deadlines without assistance aren’t especially organized, either — they just have fewer concerns in their life” like family and work responsibilities.
It’s not about whether students are organized, in other words. It’s about how much they have to organize. If that’s the real difference between meeting and missing deadlines, then sending reminders is a step toward improving equity.
What do you think of nudging students in a classroom context? Have you tried anything along those lines, and if so, how did it work? Share your thoughts with me at beckie.supiano@chronicle.com and they may appear in a future newsletter.
Last Call
Our final Talking About Teaching virtual event — a reader mailbag, where we’ll pose a bunch of your questions to our panelists — is Friday, April 29, at 2 p.m. Eastern. It’s not too late to join us: Register here. The same link will allow you to view recordings of our previous three sessions, if you missed them.
We’re always interested in your feedback. If you attended the sessions in our series, what did you think? What can we do — through the newsletter and in other formats — to provide support and community around teaching moving forward? What’s on your mind as you close out this semester? Drop us a line anytime: beckie.supiano@chronicle.com or beth.mcmurtrie@chronicle.com. Your observations are an important ingredient in our work.
Pop-Up Faculty Development
Many professors are simply too exhausted and overworked to summon the energy to attend teaching trainings, and we’ve written about how some faculty developers are responding.
Victoria Russell, who directs the teaching center at the University of Mary Washington, wrote in to share a creative approach she’s trying out. “My advisory group and I hit on an idea that started as a joke but is getting rolled out (literally) next academic year,” she wrote: “the Center for Teaching Pop Up Cart.”
Russell, also an associate professor of education, bought a mail cart — “think the kind in mailrooms in old sitcoms that hold multiple files,” she wrote. “I’m decorating it with a pennant, our school mascot (the eagle), and a bell. On the top shelf of the cart, I will have water and portable snacks; the bottom shelf of the cart will have hard copies of the latest articles, a selection of books from our center’s library, and a ‘tip jar’ with ideas for a specific topic or question.”
She plans to hold “pop up” hours in academic buildings around campus on a set schedule, and also make brief appearances at department and college meetings, by request.
“I’m at a small liberal-arts school, so the travel around campus is manageable,” Russell wrote. “I think a certain ‘pedagogy of care’ is as important for my faculty colleagues as it is for our students right now. At the very least, I might get some smiles, and those are pretty valuable lately too.”
Thanks for reading Teaching. If you have suggestions or ideas, please feel free to email us at beckie.supiano@chronicle.com or beth.mcmurtrie@chronicle.com.
— Beckie
Learn more about our Teaching newsletter, including how to contact us, at the Teaching newsletter archive page.
