It turns out the assignment went far better than he ever expected, to the point where he’s convinced that his students are now as wary as he is of outsourcing hard work to AI. “If you’re a professor who doesn’t want your students to use ChatGPT,” says Weiss, “if you want to really discourage it, do this experiment. It will scare the crap out of students, and it should.”
So what did Weiss do? First, he had his students read Joan Didion’s essay “Why I Write.” Then he asked them to post their observations on the essay to the class discussion board, and respond to their classmates’ comments. So far, pretty standard stuff.
Then, in class he put them into groups where they worked together to generate a 500-word essay on “Why I Write” entirely through ChatGPT. Each group had complete freedom in how they chose to use the tool. The key: They were asked to evaluate their essay on how well it offered a personal perspective and demonstrated a critical reading of the piece. Weiss also graded each ChatGPT-written essay and included an explanation of why he came up with that particular grade.
After that, the students were asked to record their observations on the experiment on the discussion board. Then they came together again as a class to discuss the experiment.
Weiss shared some of his students’ comments with me (with their approval). Here are a few:
I feel that the process of using chat GPT actually prolongs the time that it would have taken to write the essay if we had just used our own thoughts to write the paper. This is because thinking of prompts to put into the generator is time consuming, and also trying to decipher what is good enough to include in the paper also takes a long time.
It felt like eating a bowl of store bought pasta with nothing in it. ... To say the least, I doubt I’ll ever use it for writing again, but this opportunity was an eye opener.
I feel like it was a very bold decision for Dr. Weiss to have this class explore AI when it can possibly lead to students using it as an easy way to create an essay. In reality, the essay wasn’t as great as expected. AI is a very smart invention, but it still cannot compare to natural human thought. The essay came out bland and didn’t really express any emotion. It was very straightforward, almost like answering a question rather than expressing an opinion.
Those responses were quite typical, he says. Weiss’s (hypothetical) grades of the ChatGPT essays mostly ranged from 50 to 80 percent. One earned a near-perfect score, though, which the students in that group attributed to better prompting and more work in assembling the final product.
Much to his delight, the assignment also energized his students. The group exercise was a great introductory experience, and students’ curiosity about ChatGPT prompted a lively classroom conversation.
Weiss wrote to me after the experiment was over to share what his class did, including reflections from him and his students. I also followed up by phone to hear more about what surprised him and what he learned from the process.
One of Weiss’s first discoveries, he said, was that most of his students had not used ChatGPT and didn’t know much about it. He had assumed that students were more AI savvy than they actually were. I asked Weiss if there’s anything distinct about his students that might have made them less familiar with ChatGPT. He said that his campus is primarily a commuter school and that many of his students work, often full time. So perhaps they have had less time to experiment with this new tech.
The second surprise was that many of his students assumed that ChatGPT was designed to help people cheat. He explained to them that ChatGPT actually has many purposes and they were going to learn how it worked.
Weiss is mindful that his experiment was not designed to help people get the most out of ChatGPT. There are academics out there, he notes, like Ethan Mollick, who are continually testing the boundaries of AI and learning how to use it effectively. Weiss’s experiment was pretty basic: Students had a limited amount of time to turn out one essay.
“Some instructors who advocate for the use of AI might be critical of what I did from that standpoint,” he wrote in one reflection. “But I think it was very effective at introducing the technology to many of them and making the point, a point that I want to make, that this technology is not for ‘cheating’. If you think you cannot do the work and generate a writing assignment in a short period of time using ChatGPT, without knowing the material well yourself, you’re going to fail.”
Weiss followed the experiment by asking students to write their own essays on “Why I Write,” and allowed them to use AI, if they cited what they did. Most chose not to, he reports, and those who used it did so in a limited fashion — such as to correct their grammar near the end of the writing process.
Weiss plans to continue to have his students experiment with ChatGPT. Later in the semester, for example, he will ask them to have it generate an essay on something they have some familiarity with: say, a sports figure or a band. They’ll both critique the output and then try to refine their prompts to turn out something they consider acceptable.
The experiment is also leading Weiss to rethink the purpose of a course such as his. Traditionally, writing instructors have emphasized grammar, vocabulary, and good sentence structure.
But since students can now have ChatGPT fix those things, “I believe we now need to emphasize the importance of students writing in their own, distinctive, personal voice, even if the writing is not technically correct,” he wrote in one reflection. “As long as the level of errors does not completely distract from the message being conveyed, we need to be more forgiving than we have been in the past. That last bit can now be done with the assistance of a machine. But it is especially important to get students writing early drafts in their own voice, even if those drafts include a fair number of errors.”
Have you experimented in class with ChatGPT or other generative AI to get your students to think more carefully about using it? If so, write to me at beth.mcmurtrie@chroincle.com and your story may appear in a future newsletter.
ICYMI - AI edition
- Want to teach your students about how AI tools work? The University of Maryland’s Teaching and Learning Transformation Center and its University Libraries created a Canvas module on AI and information literacy for instructors to use in their courses. You can find the module here. It is offered for free under a Creative Commons license.
- Tackling grade inflation and discerning the goals of student writing can help instructors worry less that generative AI can replace human work, writes Michael W. Clune in this Chronicle advice piece.
- Concerned that your students might cheat with ChatGPT but don’t want to police them? Jordan Alexander Stein offers strategies that may reduce the incentive to cheat in this Chronicle advice piece.
Thanks for reading Teaching. If you have suggestions or ideas, please feel free to email us at beckie.supiano@chronicle.com or beth.mcmurtrie@chronicle.com.
— Beth
Learn more about our Teaching newsletter, including how to contact us, at the Teaching newsletter archive page.
