Since last fall, when OpenAI announced the launch of ChatGPT, academics have been consumed by its potential to disrupt education, predicting such revolutions as the demise of the college essay and the prospect that AI could pass freshman year at Harvard.
AI is forcing higher education to ask some tough questions about what it does well.
Generative AI, which has been trained on reams of human writing, had become a good enough mimic that it could create acceptable, and sometimes pretty good, prose. And its easy-to-use interface led professors to fear that students would happily outsource much of the difficult and time-consuming work of reading, thinking, and writing to a chatbot.
But recent news reports and analyses present a more complicated picture, making it harder to predict what its impact on college teaching will be.
For one, the accuracy of ChatGPT and other AI has become a significant concern in higher education, as users have found that it often produces a mix of fact and fiction, such as citing nonexistent quotes by authors or imaginary research papers.
Some skeptics have come out swinging — perhaps none harder than Gary Marcus, an emeritus professor of psychology and neuroscience at New York University who has become a leading voice on AI. In under a year, Marcus wrote on Substack, ChatGPT went from being mistaken for a form of artificial intelligence that could learn and think “to being the butt of a joke, and an insulting shorthand for robotic, incoherent, unreliable, and untrustworthy.”
There are also growing concerns about the legality of scooping up vast amounts of information from the internet, including copyrighted material, on which to train generative models, and about their economic viability, as they rely on low-paid workers and consume enormous amounts of energy. Such threats could end up limiting the scope, effectiveness, and appeal of these models, and thus their value — or threat — to teaching.
Yet while ChatGPT could evolve — or founder — most experts agree that generative AI is here to stay. Rivals to OpenAI are developing their own programs and some of them will become embedded in everyday life, including the tools and products used by professors and students.
For some faculty members, that’s reason enough to incorporate them into their teaching: Better to show students how to use them effectively and to understand their limitations than to ignore them.
Because the field is fast moving, the impact generative AI will have on teaching in the near term is uncertain. Here are a few key questions we will be asking this fall.
Will generative AI find acceptance in academe?
The use of ChatGPT dipped worldwide in June and July. Some wondered whether that meant that people were becoming disillusioned with its inherent limitations. Others assumed that it simply signified that students were out of school and that use will jump again this fall. (More than a quarter of users are 18 to 24.)
Among the limitations is the persistent problem of inaccuracy: Generative AI often just makes things up, or “hallucinates.” And tech leaders have yet to get a handle on the problem. One recent experiment showed that the share of fabricated bibliographic citations dropped from 55 percent in the previous version to 18 percent in GPT-4. As the authors noted: “Although GPT-4 is a major improvement over GPT-3.5, problems remain.”
“I don’t think that there’s any model today that doesn’t suffer from some hallucination,” Daniela Amodei, president of Anthropic, which makes Claude 2, a rival to ChatGPT, told Fortune. “They’re really just sort of designed to predict the next word. And so there will be some rate at which the model does that inaccurately.”
There are also widespread concerns about the accuracy of AI detectors. OpenAI scrapped its own detector, saying it had a low rate of accuracy. Vanderbilt University disabled Turnitin’s AI detector in August because of concerns over inaccuracy and bias, and questions about how the product actually works. Could more universities follow? And if detectors aren’t reliable, will professors be more inclined to ban AI use entirely?
Perhaps because of these and other issues, the public in general is increasingly concerned about the role of artificial intelligence in everyday life. According to the Pew Research Center, the percentage of people who said they were more concerned than excited jumped from 38 percent in December to 52 percent today.
At the same time, a growing number of new generative AI programs are being developed and tested. Google Docs, for example, is experimenting with new AI features. And learning-management systems are starting to embed AI tools designed to give students feedback or help with course design.
One of the biggest challenges to navigate now is the fact that more digital tools will come with generative AI already embedded in them, says Annette Vee, director of composition and an associate professor at the University of Pittsburgh. “It’s everywhere in professional writing.”
“We need to be fundamentally rethinking ways we teach writing, so we are thinking about integrating tools mindfully,” says Vee, who helped develop a new resource, TextGenEd, that provides guidance in this area. “The real challenge is how do we teach courses that are preparing students and that are smart about generative AI? We have very few teachers currently equipped to do that work.”
How do we teach courses that are preparing students and that are smart about generative AI? We have very few teachers currently equipped to do that work.
Michael McCreary, an educational developer at Goucher College who has written about the evolution and future of AI, notes that a lot of faculty members have spent a lot of time practicing prompt engineering with ChatGPT to create rubrics and assignments. He calls this a “fear-based discourse”: If you don’t learn how to use ChatGPT you’re going to get left behind, or so the thinking goes.
In truth, he says, learning how to engineer prompts is likely to be a transitional skill. Soon more sophisticated programs with specialized uses will be on the market. Faculty members would be better off focusing less on prompt engineering and more on determining the problems they would like to solve. For example, a writing instructor could work with students to come up with a rubric that defines the components of a good essay. Then students would generate a series of essays with AI and score them on these jointly designed criteria. “The effect that has is less about making people fearful in a way that motivates them to learn a narrow skill for AI,” says McCreary, who has helped develop a free, AI-powered brainstorming tool for instructors, “and reorients them back to the heart of their discipline.”
What AI guidance and training will colleges provide instructors?
This question is connected to the previous one, of course. If colleges provide support for professors, that will shape whether or how they use AI in the classroom.
Some colleges have begun offering workshops for faculty to learn how to use generative AI. About 600 faculty members at Auburn University signed up for a self-paced course created by its teaching center that covered the basics of teaching with AI, and that included a discussion of course redesign and how to partner with students. Others, like the University of Mississippi, offered summer workshops to prepare faculty for teaching this fall.
But it’s unclear how widespread such training is, or will become, in the coming year. Nor is it yet clear where professors are going to land in terms of embracing AI, allowing it under some circumstances, or banning it. Colleges seem to be deferring to faculty members to set their own classroom policies on appropriate use, while offering guidance on different models. (Harvard’s Office of Undergraduate Education, for one, offers three draft policies that faculty members can adapt: one for “fully encouraging” AI’s use, one for “maximally restricted” use, and one for “mixed.”)
The vast majority of colleges had no formal policy on the use of AI tools in the spring, according to a survey by Tyton Partners. And only 58 percent said they would begin to develop one “soon.” That leaves a lot of professors figuring this out for themselves.
Will the regulatory climate around generative AI heat up?
Generative AI is inherently controversial, both in terms of how it operates and what it produces. The next few months may show us whether governments and other entities will shape its structure and oversight, as well as what it produces, which could in turn affect its usefulness in teaching.
Large language models have been trained on vast quantities of human writing. OpenAI, the creator of ChatGPT, and other organizations with their own generative AI programs have placed guardrails around these tools to minimize their likelihood of churning out biased, inaccurate, or dangerous content.
But those limits have done little to satisfy AI watchdogs. Governments are pondering regulation and starting investigations, concerned about issues of privacy, bias, misinformation and transparency. Some creators have sued, or threatened to sue, generative AI producers for using their work without authorization or compensation.
In a recent Substack post, Bryan Alexander, a senior scholar at Georgetown University who focuses on the future of education, cataloged those and other forces that could well shape what generative AI looks like and can do in the near future. “We have some really huge decision points as a civilization about this,” he said in an interview. “Those are all going to shape what faculty can do.”
In addition to quality concerns, he says, critics have pointed to economic and environmental problems. AI operations create a huge carbon footprint and often rely on low-paid workers in developing countries. That alone could inspire government regulation and also turn people off of using the technology in the first place. Some professors and students, in other words, may decide it’s ethically questionable to use these tools.
Alexander says he could see AI becoming a political football for both parties. “Are you going to get a bunch of conservatives who say this is Evil Big Tech and it needs to be destroyed or smacked down? And will they push for a law? Or will you have Democrats doing the progressive side of it: We think this is biased technology that does harm to creative industries that we support?”
Even if the U.S. decides against regulation, European agencies could step in, he says. And just as the General Data Protection Regulation — passed by the European Union to protect the privacy and security of personal data — has affected what major tech companies do in the United States, so could a similar law on generative AI.
On the one hand, if government regulation makes generative AI more transparent and less prone to bias, that could make it more appealing for instructors to use. On the other, if regulators — or courts — limit access to the content AI needs to produce its models, that could render them less effective in teaching and learning.
Will AI make some courses obsolete?
When ChatGPT appeared on the scene, one of the early analogies was to the arrival of the hand-held calculator. In the same way that students no longer needed to perform tedious calculations by hand, they would also not need to produce writing that could be done by a chatbot, or respond to short essay prompts testing basic knowledge of a topic.
In higher ed, say educational developers, courses largely focused around rote skills and content memorization could become redundant. Think of large introductory classes, for example, with multiple-choice exams testing the ability to remember formulas and facts. Or poorly designed writing courses in which generic prose is considered passable.
Courses that are not interdisciplinary, that don’t focus on digging into evidence, that shy away from asking students what they value, and that instead are “just kind of hand-waving and say, ‘Well, evidence says X, Y, Z,’ could become far less useful,” says McCreary. “We may need to find a way in higher ed of helping students to get some of that content so they can move on to other skills.”
Of course such discussions could quickly become weaponized, notes Vee. ChatGPT could be the Trojan Horse that allows for budget cutting and outsourcing. She points to West Virginia University’s initial recommendation — since scaled back — to eliminate all language programs. Within the announcement was the suggestion that the university was exploring “alternative methods of delivery,” including an online language app.
Vee refers to it as part of the longer-term trend of the “cheapening of higher education,” which includes an overreliance on adjunct faculty to teach hundreds of students at a time.
Will the way courses are taught fundamentally change?
The advent of user-friendly AI that mimics human writing has led to serious soul-searching in the teaching community. If you can outsource something to a chatbot, does that mean it’s not worth learning? More fundamentally, what is it that you want students to know, and how can you tell if they have mastered it?
Professors have long struggled to design assessments to produce evidence that students are learning, with or without ChatGPT on the scene. As Betsy Barre, executive director of the Center for the Advancement of Teaching at Wake Forest University, put it earlier this year, “We can’t see inside your brain.”
If you can outsource something to a chatbot, does that mean it’s not worth learning? More fundamentally, what is it that you want students to know, and how can you tell if they have mastered it?
When Covid closed campuses in 2020, instructors redesigned traditional assignments, tests, and papers for the online classroom. With ChatGPT and other AI, they will have to do it again. Some professors have concluded that the only way they can guarantee students won’t outsource their work to AI — particularly on something that is challenging — is to do the important stuff inside the classroom.
But others have pushed back against that approach. Not only are in-class assessments limiting — many students perform poorly on timed, hand-written essays and tests — but they also skirt a central part of teaching: to motivate your students to want to learn.
John Warner, author of two books on writing, tackled that dilemma in a recent essay, arguing that motivation should not come about through grades and mitigation of cheating through AI, but through the design of the learning experience itself. Faculty members should focus on the attitudes they want to foster in students, and the experiences that will help them engage with their coursework.
“It’s best if there are real stakes attached to the work, for example, an authentic audience the student is writing to,” he writes. “A subject on which students have both sufficient interest and knowledge in order to feel as though they can write convincingly to this audience also matters a lot.”
Many others have made this argument.
“Until we shift our own identities as instructors as graders or learning cops, trying to figure out how much everyone has learned,” says McCreary, “there are going to be significant issues for everyone next semester and beyond.”
AI is forcing higher education to ask some tough questions about what it does well, says Vee. She notes that the National Survey on Student Engagement consistently demonstrates that students who work closely with a professor, do original research, and have internships and engaging classes have a better experience in college. None of those things are replicable with AI. “But those things require support and administrative ingenuity. They require funding.”
Alexander, for one, doesn’t think generative AI is going away. Nor does he think professors should shy away from using it in the classroom. While it’s not necessary in every discipline, there are certain ones — like computer science or writing — where colleges will need to prepare their students for a future with AI in it.
Even as he writes about possible legislation and lawsuits that could shut down programs that rely on scraping the internet, Alexander himself is teaching a course this fall at Georgetown on the future of higher education, in which every session incorporates some form of AI usage. In a session on a forecasting method known as horizon scanning, students will experiment to see how AI tools could help with that process. In another, they will create futuristic fiction about higher education using AI.
“We’ll see how that goes,” he says. “If three weeks from now a federal judge pulls the plug [on the generative AI industry], then I have to rethink what I’m doing.”
This year will offer pivotal developments that could answer many of these questions around teaching in the era of generative AI. If you or your campus is delving into these issues in a meaningful way, our reporters who cover teaching would like to hear from you. And if you have other questions on this topic you’d like to see us explore, write to us at beth.mcmurtrie@chronicle.com and beckie.supiano@chronicle.com
