How Artificial Intelligence Is Changing Teaching

Two years ago, Craig Coates, an entomologist at Texas A&M University, was asked to take over a science course plagued by cheating. The previous instructor of the large lecture course, “Insects in Human Society,” had tried to stay one step ahead, but in the class, oriented around online quizzes and tests, students would quickly share new material as soon as it went up. “It became an arms race between pushing out more questions faster and cheating,” recalls Coates, an instructional associate professor who has taught on the campus for nearly 20 years. “It moved everything toward rote learning.”

He wanted to reorient the course toward writing and discussion, convinced that the method would not only reduce cheating but also be a more engaging way to learn. But with 500 students — 200 in person and 300 online — grading would be a challenge. He experimented with one assignment, and it took days for him and three teaching assistants to complete the grading. “It was obviously going to be impossible,” he said, to do it by hand.

Peer review was an option, but he and his TAs needed help sorting, assigning, and evaluating submissions. That led him to try a tool that uses algorithms and analytics. The switch was a success: Students enjoyed writing about current research, including keeping an insect blog and debating topics in entomology. The response, Coates says, has been “overwhelmingly positive.”

Artificial intelligence is showing up more frequently in college classrooms, particularly at big institutions that are seeking to make large courses more intimate and interactive. A professor at Georgia Tech developed virtual teaching assistants and tutors. Researchers at Carnegie Mellon University are creating conversational agents to promote online discussion. And on a growing number of campuses, professors are using adaptive courseware that adjusts lessons according to students’ understanding and deploying AI-driven tools, like the one Coates used, to promote writing and peer review.

As artificial intelligence enters our daily lives through smart speakers and chatbots, it’s no wonder that academics are exploring its potential in teaching.

The technologies in these tools vary, of course. Some focus on sorting information to a help a professor organize and evaluate assignments. Others use automated text analysis to mine students’ writing and fashion relevant prompts. Adaptive courseware is built around the sequencing of lesson plans, selecting content based on regular assessments of what students know. Advanced tools are based on machine learning, a form of AI that learns from user behavior. And many forms of AI draw on research in learning science, cognitive psychology, data science, and computer science.

This trend prompts serious questions. When you’ve got artificial intelligence handling work that is normally done by a human, how does that change the role of the professor? And what is the right balance of technology and teacher?

Some, like Coates, feel that algorithm-driven technologies can be useful aids in large classes. They automate some of teaching’s routine tasks, so that professors can do what no machine can — challenge and inspire students to gain a deeper understanding of what they’re learning. These technologies, advocates argue, are simply tools in service of creative forms of teaching.

But skeptics worry that if education is increasingly reliant on artificial intelligence and automated responses, it will put technology in the driver’s seat and prompt formulaic approaches to learning. Some of the algorithms used in AI-driven tools are built on large data sets of student work, raising privacy and ethical questions. Turning to technology for solutions, critics say, may also short-circuit conversations about some of the structural challenges to effective teaching and learning.

That avoidance of structural issues troubles people like Kevin Gannon, a history professor at Grand View University, in Des Moines, and head of the campus Center for Excellence in Teaching and Learning. “We see a particular problem, whether it’s retention rates for minoritized students or large class sizes and heavy grading loads,” he says, “and our first instinct is to find something new and hot to address the problem rather than focus on the classroom and faculty and students.”

Perhaps nowhere are those tensions more apparent than with adaptive courseware, sometimes called intelligent tutoring systems. The programs have grown increasingly popular as an alternative to large classes that emphasize lecture and memorization. They have also given rise to the specter of the robot teacher.

With adaptive courseware, students first encounter material outside of class, often through short video lessons and readings. They take quizzes that assess their understanding of the material and, depending on the results, the courseware either advances them to the next lesson or provides supplemental instruction on concepts they don’t yet grasp. Advocates say this lets students study at their own pace and frees up the instructor’s time in class to shore up students’ knowledge or help them apply what they have learned.

Adaptive courseware has made the most inroads in introductory STEM courses, particularly math, in which it is easier to sequence content and test understanding of concepts than in, say, a literature class. Administrators at several large public universities, including the University of Central Florida and Georgia State University, have seen positive results with the use of adaptive courseware, which are often accompanied by a rethinking of classroom time, to emphasize active-learning techniques.

Susan Holechek became a convert several years ago as Arizona State University began testing adaptive courseware in a number of introductory courses. The biology instructor teaches a class for nonmajors — who probably wouldn’t be there if they didn’t have to be.

“It’s a very, very, very tough crowd,” she says. Holechek tried out a small class that used a combination of adaptive courseware and active learning, while also teaching her conventional 300-student lecture course.

For a lesson on DNA, for example, her traditional lecture focused mainly on conveying information, she says. Students in the smaller, pilot section might read the textbook, watch a video, and then take a short quiz the night before class to see how well they understood the material. If they scored poorly on some part of the lesson plan, the courseware would then send them back to review that concept, with new examples or additional information to help them grasp it better.

Using adaptive courseware enables Holechek to spend class time on exercises that encourage students to apply concepts that they encountered online. In one exercise, called “Who Kidnapped Sparky” (the university’s mascot), they work with DNA samples to understand their use in forensics. “It allows me to be more creative,” she says.

The adaptive courseware also helps her create opportunities for more points of contact — between her and her students, and her students with the material. A dashboard helps Holechek keep track of how well each student is doing on homework and quizzes, with a cumulative “mastery” rating attached to each. Every Sunday the instructor goes over the collective results from homework reviews to see which concepts her class struggled with. Then she builds a brief Monday-morning lecture around that material.

The proportion of students earning a C or better in Holechek’s course rose from 76 to 94 percent in the pilot and has continued to remain strong now that she has switched entirely to adaptive learning.

Other experiences with adaptive courseware have been more mixed. A recent study looking at results from across a range of colleges found little to no difference in course grades. This points to a larger challenge with education technology, ed-tech experts say: If instructors expect tech to fix classroom problems but don’t address underlying pedagogical issues, they are likely to see limited results.

After stumbling through some early experiments with adaptive learning, Arizona State found that training instructors in active-learning strategies increased the impact of the adaptive courseware they used. The university has also redesigned classrooms to promote collaborative work and offers a starter kit to those new to active learning that includes sample exercises focused on problem solving and critical thinking.

The connection between pedagogy and technology is often missing in the debate over AI-driven teaching tools, notes Barbara Means, executive director of learning-science research at Digital Promise Global, an independent nonprofit group that studies digital technologies in education. “We end up talking past each other because people tend to say, ‘Does it work?,’ " she says. “And it really needs to be a more nuanced question.” In other words, what works for one kind of student in one kind of learning environment with a particular set of educational goals might not work for someone else.

Ryan S. Baker, an associate professor in the Graduate School of Education at the University of Pennsylvania, who does research in this field, goes so far as to call AI a “red herring” in these discussions. “Well-designed activities, on or offline, can promote critical thinking,” he says. AI tools to support these activities are simply “a secondary part of it.”

One of the best ways to help students develop their critical-thinking skills is to get them to write. That can be accomplished by writing more — and more often — and reviewing their peers’ writing. This has been a particularly fertile area of exploration for AI in the classroom.

Kathleen West tried to encourage her students write more frequently and more meaningfully in her online psychology classes at the University of North Carolina at Charlotte. But she was stymied by clunky technology and large class sizes. Sometimes she would post a discussion question in her learning-management system and no one would answer it. Other times the conversation got so unwieldy it became impossible for her to follow.

West, who is a lecturer and academic adviser, began using an AI-driven tool called Packback to support online discussions. Each student is required to post a question relevant to the course once a week and then respond to two other students’ questions.

Packback takes care of basic monitoring, like making sure the students are on topic and are asking open-ended questions that encourage discussion. It prompts students to supply answers that are backed up with sources and to write more in depth. And it uses an algorithm to give a ‘curiosity score’ to each post based on those and other measures. Because everyone can see all the scores, some instructors say students often try harder when writing subsequent posts.

West says the tool frees up her time to do more-engaged teaching. That might include joining an online discussion to press a student to elaborate on her ideas or show better evidence of her assertions. An additional benefit of doing more writing, she says, is that students’ writing and critical-thinking skills have improved overall.

“Writing on exams has increased exponentially in quality because they’re practicing writing,” she says. “The quality is just night-and-day different.”

The tool Coates uses, called Peerceptiv, works by evaluating the reviewer, not the writing itself, says Chris Schunn, a professor of psychology, learning sciences and policy, and intelligent systems at the University of Pittsburgh and the principal investigator behind the program. It helps instructors by anonymizing and distributing student work, allowing each writing assignment to be reviewed by several classmates. Then it monitors and graphs student feedback, including feedback on the reviewers.

If a student hands out high ratings to every classmate while others are writing more-nuanced evaluations, his rank as a reviewer will drop. If he gives feedback that other students say is helpful, his score rises. Essentially the AI is looking for outliers, common ground, and dissension in the reviews, Schunn says, then feeds that information back to the professor in the form of a dashboard. If a particular assignment seems to be generating a lot of conflicting peer reviews, that might signal to the instructor that the topic itself is confusing.

“It shifts you from randomly looking at everything,” he says, “to paying attention where work is problematic.”

But writing is complicated; it can be resistant to standardized evaluation. Other researchers are using AI to better understand what kind of feedback improves writing and how it might vary by discipline. Valerie Ross, director of the Critical Writing Program at Penn, is working on a project, funded by the National Science Foundation, to build what she calls an “ecological model” in support of good writing. By that she means one not built on generic rules — which is often a criticism of writing tools — but specific to discipline, genre, and classroom environment.

To do that, she and her collaborators at the Massachusetts Institute of Technology, the University of South Florida, Dartmouth College, and North Carolina State University are mining writing samples and peer reviews from about 10,0000 students as well as instructor feedback. They will use predictive modeling to identify what seem to be the most useful parts of the process and to answer some big questions: What are valid measures of writing development? Are there particular comments or feedback that lead to improved writing? “Good writing is so socially situated,” she says. “That’s part of the limitations of the AI-big data approach. All those rules are just tools for writers.”

But even when it focuses more narrowly on evaluating writing in a specific discipline, artificial intelligence may still be of limited use. At the University of Michigan, concerns about the lack of writing in foundational STEM courses led faculty members to create an automated peer-review system in hopes of changing that dynamic. Created with support from the university and the NSF, the system, called M-Write, has been tested on 8,000 students to date.

Similar to Peerceptiv, M-Write anonymizes, sorts, and assigns work by students so that they can review one another’s writing. An integral part of the system, says Anne Ruggles Gere, an English professor who helped create it, is the use of writing fellows. These trained undergraduates act as connectors between students and professor, stepping in, for example, when students get confused by a particular assignment. “They are really the human link that makes the whole system cohere and work well,” says Gere, who heads the university’s Sweetland Center for Writing.

M-Write researchers would like their system to be able to evaluate how well students understand the concepts they’re writing about, but that goal has proved to be elusive, says Ginger Shultz, an assistant professor of chemistry and co-creator of M-Write. She and her colleagues have figured out ways to tailor messages to students based on, say, whether they’ve made substantial revisions in their writing.

But evaluating their conceptual learning through AI has proved far harder, in part because it is so specific to a discipline or even a particular course. To analyze short essay answers of a few sentences for conceptual learning, some software uses what’s called a “bag of words” model, in which the program searches to see whether certain types of words are in the text and how often they appear. A question about chemical equilibrium, Shultz says, might prompt answers that include the words “increasing,” “Le Chatelier’s Principle,” or “reactants.”

But that approach doesn’t work for essays of one or two pages, which might cover several ideas. “We’ve been poking at this for two years,” she says, “but accuracy isn’t where we want it to be.”

Gere, who is also president of the Modern Language Association, sees AI-driven teaching tools as part of a spectrum of technologies increasingly prevalent in higher education, like advising apps and predictive analytics. Because faculty members are becoming more familiar with how technology is being used on campus, she believes they are more willing to experiment with it in the classroom. “There’s a general awareness,” she says, “that this is, in many ways, going to be part of our lives as academics as we move forward.”

Kevin Gannon, the Grand View history professor, isn’t sure that AI-driven teaching is a positive trend. He’s no Luddite: he uses technology in his teaching, runs flipped classrooms, and keeps a blog called The Tattooed Professor. But if colleges are dropping thousands of dollars on tech-driven solutions, he argues, that’s money they’re not spending on hiring more faculty members and teaching assistants. If decision makers believe that AI tutors are effective teachers, he asks, why should they increase salaries and budgets? “I worry that this will be the cost-efficient solution to tuition-dependent systems. And we grow further and further away from conversations about, Is this a public good or not, because we have Auto Teach English 101.”

He also worries that AI is creating an even deeper divide among the institutional haves and have-nots. No elite colleges, he says, will ever brag about using AI to automate teaching. Their gold standard will remain small class sizes and close contact with professors.

Deborah Beck shares Gannon’s skepticism about the use of automation in the classroom. She teaches an introductory course in classical mythology at the University of Texas at Austin. Her class is large — about 200 students — but she gives weekly writing assignments and reads them all.

She sees that as integral to her job. Modeling good writing, thoughtful interaction, and respectful disagreement is part of the teaching process, says Beck, an associate professor in the department of classics. “That’s really hard to outsource to AI.”

She also believes that her students value her close attention to their work. They tell her the discussion boards were among the most valuable tools they used. In one assignment, she asks her students to analyze how the characters in a reading talk about the ethical issues surrounding the rape of Lucretia, a foundational story in Roman history. Then she shares with the class why she particularly liked one response, explaining how it was written in a lively manner, gave insights beyond what others had already written, and offered specific examples to support the writer’s argument.

Beck wonders how automated writing prompts could be as specific, or work as well for students who may have come to college not really knowing how to study. “They need help in learning how to learn,” she says. “And that’s something we all need to think about. Especially in ed tech.”

As academics experiment with AI in the classroom, privacy experts say more attention needs to be paid to big-picture issues of ethics and efficacy.

Technology alters teaching environments in critical ways, yet there is little public scrutiny of those changes, says Elana Zeide, a technology-law expert and fellow at the University of California at Los Angeles’ School of Law. “It’s being adopted without much thoughtfulness or much education of the people using the tools.”

One factor is the reliance on algorithms and continuous data collection to make these tools work. Do professors really understand how those tools make their decisions? Probably not, Zeide says, since the algorithms are proprietary. Some tools also control content, determine how learning is measured, and define outcomes, which shifts pedagogical decision-making away from educators toward private, for-profit companies that sell these products, she says. “In contrast to the public and heated debates that accompany textbook choices,” she notes in a recent article, “schools often adopt education technologies ad hoc.”

Figuring out which tools might be beneficial, and how they work, is the hard part, of course. Vendors use the language of learning science to describe the benefits of their tools, but their promotional materials can make it difficult to distinguish between broad claims and solid grounding in science and experimentation.

Packback, for example, says it uses Bloom’s Taxonomy to prompt higher forms of thinking that shift students away from simple recall and toward analysis and evaluation. The company also claims to help awaken “fearless, relentless curiosity” by encouraging students to ask open-ended questions.

Experts say vendors should show the research that backs up their products and agree to test runs so that instructors can see how well their programs operate. The technologies should also be adaptable to a particular professor’s course design.

“Vendors say we literally can’t tell you how the AI is making decisions on any given case, because the whole point is that it is learned and developed with a set of criteria we input at the start,” says Martin Kurzweil, director of the educational-transformation program at Ithaka S+R, a nonprofit group that studies and supports the use of technology in higher education. “I don’t totally buy that.”

While the ‘intelligence” part of AI in teaching is still limited, experts envision a future in which the technology becomes broadly multifunctional. Artificial assistants could help design textbooks, deliver course content, develop quiz questions, evaluate the answers, monitor online discussions, adjust to students’ learning styles, and advise students on their path through college.

Researchers are already making advances on these fronts.

Ryan Baker, of Penn, is one of many researchers studying ways to identify students’ habits and attitudes in hopes of developing courseware that can strengthen study skills through tailored messages and tips.

Researchers at Pennsylvania State University are piloting a so-called distractor generator that creates the false answers needed to populate multiple-choice quizzes. They are also using AI and machine learning in a program called BBookX to help professors design textbooks.

And at Carnegie Mellon, researchers are experimenting with a technology to promote better discussion among students online. Known as a conversational agent, it aims to spur deeper interactions among students by prompting them to react to classmates’ ideas.

Robot tutors aren’t about to replicate the full array of teaching-and-learning behaviors that take place as a matter of course among people anytime soon. But artificial intelligence does raise a provocative question, one no doubt on the minds of educators worried about the decline in public higher-education funding: If administrators are willing to cut corners by paying low wages to adjuncts and giving them heavy courseloads, what’s to stop them from trimming their costs even further by offering students some adaptive courseware and a teaching assistant instead?

Institutions inclined that way, says Baker, “are probably going to be willing to accept low-quality solutions.”

He and other educator-advocates say AI can be of real value to learning. Algorithms can reveal patterns of student behavior not immediately noticeable to a professor. Adaptive courseware can nudge students toward effective learning strategies. Tools that can outsource lower-level tasks are worthy of consideration.

Just as long as the instructor remains in charge of the classroom.

Beth McMurtrie writes about technology’s influence on teaching and the future of learning. Follow her on Twitter @bethmcmurtrie, or email her at beth.mcmurtrie@chronicle.com.