Virtual Reality Comes to the Classroom

Last year, Nhora Lucía Serrano added a twist to her literature course at Hamilton College. She asked her students to design worlds in virtual reality, inspired by novels such as Alice in Wonderland and The Wizard of Oz.

A colleague in the biology department, Natalie Nannas, is helping develop virtual DNA, one of the trickiest structures for undergraduates to comprehend, particularly in two dimensions.

And before the end of this semester, students in Heather Buchman’s conducting course practiced in front of a virtual orchestra before leading a live ensemble as part of their final grade.

Extended reality isn’t just coming to college. It’s already there. Headsets and glasses, 3-D models, and 360-degree videos are rapidly becoming tools of choice for curious professors. From studying microscopic organisms to exploring outer space, they are taking advantage of technology to experiment with augmented reality in teaching.

But what is the pedagogical value of a virtual or enhanced experience? Just because students may like it, does that mean they will actually learn more than they would through a simple computer program or a textbook and lecture? And how do you build a meaningful virtual lesson into your course when you are a learner yourself?

While ed-tech experts predict an acceleration of activity in the next few years, they caution that without a strong pedagogical foundation, extended reality will never have more than a visceral impact on students.

“Start with a problem that needs to be solved, as opposed to, ‘We have this cool VR thing and let’s see how to use it,’ ” says Jeremy Bailenson, founding director of the Virtual Human Interaction Lab at Stanford University. “That may sound obvious, but I promise you that’s not the norm.”

Hamilton, a small liberal-arts college in upstate New York, is probably not the first place you’d picture as a hotbed of virtual experimentation. But its relationship with immersive tech is no accident. The college has an active digital-humanities program and has emphasized digital fluency across the curriculum. It’s also part of the Campus of the Future Project, a collaboration among several colleges, Hewlett-Packard, and Educause to identify the types and uses of 3-D technology that show the greatest potential for learning and research.

For the past three years, Hamilton has been experimenting with 3-D scanners and printers, workstations that can handle virtual graphic processing, and virtual-reality headsets, among other tools. It also has a roster of educational technologists, librarians, and instructional designers who collaborate with professors and students.

Serrano, who created the virtual-reality-and-literature course, has a longstanding interest in visual studies. Her syllabus included novels, graphic novels, short stories, and movies, as well virtual-reality design workshops.

Over the course of the semester, students dove into dystopian works like The Handmaid’s Tale and pioneering movies like The Matrix. They produced blogs, papers, and a final project, in which teams created virtual worlds to represent works they had read. The projects stretched out over 10 weeks, giving students the time to develop their technical skills in Unity, a virtual-reality software, as they debated how to evoke specific moods and events.

Andrew Groll, a senior, took Serrano’s course because it married his interests in literature and digital arts. His team included students with a mix of academic and technical backgrounds, he says, who designed a virtual journey based on the short story “The Night Face Up.” In the story, a nameless man in a nameless city ends up in the hospital, his consciousness drifting between modern and ancient times. Groll’s team set about creating worlds for the user to move through, prompting discussions about how to express fear, confusion, and wonder. “It wasn’t ‘Here’s the book, talk about it, and go do something digital,’ ” Groll says. “It was using digital to keep the conversation going around the book.”

Serrano believes the VR project encouraged students to think more deeply about the readings because they had to try, in part, to recreate them. “I noticed early the switch,” she says. “They were not just analyzing stories but asking questions. Does color matter? What does it mean to be stepping between one world and another? And that wouldn’t have happened without the VR experience.”

Hamilton College literature students built virtual worlds based on novels and stories they read in class. One team recreated scenes (above) from Julio Cortazar’s “The Night Face Up.,” a short story in which a man, hospitalized after an accident, drifts between two worlds. (Andrew Groll)

Now Hamilton’s associate director of digital learning and research, Serrano attributes the course’s success to meticulous planning and the support she received from the library and IT-services staffs. Early in the semester, for example, students designed simple virtual objects that might appear in, say, The Wizard of Oz, to familiarize themselves with Unity. These workshops were led by Ben Salzman, an instructional designer and 3-D-technology specialist, who has worked with students and faculty members across campus on immersive-reality projects.

Serrano’s experience reflects some of what intrigues academics about extended reality, a term that covers both virtual reality, which immerses you in an alternate world through a headset, and augmented reality, in which you see your surroundings — through glasses or a handheld device — enhanced by virtual objects.

Extended reality can make you feel as if you’re in a place rather than just reading about it. Virtual tours are some of the earliest and most widespread examples of this work.

In some cases, these tours provide an immersive rather than curated experience, allowing students to focus on what they find most interesting. At the University of Pennsylvania, Peter Decherney and his students in the Cinema and Media Studies Program created 360-degree and virtual-reality films focused on refugee communities in Kenya and around Philadelphia. They also produced a 12-part series on Puerto Rico this past summer. “In an immersive experience, you have to give up a lot of control. I compare it to a museum visit,” says Decherney. “Spectators have to become more active in the process.”

Extended reality is also often hands-on. Students grasp virtual objects and take them apart. They may dive into a virus to look at its components or crouch down to look underneath a rock formation inside a cave.

Bob Handelman

Last year Salzman’s team at Hamilton built a five-foot-long DNA model for Nannas, an assistant professor of biology, to use in her classes. It was a hit. “I’ve had a lot of students coming in playing with the model, twisting it. And I’ve had a lot of comments that wow, this really helps,” she says. “I taught the same concept without a model, and students really struggled with it. We went over it several times. I tried 2-D images. It was just hard to demonstrate.”

Now the ed-tech team is building a virtual environment in which students can manipulate DNA in three dimensions. They will be able to zoom out to get a sense of its scale, or zoom in and see its components. “It’s not just informing them of one concept, but building a map of how everything connects,” Nannas says. “That’s my hope for 3-D models or immersive technology. Do we help students get that mental picture quicker and more completely, so they can analyze problems from different angles and make those connections across materials and disciplines?”

Extended reality has also been shown to help people to develop skills that benefit from repeated practice.

When Buchman, the music professor, heard that Hamilton had the technology to record both a 360-degree video and sound that shifts with the listener, she jumped at the chance to create a virtual experience through which students could practice conducting.

“Podium time is a conducting student’s scarcest resource,” she says. “When you do get a few minutes in front of a large ensemble, it goes by very quickly, and you don’t have time to get used to it before it’s over.”

Yet it can also be terrifying. “To conduct your own piece, to be in front of an ensemble for the very first time,” says Buchman, “it can feel like you’re going in front of a firing squad.”

Her hope is that virtual preparation can both improve students’ performance and reduce their anxiety.

A Virtual Orchestra

Andrew Groll

Students later practiced conducting the virtual orchestra by wearing a special headset through which the sound of the orchestra changes as they turn their head. (Ben Salzman)

So what do we actually know about the pedagogical impact of extended reality? In short, not a lot.

Researchers are pretty sure about a couple of things, says Bailenson, of Stanford. One, that virtual reality gets people excited, and perhaps primed, to learn. That’s a big deal if you’re tackling a subject that could otherwise seem dry or abstract.

Grinnell College Immersive Experiences Lab

The other area where virtual reality has proved its merit, Bailenson says, is spatial training, which he describes in his new book, Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do (Norton, 2018). A golfer practicing her swing, a technician operating a machine, a doctor wielding a scalpel — all seem to benefit from the realistic repetition of a VR experience.

Colleges are doing more in this area as well, as with Hamilton’s experiment with a conducting class. At the University of Oklahoma, students in an undergraduate architecture course import their work into a 3-D system that allows them to see how their designs would function in real life. They might discover that a ceiling is too low or a door too narrow. One study by an emerging-technologies team found that a VR assignment improved students’ sense of their ability to carry out spatial-analysis tasks.

Case Western Reserve U.

The more complicated questions raised by extended reality center on the kinds of learning more likely found in a college classroom, such as conceptual understanding and analytical thinking. From his review of the 15 to 20 studies he has read on the topic, Bailenson says, “there’s not a lot of data that shows virtual reality increases learning transfer.”

He posits a couple of reasons. First, the work of getting comfortable with the technology could distract from actual learning. As headsets and other devices become simpler to use, he says, that problem is likely to diminish.

Archaeology Southwest

The bigger challenge is that what makes extended reality appealing may also make it a more complicated teaching tool. For example, in a virtual world, it’s hard to control users’ attention. They could be looking at something to their left, when you, the instructor, want them to see what’s going on to their right.

This problem, too, may also diminish with time, he says, as experts in learning science or content development get more involved in developing the virtual elements of a course. The VR experience of Hamilton’s conducting students illustrates both of these points. In written feedback, several students described how much they liked it. One called it “staggeringly helpful” in preparing him for the experience of live conducting. Others were more critical, detailing the limitations of the technology. Even then, though, they advocated for its use, provided developers could work out those glitches. “It was one of my favorite parts of the course,” one student wrote.

The lack of high-quality content, not problematic technology, is what’s holding extended reality back from being more widely used in the classroom, says Bailenson. In 2013, he notes, a VR headset cost thousands of dollars and was uncomfortable to wear. By 2016 both Oculus Rift and HTC Vive were on the market, making it possible for colleges to purchase more affordable, portable equipment. Meanwhile Microsoft, Oculus, Google, HTC, Hewlett-Packard and other vendors are collaborating with colleges to create a feedback loop: Try out our stuff, show us what you created, and tell us what we can do better.

Bailenson, among others, also thinks that augmented reality — which allows you to see the world around you — will open up far more possibilities for teaching and learning than strapping on a VR headset could. That’s the kind of technology that fits more easily into the classroom, he says, because it can become a shared experience. Think of a group of students gathered around a virtual heart or a brain cell, as their professor takes it apart to show them the components.

Case Western Reserve University, for example, has been working with Microsoft HoloLens augmented-reality glasses for several years. One of their early projects involved the creation of virtual anatomy lessons for students on the health-education campus. Preliminary research indicates that students learned the human musculoskeletal system equally well through the HoloLens or a cadaver lab but that the virtual-body group learned faster.

Unlike VR headsets, augmented-reality glasses allow users to see the world around them, opening up possibilities for shared learning experiences. A group of students could gather around a virtual heart as their professor takes it apart to show them the components. (Microsoft)

Someday academic libraries will collect and store virtual assets, not just physical ones — of sculptures from far-away museums and archaeological sites, ancient skulls and tools, and medical images, says Matthew Cook, head of emerging technologies at the University of Oklahoma Libraries. Working in extended reality, he believes, helps students develop an understanding of their discipline and prepares them for a world that increasingly incorporates immersive technology. “These are the job skills students are going to need,” he says. “They’re entering fields where these technologies are being used.”

Some colleges are going a step further. This fall Shenandoah University, in Virginia, will be among the first institutions to offer undergraduate degrees in virtual-reality design.

Philip Skemer, director of the Fossett Laboratory for Virtual Planetary Exploration at Washington University in St. Louis, sees tremendous potential for virtual reality in undergraduate studies. “Virtually everything we do is inherently 3-D,” he says of earth and planetary sciences. “So if you’re trying to picture a complicated structure formed by geological processes, it can be hard to convey in the classroom.”

Not many colleges have the kinds of resources his lab does, so Skemer plans to push out its work as it’s finished, including through apps in the Microsoft HoloLens store.

He envisions a day when the surface of Mars, the earth’s core, glaciers, ice sheets, and rock formations will all be available for study in three dimensions. “From a teaching and learning perspective, this sounds kind of corny,” he says. “But I think it’s the future of classroom education.”

David Toccafondi

How to Get Started With Extended Reality

Tips from emerging-technology experts and instructional designers

Work with tech and learning-design experts.
Sometimes immersive technology has unexpected consequences, like motion sickness. It may also be hard to figure out what, if anything, a 3-D experience can add to your teaching. Before diving in, discuss these issues with your campus experts.

Look for low-hanging fruit.
Create a few easy projects with 3-D technology, like a virtual tour with a 360-degree camera, a virtual version of a real object, or a simple science experiment.

Involve students.
Students are some of the most creative and enthusiastic users of extended reality. At smaller institutions, which may have fewer experts on staff, students can work alongside faculty members and tech specialists to help design and develop course modules.

Include assessment.
Do learning outcomes improve when an extended-reality module is added to a course? Do students like their experiences with the technology? Measure that as you go to determine whether your investments in technology and course design are paying off.