Want to Improve Technology on Your Campus? Ask Users What They Want

W. Rance Cleaveland was due to start his class in minutes, and nothing was working. He had been a professor of computer science at the University of Maryland at College Park for a decade, but he was teaching a course in an unfamiliar building and couldn’t turn on the classroom projector through the touch-screen controls mounted in the wall. “It would look like I was clicking on things, but nothing was happening,” he says. He wound up writing out the text of his slides on a whiteboard.

He was not the first Maryland professor to have trouble with the panels controlling the projectors found in almost every classroom on the College Park campus, nor was he the last. Based on 20-year-old software, the control panels flummoxed even the instructors who used them regularly. As at many universities, Maryland’s ambitions to infuse technology in its teaching had outstripped aging infrastructure that had been updated and patched over time. But it had never been rethought.

To improve the panels, the university’s Division of Information Technology tried something that remains somewhat radical in higher education: asking users what they want.

The effort started with Marcio A. Oliveira, the university’s assistant vice president for academic technology and innovation. As an early adopter of classroom technology in his previous role as a research assistant professor of kinesiology, he would often be summoned by colleagues who were having trouble lowering projection screens or getting the sound to work in their classrooms. It happened with such regularity that “I was, myself, frustrated,” he says. “I thought, This is not right.”

When Mr. Oliveira was promoted to assistant vice president for academic technology and innovation at the university three years ago, he put rethinking the control panels first on his to-do list. But rather than shopping for off-the-shelf software from an existing vendor, as colleges usually do, he created a team of administrators and technical staff to redesign the panels from scratch. He then invited faculty and other users to test a prototype version to look for bugs and to solicit other feedback. User testing is common in the development of commercial products, but “it pretty much hasn’t been done” by colleges developing their own tech, says Jeff Seaman, co-director of the Babson Survey Research Group and an expert on information technology in higher education.

Mr. Oliveira was convinced that such a process was needed at Maryland to make classroom technology easy for faculty to use. “If someone has a question about how it works,” he says, “it’s a bad design.”

‘Enticing and Intimidating’

The classroom control panels at Maryland were state of the art about two decades ago. Since then, teaching technology has evolved and expanded. As technicians at Maryland swapped out VHS decks for Blu-ray players and added cameras and other gadgets, they changed and added buttons on the touch panels, but the ungainly functionality remained the same. Francis G. Bass, manager of classroom technology design at the university, has worked at Maryland for 18 years and says that the recent system overhaul is its first.

The old panels made little intuitive sense to the average faculty member. When classes changed over, professors entering a room would see a prompt asking them if they wanted to “keep current audio.” They often pressed “yes” automatically, said Mr. Oliveira, but it would prevent them from switching to a new audio source, like their laptops. The panel offered “simplified” or “manual” options for setting up the screens and projectors. The latter was complicated, but the former was inflexible, lowering all screens, for example. Cameras, which professors might use to record their lectures, were identified inconsistently on the panels. The cameras were labeled in some classrooms by numbers, and in others by “left” and “right” — but not by “stage left” and “stage right.” The controls for panning a classroom camera were laid out so that it was easy to switch it off by accident.

Professors who wanted to augment their teaching with video or other classroom tech had to wrestle with the panels or go without. “Technology is both enticing and intimidating for a lot of people,” says Helene Cohen, a professor of education and the executive director of the Office of Innovative Technology and Partnerships at Maryland. Her colleagues wanted to use technology to help better educate their students, “but they didn’t want to be in a position of struggling or having to do a workaround. They wanted it to be seamless. Especially in front of their students.”

The redesign team came up with a simplified version of the panels that got rid of the most common hang-ups, then invited a group of faculty members from different disciplines, and with different levels of seniority and tech savvy, to try to perform specific tasks. The learning began immediately.

One user had trouble figuring out which cable to plug into a laptop. So now the cables are identified by colored tags that have corresponding buttons on the control panel. Users tended to press buttons more than once if they weren’t sure whether a command had been received, overloading the software with multiple requests. Now an hourglass symbol lets them know the system is processing. Testers who were colorblind or had poor eyesight offered advice on color choices and font size. One user had shaky hands, and so the buttons of a subsequent version of the panel were placed farther apart.

Some improvements were no-brainers. The redesigned panel features an on-screen button to contact tech support, in place of paper cheat sheets or a help-line number taped to the podium. Ms. Cohen, who was a test user, says she remembers “seeing that on the screen and going, Oh, well, that’s obvious.”

But user feedback also led to some less obvious improvements. One professor asked if the panel could include a 911 button. “That was pretty telling to us,” Mr. Oliveira says, because it gave a glimpse of “what faculty need in the classroom that they can think about but we can’t anticipate.” The function is still being developed, but will be included in a future update of the panel software.

Changing Minds

The new panels are still being rolled out in classrooms at Maryland, but the process that led to their creation has already had important effects. Tech staff, administrators, and faculty members worked together toward a common cause — improving teaching.

The first round of feedback about the panels generated such detailed tweaks to the look and function of the panels that it was “borderline irritating” to the technicians who worked on the project, Mr. Oliveira says. "’I can’t believe someone’s asking that.’ But that’s what they were asking.” Now that the process is over and the panels complete, he says, “the learning team, even the technicians, really understand that their role here is to serve the faculty. That changed the mind-set a lot.”

The project also helped erode institutional silos, according to Megan C. Masters, director of academic-technology experience and a member of the redesign team, especially between the information-technology staff and the faculty. Even within an environment full of people with advanced degrees and deep knowledge, “people are used to working within their own contexts, and their own domains, and over time I think that leads to these assumptions that everyone else understands what you’re talking about.” Starting a conversation between designers and users led to a simpler and more useful teaching tool, and a better understanding of each side’s perspective.

The real test going forward, according to Mr. Seaman, the information-technology expert, will be not only whether the panels work better for the instructors already using them, but whether they work for those who aren’t. “How many of them are convinced that this is actually useful, or that it’s removed enough of that barrier to entry that they’re actually willing to try it?” he says.

Mr. Bass, the classroom-technology designer, says he’s hopeful that if the technicians he manages aren’t constantly answering faculty distress calls, they can spend more time focusing on other projects. Ms. Masters is already turning to the same user-testing approach on a new campuswide technology project: creating e-portfolios that are both easy to use and do a good job of allowing students to demonstrate their learning to prospective employers. “We’re trying to make sure that we’re implementing technology in a way that is aimed at the entire distribution,” she says, “not just those that happen to have high-level digital literacy.”

Lee Gardner writes about the management of colleges and universities, higher-education marketing, and other topics. Follow him on Twitter @_lee_g, or email him at lee.gardner@chronicle.com.