It’s a little strange to hear students being referred to as “potential test subjects” by a college president. But when the college in question is online-only and enrolls 80,000 students worldwide, it makes more sense: The possibilities for institutional researchers are just about endless.
That’s why Western Governors University, a nonprofit, online institution based in Salt Lake City, has beefed up its institutional-research office over the past few years and initiated a bevy of projects focused on improving student success.
Colleges want to track students and help them succeed, to find out what works in the classroom, and to measure professors' productivity. Read a special report that unpacks what big data can and can't do.
“Our job here is, How do we best serve every individual student given how they learn, what knowledge they come in with, what pace they learn at?” says Scott D. Pulsipher, who has been WGU’s president for a year. “We’re trying to adapt the learning such that we increase the probability for any type of student to graduate.”
Virtually every student interaction at WGU — collaborating with classmates, using course resources, submitting assignments — is tracked online. The institution, which includes half a dozen state affiliates like WGU Texas and WGU Indiana, begins a new six-month term every month, and 4,000 to 5,000 students enroll in each term. Such a structure is ideal for running large-scale experiments, says Jason Levin, the university’s vice president for institutional research. Since he joined Western Governors in 2012, its institutional-research office has roughly doubled in size, thanks to the addition of several business-intelligence analysts.
Not only is there an unusually rich repository of data for institutional researchers to take advantage of as they try to improve student outcomes, but they can also test hypotheses much more quickly and accelerate the pace of innovation, Mr. Pulsipher says.
“We have visibility into everything that a student is doing,” he says. (A WGU spokesman said officials “are very open with our students about our use of data to help us improve student outcomes and satisfaction,” and added that none of students’ personal information is shared outside of the university.)
At many traditional colleges, terms like “assessments” and “learning outcomes” often draw skepticism because they don’t seem to account for the often-intangible benefits that a college education can provide. Not so at WGU, a competency-based institution where standardized measurements and goals are the university’s bread and butter. That opens up new possibilities for institutional researchers, who can hold certain variables constant while testing tools and interventions to see how they influence students.
WGU has been a pioneer of using competencies, or demonstrated skills, to help students advance toward credentials and degrees; students master skills at their own pace, rather than taking courses over a defined semester-long period. “That’s the notion of competency-based — you keep the standard of the learning constant and you let the time vary,” Mr. Pulsipher says. “We believe every single person can learn and demonstrate proficiency in the standard.”
One focus for the institutional research office at the moment is “learning personas.” Personas — composite profiles of specific types of learners — describe students’ ability and level of motivation. WGU wants to use that information to craft a “learning path” that’s specific to each student. One persona might reflect a student who’s a laggard, and not particularly engaged in courses. Another might reflect a “fader” — a student who tends to start strong but has a difficult time staying focused as each course progresses.
The idea, Mr. Pulsipher says, is to figure out which type of mentor is going to best support a student, which prompts might best keep that student on track, and what course plan would be most effective. Does the student need general-education courses? Or does he or she have the foundational knowledge and study skills to jump right into more advanced work?
Western Governors will begin a pilot program this month to test a tool that makes use of these personas, Mr. Levin says. He and his research team are collaborating with Excelsior College, an online institution based in Albany, N.Y., to run a six-month randomized control experiment, he says.
WGU’s institutional-research office often teams up with scholars at other institutions. What’s unique about WGU is that researchers can easily observe particular aspects of the student experience — for instance, students collaborating with one another in webinars, says Richard W. Patterson, an assistant professor of economics at the United States Military Academy at West Point.
“At traditional universities, you can see grades and outcomes, but no inputs,” Mr. Patterson says. That can make it tough to know which specific interventions or activities helped students get over the finish line.
Mr. Patterson and WGU are working together on a project that involves designing and testing online tools with the goal of retaining students, improving their participation rates in courses, and encouraging them to earn degrees. Mr. Patterson says they are analyzing microdata (did particular tools help students submit assignments on time?) and examining broad-scale responses (did those tools also help students complete the course?).
“We’re able to see if they log on to look at their courses,” Mr. Patterson says, “but we’re also able to see whether they’re reaching those more important benchmarks.”
If students are weak self-regulated learners, Mr. Levin says, one remedy might involve technology-based interventions that would give them a nudge to stay on top of coursework. His staff is working with a team of behavioral economists on a weekly planner that integrates with Google Calendar, so students would have a visual representation of their week ahead and could receive text reminders about study plans, assignments, and tests.
Another recent institutional-research project involved designing and testing a new “Leadership and Communication” curriculum aimed at making students more resilient, Mr. Levin says. In partnership with the Academy for College Excellence, WGU researchers set up three pilot programs and tweaked the course as they figured out which aspects of the curriculum were improving students’ ability to progress on time toward a credential of some kind, he says.
WGU often touts its high proportion of graduates who go on to obtain jobs in their respective fields, as well as their salaries and their general well-being. That’s all thanks to the institutional-research office, which works with companies like Gallup and PayScale to, as Mr. Levin puts it, “get a broad sense of how our alumni are doing professionally and personally in their lives, compared to students from other institutions.”
WGU and Gallup recently published a study that found that the employment rate of WGU alumni five years after graduation was higher than the national average. It also found that 73 percent of alumni said their educational experience was worth the cost, compared with a national average of 38 percent.
In the future, Mr. Pulsipher would like the institutional-research office to delve into the roadblocks that may prevent adult students from obtaining credentials and degrees.
Mr. Pulsipher also believes that traditional colleges can learn some lessons from the way WGU uses data to adapt the education it offers. For instance, why leave some students floundering in lecture halls if you have data suggesting that they might not learn well that way? he asks.
At many colleges, he says, “it’s kind of like, You’re admitted and we’re done.”
