Many college leaders believe that online education can be as effective as the old-fashioned kind while also reducing costs to colleges and students. But for William G. Bowen, faith is not enough.
If higher education is going to change drastically in coming years, Mr. Bowen, a 79-year-old economist and former president of Princeton University, wants to make sure that change is guided by rigorous research.
The research standard that Mr. Bowen wants to apply—randomized experimental trials—is hardly new; researchers in education have been using the method for the better part of a century. But at a time of great fear, excitement, and hyperbole, advocating for such a painstaking approach is, in a way, innovative.
There is much at stake. Prophesies of a gathering wave of “disruption” have made presidents and boards at vulnerable colleges jumpy and suggestible. The fates of many institutions, let alone their missions, appear uncertain. And there is a weighty body of research suggesting that online learning can be just as good as face-to-face.
THE INNOVATOR: William G. Bowen, Ithaka S+R
THE BIG IDEA: Rigorous research experiments can help determine the effectiveness of online education.
But Mr. Bowen believes that past research on online learning has holes. Along with his colleagues at Ithaka S+R, the research arm of the nonprofit Ithaka, which he founded, Mr. Bowen cataloged more than a thousand studies of online higher education. They were not impressed.
“Very few of these studies are relevant to the teaching of undergraduates, and the few that are relevant almost always suffer from serious methodological deficiencies,” he writes in his new book, Higher Education in a Digital Age (Princeton University Press). “The most common problems,” he continues, “are small sample size; inability to control for ubiquitous selection effects; and, on the cost side, the lack of good estimates of likely cost savings in a steady state.”
In other words: We don’t know if online education is the answer to the challenges facing many universities, and we don’t know if it’s not.
Mr. Bowen is no mere gadfly. He led a study last year, using the randomized-trials approach, that compared outcomes for students in partially online statistics courses to those of students in fully classroom-based versions of the same course. Unlike previous studies of the same online tool, an automated tutor developed by Carnegie Mellon’s Open Learning Initiative, Mr. Bowen’s study spanned a half-dozen public university campuses and included students from a range of academic and demographic backgrounds.
Setting up the experiment was a logistical “nightmare,” Mr. Bowen has said, and in the end its findings did not contradict the majority of studies already on the books. In fact, the results suggest that this particular mode of “blended” online teaching does not leave vulnerable students at a disadvantage.
But Mr. Bowen does believe, and others have agreed, that his methodology offers the most definitive affirmation of the “no harm” hypothesis to date. Ithaka is gearing up for another study, using the same approach, to test whether massive open online courses can be effective as part of a traditional college curriculum, at the University System of Maryland. Without investing in rigorous, neutral interrogations of online technologies, Mr. Bowen says, higher education risks falling sway to reactionaries and hype-mongers.
“What’s really important is first to recognize that online learning isn’t any one thing,” he told The Chronicle. “It’s a lot of things. People want to simplify, and sometimes they want to oversimplify, and that’s not wise.”
As MOOC fever has transformed online laggards like Harvard University and the University of Pennsylvania into the vanguard of a new generation of online teaching, Mr. Bowen, who has a lot of credibility with elite institutions, has been invited to address several notable gatherings, including those sponsored by Coursera and edX, two major MOOC providers. He has summarily doused the audiences with cold water. “To ‘disrupt’ or not ‘disrupt’ is not the way to intelligently discuss online learning,” he said last month in a talk to Coursera’s partner universities.
But the former Princeton president has his critics. Carol Twigg, director of the National Center for Academic Transformation, says that randomized experimental trials are tedious and often beside the point. Colleges that wait for perfect evidence risk sinking deeper into a hole, she says, and many colleges do not have that luxury. There can be a fine line between deliberation and inertia.
As for Mr. Bowen’s Ithaka study, “I think it was very thorough,” says Ms. Twigg, “but I don’t think studies convince people to do much.”
Mr. Bowen, for his part, acknowledges the limitations of his approach. In his book, the economist admits that one of the most significant lessons of his research with Ithaka was the difficulty of vetting online technologies using such a thoroughgoing methodology. Thus Mr. Bowen’s skepticism is turned on itself; higher education’s ombudsman begins to gnaw on his own tail.
“Looking ahead, I now think—heresy of heresies!—that the case for using randomized trials should itself be subject to careful cost-benefit analysis,” writes Mr. Bowen in his new book. “Appealing as they are, this may be an instance in which, in at least some cases, ‘the best is the enemy of the good.’”
