San Jose State U. Puts MOOC Project With Udacity on Hold

[Updated (7/19/2013, 12:13 a.m.) with additional material on the trial courses and analysis of the news.]

After two semesters of experimentation, San Jose State University has decided to take a “breather” in a project aimed at determining whether Udacity, a Silicon Valley-based company that specializes in massive open online courses, can help the university deliver credit-bearing online courses.

The university made waves in January, when it announced that it would be developing credit-bearing online courses with the MOOC company. Now, after offering Udacity-powered courses on a trial basis this spring and summer, San Jose State has decided not to offer any courses with Udacity in September.

News of the decision was first reported by Inside Higher Ed, an online news service.

Ellen Junn, provost of San Jose State, described the break as temporary. The university has not suspended its partnership with Udacity, she said, and the university and the company will probably create new courses together by next spring.

“This is a natural breather time, that’s all we’re doing,” she said.

News of the break coincided with the leaking of a slide show containing preliminary data on the spring trials, which included three mathematics courses that San Jose State instructors built with Udacity. The courses were offered to a mix of students, some who were enrolled at the university and others who were not, including some high-school students.

The pass rates for the San Jose State students in those courses ranged from 29 percent to 51 percent. For nonenrolled students, the range was 12 percent to 45 percent.

Data in the slide show, which Ms. Junn presented last month to the California State University Academic Council, are marked as “VERY general, preliminary, and not yet confirmed for accuracy.” The presentation was never meant to be made public, the provost said. The California Faculty Association, a union that has been vocal in its criticism of the Udacity partnership, leaked the slides this week to Inside Higher Ed and The Chronicle.

‘A Joint Decision’

Meanwhile, San Jose State has been working with an outside research group on a more-thorough report describing early results of its Udacity partnership on a grant from the National Science Foundation. The university expects to release that report early next month.

Asked whether it was the preliminary findings from the spring trials that had prompted the university to take a “breather” from its experiments with Udacity, Ms. Junn said that the early data “did not necessarily cause this to happen” and that no planned courses had been canceled. She said she does not recall when the decision to put the trials on hold was made, only that “it was a joint decision” with Udacity and that many factors went into it.

Sebastian Thrun, the founder of Udacity, said the company was merely taking time to restructure its courses with San Jose State so that students could work through the material more at their own pace. “The No. 1 complaint we’re getting is that students need more time, they feel rushed,” said Mr. Thrun. “We never made the decision to stop” the pilot entirely, he added.

San Jose State has been ground zero for several projects aimed at testing how MOOCs—or, more precisely, technology developed by MOOC providers—might help traditional universities improve their online courses while lowering the cost to students. In a separate pilot, the university has pushed its instructors to use content and technology from edX, a nonprofit MOOC provider, in their courses, an experiment that has garnered both promising data and skepticism from faculty members.

But while one San Jose State professor likened using the edX materials to adopting a new textbook, the Udacity pilot has university instructors teaming up with Udacity designers to build new online courses, resembling the company’s MOOCs, from scratch.

‘Not Bad’

Sandra Desousa, a lecturer in math at the university, was one of those instructors. Ms. Desousa has taught developmental math in a traditional format for seven years. This spring she and Susan McClory, another math lecturer, taught an adapted version of that course on the Udacity platform.

The instructors played to two audiences: 3,500 learners who had signed up for the course as a MOOC through Udacity, and just under 100 others who were taking the course for credit, about half of whom were enrolled at San Jose State. The two groups watched the same video demonstrations and completed the same routine assignments. The only difference was that the students taking the course for credit also were required to complete three online examinations, with monitoring by ProctorU, the online proctoring company.

Only 29 percent of the San Jose State students in the credit-bearing section passed the course, along with 12 percent of the non-matriculated students, according to Ms. Junn’s slide show.

Those rates might seem quite low, but Ms. Desousa said they were “not bad” for her developmental math course, especially given the online format. Every university-enrolled student taking the course had already failed the traditional version once, she said.

Other factors might have affected pass rates as well. For example, a number of students in the non-matriculated group, from a charter high school in Oakland, Calif., did not have sufficient computer and Internet access to use the Udacity platform until several weeks into the semester.

All things considered, said Ms. Desousa, “I would say, for a pilot, it was very successful.”

Given the small sample sizes and possible confounding variables, the early findings cannot definitively answer the question of whether MOOC companies can help San Jose State offer better or less-expensive online courses.

Still, data from the spring trials can serve as a useful reminder that the outside providers cannot provide a miracle cure for the problems facing public higher education in California, said Lillian Taiz, president of the California Faculty Association.

“The problem is all the hype,” she said. “I think what these data do is bring us down to earth.”