San Jose State University’s experiment with online video lectures featuring professors at the Massachusetts Institute of Technology—by way of edX, the nonprofit provider of massive open online courses—produced some promising early results. In the fall of 2012, students in two traditional sections of an introductory electrical-engineering course earned passing grades at rates of 57 percent and 74 percent, respectively. In an experimental third section, which was “flipped” to incorporate the MIT videos, the pass rate was 95 percent.
So what’s happened since? San Jose State has remained in the spotlight, but interest in the outcomes of a second and a third trial has taken a back seat to big-picture battles over the role of outside content providers in technology-intensive classrooms.
The university has not released data from last year’s experiments with the MIT content. But slides from a presentation that edX’s president, Anant Agarwal, gave to edX members at a private conference in November showed the outcome of the second trial, which happened in the spring of 2013, edX said.
The spring trial also involved three sections of the introductory electrical-engineering course, one of which used edX content. In the traditional sections, students passed at rates of 79 percent and 82 percent, according to the slides. In the experimental section, the pass rate was 87 percent.
The experimental section hewed much more closely to the MIT professors’ syllabus in the spring of 2013 than it had in the fall of 2012. Instead of using the edX videos only when they complemented his own syllabus, Khosrow Ghadiri, the adjunct instructor who taught that section, adopted the entire edX course.
“We adopted the content of MIT, which covered more material,” Mr. Ghadiri told The Chronicle this week. It was sort of like an accelerated version of the traditional San Jose State course, he said.
What to make of the numbers from the spring trial? The pass rates in the traditional sections were higher in the spring than they had been in the fall—79 percent and 82 percent, versus 57 percent and 74 percent—but that could have been simple statistical variation. In both trials, the samples were quite small. And, as before, the effects of “flipping” the classroom to include more collaboration with instructors and classmates cannot be separated from the effects of using the edX platform or the MIT lectures.
There could also have been selection bias. In the spring, the university’s course catalog distinguished between the traditional and experimental sections. In the fall, no difference had been mentioned in the course listings. Spring students may have opted into the section they thought would suit them best.
Not all students noticed those distinctions, though. Mr. Ghadiri said that 11 students had stopped showing up for class once they realized that they had signed up for the MIT version of the course. They all failed the course as a result. Without them, Mr. Ghadiri said, the pass rate in the experimental section would have been the same in the spring as it had been in the fall—95 percent.
In any case, Mr. Ghadiri said, the pass rates of the spring-2013 trial should not be compared with those of the fall-2012 trial. Why? Because the students learned different material and took different examinations. In the fall, the instructor used the MIT content to help teach his own syllabus. In the spring, he used the MIT professor’s content and the MIT professor’s learning objectives. “It’s no longer apples to apples,” said Mr. Ghadiri.
Last fall San Jose State used edX content in two out of the three course sections, he said. The pass rates were 95 and 96 percent, according to Mr. Ghadiri.
The most controversial part of San Jose State’s experiment with edX has never been MOOCs, per se, but the role that content developed by outsiders—such as video lectures and self-grading quizzes—should play in blended online courses taught for credit by local instructors at places like San Jose State. If the role of a local instructor is to teach a more famous professor’s course, does that reduce the local instructor to a widget?
As he has for the past year, Mr. Ghadiri continues to argue that local instructors have no cause for concern. Even as he has yielded the content and learning objectives of his electrical-engineering course to the MIT professors, Mr. Ghadiri said he and his colleagues had developed a homegrown system for generating and automatically grading quizzes. edX offers those tools, but the San Jose State professors wanted to keep assessment local.
