Let’s go back to the research paper on screencasting that I first blogged about here. In that post, we saw that students on the study generally watched the screencasts, even without explicit rewards like grades, and the tended to do so strategically. But what about student learning? Did it help?
To answer that question, we have to go back to a previous paper by the authors [PDF]. (That one is in the queue this week to read and blog about.) In that paper, the authors did find a positive correlation between screencast use (which they tracked using stats for the class’ course management system) and overall performance. But - this correlation does not imply causation, and indeed when the data are sliced along various demographic lines, sometimes the students’ performance was better explained by GPA than by screencast use.
I haven’t gotten into that second paper yet, but what I think that means is that among some subgroups, there are students who are high achievers who get good grades and watch a lot of screencasts, because they are high achievers. The grades and the screencast viewing tendencies both stem from the same behavioral source. Makes sense.
However, there’s one situation where the frequency of screencast viewing was significantly correlated with course grade: When you look at screencast usage across academic majors. In particular, in this study, there were students from a variety of engineering disciplines taking this materials science course. Some of the students were coming from engineering majors where they’d seen the course concepts before and were closer thematically to materials science (such as Chemical Engineering). Others didn’t, and in fact one group - Industrial and Organizational Engineering (IOE) - came in as complete newbies to materials science.
The IOE people, prior to screencasts being introduced, had the lowest final grades in the class even though their academic indicators suggested they were on the same level, academically, as the other engineers. After screencasts were introduced, there was a significant positive relationship between that group’s final grade and their screencast usage.
That was in the previous study. In the current one, the authors tracked student performance on a single exam question on polymer structure. Two screencasts were made after a class session on polymer structure in response to student questions and was posted prior to the exam. The authors found a significant correlation for all students between the use of the screencasts and student performance on that question. When the data were broken down among different engineering disciplines, some groups showed positive but not statistically significant correlation, but some did - and the biggest and most significant of all were the IOE students, exactly the group that needed the help the most.
What does this mean? I think it suggests that screencasts, when done well and deployed properly, help all students – they certainly don’t hurt – and they help most thise students who need the most help. The analogies to mathematics courses are clear. In any math course there will be a large contingent of students whose backgrounds aren’t congruous with the course: students whose prior math background is weak to rusty, students from non-STEM disciplines, and so on. For those students, if they use the screencasts, then they may be likely to improve at a surprisingly fast rate.
Finally, let me reiterate this was not a flipped classroom situation in this study. It was a traditional classroom with screencasts made on demand based on what students said was the “muddiest point” in a class meeting. So there was a high degree of student engagement from the beginning with these screencasts -- students asked for them, they watched them voluntarily, and they did so with specific agendas in mind. This makes a good case for using screencasts to augment an existing course that might be quite traditional in its setup.
But what if this had been a flipped classroom, and the screencasts were not homework review but rather the first contact students had with the material, and students were required to watch them before coming to class? What if the element of choice is missing, and what if students don’t approach the videos with a specific to-do list? Would we see similar learning gains when compared to a traditional class (without screencasts)? I suspect we might, but since the flipped classroom is not really about videos but about meaningful classroom experiences, it would be a lot hard to say just exactly what the screencasts are doing for the students. What do you think?