The trouble with Khan Academy

At some point around the beginning of February 2012, David Coffey -- a co-worker of mine in the math department at Grand Valley State University and my faculty mentor during my first year -- mentioned something to me in our weekly mentoring meetings. We were talking about screencasting and the flipped classroom concept, and the conversation got around to Khan Academy. Being a screencaster and flipped classroom person myself, we’d talked about making screencasts more pedagogically sound many times in the past.

That particular day, Dave mentioned this idea about projecting a Khan Academy video onto the screen in a classroom and having three of us sit in front of it, offering snarky critiques -- but with a serious mathematical and pedagogical focus -- in the style of Mystery Science Theater 3000. I told him to sign me up to help, but I got too busy to stay in the loop with it.

It turns out I missed my chance at viral internet stardom, because Dave finally made the video along with John Golden (another GVSU math person):

The video was first picked up by Dan Meyer’s blog, and from there made it to Education Week... then on to Slate, the Chronicle, the Huffington Post, and Wired. The video now has over 11,000 views and has spawned a Mystery Teacher Theatre 2000 contest (with Twitter hashtag #mtt2k). Khan Academy took down the video that Dave and John critiqued and replaced it with two new ones. So this has turned into kind of a big deal.

Another thing it’s spawned is a slew of comments and conversations about Khan Academy. More comments at this point than conversations. And unfortunately many of those comments are uncritical defenses of Khan Academy that often adopt a much nastier tone than John and Dave’s snarkiness from the video. (Just look at the comments below the YouTube video.) It seems like the #MTT2K project/phenomenon has pushed some issues about math education from simmering to boiling -- which I think was Dave and John’s intent. As Dave has explained, the snarkiness of their video may not rub everyone the right way, but Khan Academy has an almost impenetrable veneer of rightness about it that only biting satire could cut through. And they’ve certainly cut through.

I don’t plan on joining in to the #mtt2k contest because my criticisms of Khan Academy are more at the top level than in the specifics of any one video, and I hope my ongoing screencasting work embodies the kind of pedagogical approach I’d like to see video resources take. Some readers might be surprised I have any criticisms at all, since my screencasting is so obviously inspired by Khan; I even once openly wondered if Khan Academy is the future of education. But some criticisms remain, and since the conversation is happening, I thought I’d briefly lay those out.

Let’s start with what Khan Academy is. Khan Academy is a collection of video lectures that give demonstrations of mechanical processes. When it comes to this purpose, KA videos are, on the average, pretty good. Sal Khan is the main reason; he is approachable and has a knack for making mechanical processes seem understandable. Of course, his videos are not perfect. He tends to ramble a lot and get sidetracked; he doesn’t use visuals as effectively as he could; he’s often sloppy and sometimes downright wrong with his math; and he sometimes omits topics from his subjects that really need to be there (LU decomposition in linear algebra, for example). But on balance, KA is a great resource for the niche in which it was designed to work: giving demonstrations of mechanical processes.

But let’s also be honest about what Khan Academy is not. Khan Academy is not a substitute for an actual course of study in mathematics. It is not a substitute for a live teacher. And it is not a coherent curriculum of study that engages students at all the cognitive levels at which they need to be engaged. It’s OK that it’s not these things. We don’t walk into a Mexican restaurant and fault it for not serving spaghetti. I don’t fault Khan Academy for not being a complete educational resource, because it wasn’t designed for that purpose. Again, Khan Academy is a great resource for the niche in which it was designed to work. But when you try to extend it out of that niche -- as Bill Gates and others would very much like to do -- all kinds of things go wrong.

When we say that someone has “learned” a subject, we typically mean that they have shown evidence of mastery not only of basic cognitive processes like factual recall and working mechanical exercises but also higher-level tasks like applying concepts to new problems and judging between two equivalent concepts. A student learning calculus, for instance, needs to demonstrate that s/he can do things like take derivatives of polynomials and use the Chain Rule. But if this is all they can demonstrate, then it’s stretching it to say that the student has “learned calculus”, because calculus is a lot more than just executing mechanical processes correctly and quickly. To say that it is not -- that knowledge of calculus consists in the ability to perform algorithmic processes quickly and accurately -- is to adopt an impoverished definition of the subject that renders a great intellectual pursuit into a collection of party tricks.

Even if the student can solve optimization or related rates problems just like the ones in the book and in the lecture -- but doesn’t know how to start if the optimization or related rates problem does not match their template -- then the student hasn’t really learned calculus. At that point, those “applied” problems are just more mechanical processes. We may say the student has learned about calculus, but when it comes to the uses of the subject that really matter -- applying calculus concepts to ambiguous and/or complex problems, choosing the best of equivalent methods or results, creating models to solve novel problems -- this student’s calculus knowledge is not of much use.

Khan Academy is great for learning about lots of different subjects. But it’s not really adequate for learning those subjects on a level that really makes a difference in the world. Learning at these levels requires more than watching videos (or lectures) and doing exercises. It takes hard work (by both the learner and the instructor), difficult assignments that get students to work at these higher levels, open channels of communication that do not just go one way, and above all a relationship between learner and instructor that engenders trust.

This is not to say that Khan Academy can’t play a useful role in learning calculus or some other subject. I don’t deny that mechanical skill is important for getting to the higher-level cognitive tasks. But mechanical skill is a proper subset of the set of all tasks a student needs to master in order to really learn a subject. And a lecture, when well done, can teach novice learners how to think like expert learners; but in my experience with Khan Academy videos, this isn’t what happens -- the videos are demos on how to finish mathematics exercises, with little modeling of the higher-level thinking skills that are so important for using mathematics in the real world. So the kinds of learning objectives that Khan Academy videos focus on are important -- but they’re not enough. And I’m troubled when people say that it is enough, that Khan Academy videos are great because “they work”, and redefine mathematics to be the study of how to perform hand-calculations and pass mathematics exams.

One last thing for this post. I’m not a Khan Academy hater. I’ve used the videos for a long time -- I think as far back as 2008, and definitely before KA made it big. I’ve assigned KA videos many times and will continue to do so, in the right amounts and the right contexts. And I believe online video is an idea whose time has really come in education. I’m not jealous of, or threatened by, Khan Academy in the slightest. But I’m not an uncritical fan, either, and we need to look at carefully at Khan Academy before we adopt it, whole-cloth, as the future of education.