Wednesday, February 21, 2001

LOGGING IN WITH . . . Barbara B. Lockee What Matters in Judging Distance Teaching? Not How Much It's Like a Classroom Course By DAN CARNEVALE Barbara B. Lockee, an assistant professor of instructional technology at Virginia Tech, is an instructor in a graduate program on how to teach at a distance. The classes are held both face to face and -- not surprisingly -- at a distance. She says that trying to judge the quality of a distance course by comparing it with a traditional course is misguided. Q. What's wrong with comparing distance courses with traditional instruction? A. It's very common whenever a new medium comes out to compare it to the effectiveness of either face-to-face instruction or older media, because we're looking to answer the quality question: Is this as good as or better than other techniques or approaches we have used? The primary assumption, which is flawed, is that the instructional effectiveness of each medium studied is constant across all content and all students. You're lumping all the students together, and you're ignoring their qualities and attributes as well as the qualities and attributes of the content. So by treating students, content, and instructional content as homogenous, we are ignoring some very important variables that we know for a fact do impact learning. Q. What are some of those variables? A. Oh, cognitive styles, learning styles, instructional strategies, different methodologies for teaching particular levels of objectives and different domains. Q. Do you have any specific examples? A. The primary instructional strategy that we utilize in our online master's program for teachers is what we call a creative context. And by that I mean that the students create projects or products that demonstrate their learning, that demonstrate that they have indeed acquired the skills or knowledge or attitudes that we targeted in the design of the instruction. So we have teachers create electronic portfolios -- the teachers are our students -- but we have these teachers creating electronic portfolios that contain the deliverable projects from each course that demonstrates how well they learned the intended outcomes of the course. Q. So the online courses often have completely different types of assignments? A. Right. So for example, [John Burton, a professor of teaching and learning] teaches instructional-multimedia offerings. Instead of him just doing direct instruction, where he describes the process, and stopping there and giving, say, a multiple-choice quiz on it, he will actually have them create a project that demonstrates that they learned how to create an effective instructional program. They work through the entire process and demonstrate at each step of the way that they have acquired the skills in designing and developing a instructional multimedia. Q. Can you assume that the end result of what the students learn will be the same either way? A. Yes. The delivery mode we know for a fact does not impact the learning. It's the design of the instruction that impacts the learning, and also what the students bring to the instructional situation. Instead of comparing, say, our online multimedia-authoring course to the face-to-face course, we would look to see that our distance learners are achieving our intended outcomes no matter how they're getting it. That's the question that we need to answer, not comparing the two. But are they learning what we intended for them to learn? Q. So what are the typical flaws when people compare distance with traditional education? A. What happens typically in those kinds of research designs, the design itself creates typically no significant difference. The interpretation of that is misapplied. The interpretation is, "Oh look, there's no difference. That means distance-education experiences are equal to face-to-face experiences." We know that's not a fair comparison. What really matters, like I said, is the design of the instruction and what the learner brings and the strategies used and the context used, those are the factors. And there are so many factors, too, that can be analyzed and considered. Learning is a very complex process composed of many variables. The main problem with media-comparison studies is, you're either crediting or blaming the media for learning, when in fact it's not the media at all. It's how the instructional event is put together and what the learner brings to it. Q. Do people incorrectly judge the design or incorrectly judge the result? A. They're looking at the results, and the design by default generates no significant difference if you're doing a statistical test on it. And they're interpreting that to mean that distance is just as good as face to face. While we know that the delivery mode doesn't impact the learning, it's the comparison that's flawed. It's an oversimplification of what contributes to learning. Q. So a distance-learning course could be worse than a face-to-face one even if the results are statistically the same? A. Yes. It's a flawed test. It's just a bad question to start with. So a bad question leads to a bad answer, which is misinterpreted and misapplied. Q. What should people look at when they evaluate a course? A. We have a tendency to look at learning outcomes -- which is good because, I mean, that's our job. But we also need to be sure that we look at student attitudes because learners at a distance can feel disconnected from the learning community. And there's strategies that instructors can use to enhance the feeling of connectiveness. Oftentimes if we neglect that in our teaching they might drop out -- especially in distance teaching. But if we neglect to look at those attitude factors in evaluation, we won't find out how they feel. And we know it's important to keep students involved and to reduce our attrition rates in distance education. Easy-to-print version E-mail this article

Coalition calls for sharp increase in U.S. education spending U.S. education secretary urges colleges to play a part in improving nation's schools Diversity question on college's job application amounts to "loyalty oath," group contends University chancellor in Louisiana is arrested for drunken driving Britain aims to raise its global standing in science with $979-million in grants Created at M.I.T., a "Bonsai Kitten" Web site stirs animal lovers' passions What matters in judging distance teaching? Not how much it's like a classroom course