In astronomy, the object of study is heavenly bodies. Focusing telescopes on the moon, for example, scientists collect data to gain insights into its characteristics. In education, the object of study is student learning. But until digital means of gathering data were introduced with education software — like reaching the dark side of the moon — little data emerged directly from the classroom, the very site where institutional learning occurs. Unless a professor videotapes an on-campus class, what happens there is rarely captured, except perhaps by those students who take a lot of notes. On-campus student data disappear at the end of each class, escaping the process of gathering and measuring what goes on.
In most fields of inquiry, the goal of data collection is to capture good evidence, allowing investigators to respond to questions that have been posed and to make educated guesses about what the future holds. While colleges routinely collect vast amounts of data about course completion, graduation and retention rates, and other measures of student and institutional performance, reliable information about what actually happens in class is essentially missing.
Colleges have been good at looking closely at the past but far less effective at uncovering what’s happening in real time and how students are likely to perform in the future.
Online, however, nearly every action and interaction can be captured. Using learning-analytics software, every moment can be secured, collected, and displayed, open to inspection and analysis. As a field of inquiry, learning analytics emerges from data drawn from course-management systems and other education software that uncover digital evidence generated by students and teachers in virtual classes. Learning-management systems — now commonly used at colleges across the country — routinely track students’ online participation, monitoring discussion-board postings and following students’ access to digital materials, quiz results, and assessments.
Results can predict students’ performance, provide them with personalized learning pathways, and intervene on behalf of students at risk or in need of faculty guidance. Some of the software displays data on dashboards, providing students and instructors with a graphic presentation of findings. Inherently interdisciplinary, learning analytics draws on such established scholarly areas as statistics, data mining, artificial intelligence, social-network analysis, visualization, and machine learning.
Certainly lecture-capture technology is available at many colleges, where cameras record content delivered by instructors, but not student behavior, participation, outcomes, or other data reflecting student learning. On campus, we don’t know if students have read the most recent chapter or how often they watched a video clip. Without substantive data, instructors cannot intervene until tests are graded or papers are read. Even then they have no idea how students are likely to fare on their next exam or how they will do in the course.
In contrast, online instructors have access to a continuous flow of student data. Mary Grush, an editor at Campus Technology magazine, has found three main ways of forecasting how students will perform — how often they log on to the course, how often they read or engage with course materials and practice exercises, and how they do on assignments. After the first week of a course, Ms. Grush said, instructors can predict, with 70-percent accuracy, whether students will complete it. By identifying levels of risk for each student, learning analytics allows instructors, advisers, and support-staff members to move in quickly to help those most at risk.
My colleague John Vivolo has proposed that in order to avoid being overwhelmed by a flood of data, online instructors focus on a narrow pattern of student behavior — say, in a single virtual course — rather than dig through large-scale data sets. He also recommends that teachers get a good idea of learner performance by examining student data in a targeted period, perhaps over a week — or even just a day — exploiting course analytics as a practical tool to provide online student support. He cautions, however, that deciphering student data is not simple; it’s prudent to grasp best practices before you dive in.
Without data to guide them, faculty members can only guess which parts of their classroom instruction are effective. Was last week’s lecture on track? Is this slide too complex? Should the class begin with an overview? Or should they plunge right in? If they feel they’re not getting through, the most common recourse is to wait until next semester, juggling things to fix the problem.
Using digital-learning analytics, faculty members can drive continuous improvement by understanding how students actually navigate through online courses. Data can show which elements students are ignoring or which ones they find puzzling or difficult. Instructors can modify curricula by restructuring content to make it more accessible or hone exams to increase chances of student success.
To give students greater flexibility, matching options with learner styles, it is possible to test outcomes against various delivery modes — text-based documents, audio lectures, slide presentations, video streaming — uncovering which approach might be most effective. Today, course modules can be reassembled, altered, inserted, or deleted with ease. Some instructors give students a smorgasbord of options, allowing them to pick what works for them, letting them decide how they learn best — reading a text, opening a video, clicking through slides, playing a game, joining a virtual discussion, or performing a digital experiment.
A 2014 report by the U.S. Department of Education explained that learning analytics can be used to build models revealing “what a learner knows, what a learner’s behavior and motivation are, what the user experience is like, and how satisfied users are with online learning. … Because these data are gathered in real time, there is a real possibility of continuous improvement via multiple feedback loops.”
A major concern is how faculty members and institutions maintain the confidentiality of student data. Personal information can be disclosed inadvertently or, worse, revealed by design — for example, when sold to commercial vendors without students’ permission. To protect students, federal rules require that colleges remove names, email addresses, and other identifiers from data sets.
As predictive analytics penetrates higher education, colleges must introduce formal policies guaranteeing that students own the rights to their data generated in online classes, that they have the right to correct errors posted in their files, and that they have control over how colleges share their data with others.
Robert Ubell is vice dean for online learning at New York University’s Tandon School of Engineering. This essay is adapted from his book, Going Online: Reflections on Digital Education, available December 16 from Routledge.
