There’s big talk these days about “big data” in education—looking for patterns of behavior as students click through online classrooms and using the insights to improve instruction. One start-up company that manages online discussion forums for thousands of courses recently performed its first major analysis of behavioral trends among students, and found what its leaders say amounts to advice for instructors.
The company, Piazza, shared the analysis with The Chronicle, without identifying any of the professors or students involved. The data set included online interactions among students and professors in 3,600 courses at 545 colleges and universities over a period of 18 months.
Professors may want to think carefully before giving formal grades for participation in online discussions, the data suggest. When professors required a set number of discussion posts, the number of submissions was higher than in courses where professors left participation up to students. But instructors reported the highest gains in student understanding when discussion was less strictly marked.
“Correlation is not causation, but perhaps this middle way preserves the feeling of altruism common to successful online environments while keeping content quality higher, while grading rigorously turns the online environment into yet another place for students to grind for grades,” wrote Philip Soffer, Piazza’s vice president for operations, in a report on the data.
One of the clearest trends was that students at highly selective universities are far more likely to ask questions anonymously than are students at other institutions. Students at the Massachusetts Institute of Technology were the most likely to use the cloak of anonymity when participating in online discussions for their courses, with 36 percent of comments in courses using Piazza coming anonymously. Students at Arizona State University were the least shy about using their real names in online course discussions, with only 1.7 percent of comments in Piazza forums made anonymously.
“It tends to be nervousness in front of their peers, not in front of their instructor,” said Mr. Soffer in an interview, explaining that the professor is able to see the identities of commenters even when they are anonymous. “I guess the assumption that the other classmates are smarter than them is stronger at those schools,” he added, noting that the finding surprised him. “I would have thought that at big state schools, where you’re thought of as anonymous, there would have more anonymity” in the forums.
In another finding, the practice of asking students to post a comment to introduce themselves correlated with more-robust discussions. When professors asked for introductions, students posted an average of 25 comments per term, compared with 9.5 comments per term in other courses.
Mr. Soffer said he did not have enough evidence to support one of his hypotheses: that there’s such thing as too much class discussion. “One of the questions is, Is it better to have more people participating or not?,” he said, noting that he suspects there is a “tipping point” where too many people talking make it difficult for students to separate signal from noise.
Officials at the company will be looking at that, among other things, as they continue to quietly watch over the millions of interactions happening through its discussion platform.
