Anaheim, Calif. — One way to improve peer grading in MOOCs could be to let students grade their peers who graded them.
That’s what a team of writing instructors at Ohio State University decided last spring when they were designing a massive open online course on rhetorical composition, known as WExMOOC.
They built a custom peer-grading system designed to assess not only the quality of the essays submitted by their MOOC students, but also the quality of the feedback that other students in the course contributed after reading the essays.
“We understand writing as an exchange,” said Evonne Kay Halasek, an associate professor of English at Ohio State, during a presentation here on Wednesday at the Educause conference, the annual summit of higher-ed technologists.
“We wanted the opportunity for the writers of the piece that received the peer reviews to be able to, in effect, continue that exchange by commenting back with ‘helpfulness’ scores to their peers,” she said.
One of the great unknowns about MOOCs is whether professors can use them to teach students how to write. Peer review may offer an answer, but the quality of peer-grading systems depends on whether students are getting constructive feedback from peers.
That can present a challenge, especially in a course such as WExMOOC, which initially drew 37,000 students, only 37 percent of whom listed English as their first language.
“We recognized that the peer-review systems, in Coursera in particular, very often led to disgruntled writers,” said Ms. Halasek. She said the writers “had all kinds of problematic assumptions about the abilities of peer reviewers whose first language wasn’t English.”
Yet even the most talented essay writers were sometimes lousy at assessing the work of others. The Ohio State instructors found no significant correlation between the students who received high marks from their peers on essays and those who won praise for their helpful feedback.
Ultimately, the instructors decided to award certificates of distinction not only to students who got high scores on their essays, but also to those who scored well on feedback.