Is Political Bias in Grading a Myth?

A common criticism of academe holds that conservative college students feel uncomfortable expressing themselves, and may self-censor for fear that liberal instructors will knock them on grades or evaluations.

But do instructors actually do that?

In a Twitter thread he called “The Myth of Partisan Grading,” Jeffrey A. Sachs, a lecturer in the department of politics at Acadia University, in Nova Scotia, posted studies suggesting political bias doesn’t change how instructors grade their students.

But his argument was nuanced: He agrees that the data also show that conservative students feel less comfortable than liberals sharing their views on campus.

Sachs’s post, almost immediately, spawned a flurry of threads and counterthreads. Studies on the subject, and on how political beliefs affect teaching more broadly, remain hotly controversial, and scholars on both sides of the question often dispute their methods.

The Myth of Partisan Grading!

There’s been a lot of talk lately about self-censorship on campus. According to one popular theory, conservative students censor themselves because they believe that if they state their true views, they will get a lower grade.

This theory is true.

— Jeffrey Sachs (@JeffreyASachs) February 12, 2019

The debate raises questions about the influence of broader political polarization on college campuses. While most students, conservative and liberal, report feeling comfortable sharing their opinions in class, very conservative students are less comfortable than very liberal ones by a margin of 14 percentage points, according to surveys by the Foundation for Individual Rights in Education, the free-speech group. That gap grows when students leave the classroom and enter the campus.

Conservative students are also more likely to say they’ve stopped themselves from sharing an opinion in class, that report found, for fear that they would offend their classmates or that it might hurt their grade.

To some critics, that gap in students’ perceptions is in line with a body of evidence showing bias in other parts of the academy — say, in peer review, or liberal professors’ acknowledgment that they’d discriminate against conservative peers. That can naturally filter down to a professor’s relationship with students, they say.

It is impossible to debunk all these sorts of ridiculous claims, because they are often ridiculous in subtle ways. In this thread, I merely debunk one of the articles linked to support this entirely unjustified statement (the “they’re wrong” statement). https://t.co/eOn0Zr10aS

— Lee Jussim (@PsychRabble) February 12, 2019

One reason for the controversy around grades is that there aren’t that many experts on this subject, said Paul Musgrave, an assistant professor of political science at the University of Massachusetts at Amherst.

“In general, we need to know more about this,” he added. “And that goes beyond the normal ‘more research is needed.’”

The Challenges of Studying Grading Bias

Musgrave helped study partisan grading as a graduate student at Georgetown University. If faculty members’ political inclinations do show up, he said, it’s clearest outside the classroom. He and Mark Carl Rom, an associate professor of government and public policy at Georgetown, wanted to see “whether this was something that was spilling over into the classroom.”

Here’s how they did it: Rom had introductory political-science students write essays describing the characteristics of the Democratic and Republican parties. Then he and Musgrave sent the essays to teaching assistants at other colleges for anonymous grading. They surveyed both the students’ and the TAs’ partisan and ideological leanings.

They found that while Democratic students scored higher, on average, than Republican ones, it wasn’t because they matched the grading TA’s party affiliation. A TA’s “nonpolitical” traits, like gender, experience level, and type of institution, were more likely to affect how he or she graded.

Sachs pointed to Musgrave and Rom’s research in his thread. Critics quickly jumped aboard: How useful can this study be? The essay gave students a descriptive prompt, not a normative one that might better expose students’ beliefs to grader bias, said Orin S. Kerr, a law professor at the University of Southern California and a member of Checks and Balances, a conservative lawyers’ group.

That limit was partly by design, the authors wrote. Choosing “the most objective question we could think of” showed that, at the very least, critics’ claims about the state of academe were not as dire as they seemed. “This study has not exonerated academia,” Musgrave and Rom wrote, “but it has shown that the extent of bias can at least be delimited.”

More recently, Musgrave pointed out, scholars have studied another venue for students’ unfiltered thoughts about professor bias: Twitter. After analyzing students’ tweets about their instructors’ ideology, researchers found that 42 percent simply vented about it: “My professor is such a liberal it honestly makes me want to drop the class,” for one, or, “My professor is a hardcore republican please save me.” Students tweeted disproportionately about conservative professors, though more data are needed, the authors wrote.

A smaller share, 11 percent, worried about grade bias: “I hate when I have to write a paper that agrees with my professor’s shitty political views to ensure that I get an A.” And only 15 percent of those students believed their instructor actually lowered their grades because of different beliefs.

Other studies have shown partisan bias in grading — but, one 2012 study reported, in the other direction: Republican professors graded more harshly based not on a student’s party, but on his or her race.

Divided Campuses

That’s not to say that scholars on the left and right don’t worry about political polarization on campus. College freshmen in 2017 were more politically divided than they’d been in half a century, according to the Higher Education Research Institute at the University of California at Los Angeles. That divergence reflects a gender and education gap among white voters, too, which split further in the 2016 election. Whether or not students’ politics directly influence grades, they’re clearly changing perceptions on campus.

“Even though this is framed as conservative students versus faculty,” Musgrave said, “I think the much bigger part here of what conservative students might be reporting doesn’t come from faculty but from other students.”

“It’s common for conservative students at predominantly liberal campuses to feel that it’s better for them to be quiet,” especially in class, said Kerr, the law professor. That’s a separate issue from how students might express themselves in assignments, he added. “We need good studies on this.”

To Musgrave and Sachs, the body of literature suggests little evidence for political bias in grading. A 2017 paper put it simply: “In short, no empirical research has shown a systematic relationship between instructor ideology and any student outcome,” such as grades.

But an instructor’s ideology can affect students’ perceptions of the course, the paper continued, in cases like end-of-course student evaluations. Kerr, Musgrave, and Sachs all agree that such perceptions constitute a serious issue.

Even before Musgrave and Rom completed their study on bias in grading, they gave suggestions for how to limit its potential: Faculty members dealing with political subjects can use “paired assignments,” in which students explore two sides of an issue. Then they’re graded on their mastery of arguments and use of evidence. And learning-management systems offer a means of anonymized grading, he added.

With those added safeguards, “you’d have to be really motivated as an instructor to search out and destroy opposing viewpoints,” Musgrave said. The most important thing — especially considering students’ reported perceptions — is designing assignments so that students know instructors are assessing their abilities, not their beliefs, he said.

The scholars also agree that, as the rest of the country polarizes, these questions will only get more important at colleges. But the instructor’s job remains the same, Sachs said.

“Fundamentally, our job is to educate these students … to the best of our ability,” he said. While he feels there’s “hysteria” on the topic from the right, Sachs also feels liberals can do more to make conservative students feel they can express themselves openly. That, he said, is simply a professor’s duty.

Follow Steven Johnson on Twitter at @stetyjohn, or email him at steve.johnson@chronicle.com.