Harvard Last in Free Speech? Don’t Trust FIRE’s Rankings.

In the twisted mess that has characterized debates about higher education in the United States since October 7, one claim is recited over and over again: Harvard ranks dead last in free speech. This has been repeated by countless news outlets, members of Congress from both parties, and pretty much everybody else with a score to settle against Harvard and other elite institutions. That Harvard, perhaps the most visible university in the world, would rank last on something as important as free speech is disconcerting, especially because free speech — including whether it is selectively being applied to protect certain ideologies, while not others — has been central to recent debates about campus climate.

But here’s the problem: The claim that Harvard is last in free speech doesn’t withstand scrutiny. It’s based on an arbitrary and misleading methodology from the free-speech rankings of the Foundation for Individual Rights and Expression, or FIRE. The organization’s methodology only makes sense if, rather than truly measuring the quality of free speech on campuses, you are interested in generating headlines by criticizing America’s most prominent colleges. To put it plainly: The claim that Harvard is last in free speech shouldn’t be trusted.

Before explaining why, let me dispense with a few preliminaries. First, free speech and open inquiry are essential to the mission of higher education. I can’t imagine a university without them. Second, this is not a criticism of FIRE more generally. FIRE has shown an admirable consistency in its support for free speech. (It is also admirably transparent about its methodology and data. Staff members there supplied me with all the data that they used to make their rankings.) Third, I have no interest in defending Harvard because it is my employer. In fact, because academics are natural cynics, you don’t get cool points from other professors by saying nice things about Harvard. Nor is it my intention to give Harvard a pass on issues of campus climate and free speech; on the contrary, I think Harvard has many shortcomings when it comes to free speech and I have not been shy in saying so.

Rather, the reason I think it is important to be clear about the faulty methodology underlying the FIRE rankings is because these rankings have been weaponized by critics of America’s universities as part of an ongoing culture war. But if you remove the arbitrary changes that FIRE makes to its scores, the campus with the worst free speech is Franklin & Marshall College, not Harvard. My guess is that we wouldn’t hear as much about these rankings if it was Franklin & Marshall at the bottom because it wouldn’t serve the same political purpose.

It is important to understand that the rankings are based on a numeric score. Harvard scored a zero, while the top-ranked university, Michigan Technological University, scored 78.01. Like all numeric scores of this type, the score has no natural meaning. To understand how Michigan Tech ended up with 78 more points, we must examine what goes into this score. This numeric score is based on a combination of three ingredients: 1) a survey of students at 254 colleges (there are about 2,600 four-year institutions of higher education in the United States, so Harvard only ranks last, according to FIRE, among the less than 10 percent of schools represented by this survey); this is the main component and is set to have a mean score of 50; 2) “campus indicators” that give bonuses or penalties to colleges for supporting or sanctioning speakers, students, and scholars under criticism for their speech; and 3) an addition or deduction from the score based on FIRE’s Spotlight Database ratings, which assess colleges’ written policies that regulate campus speech.

For the purposes of this critique, I’m going to accept the results of the survey as valid — even though there are some aspects of it that might make one worry about its reliability, most notably the small sample within each college (an average of 217 respondents per institution and as few as 39 at the low end) that is based on an opt-in panel from a commercial polling firm. The small sample sizes and potential bias from an opt-in survey make obtaining representative responses from any single college difficult.

As is common, the survey data is reweighted to attempt to achieve a demographically representative sample of college students in the United States, but this does not ensure representativeness on the particular issues polled in the survey, and it does not ensure representativeness within a single institution, which is a key shortcoming when comparing colleges. Nor is it clear if the survey is capturing attitudes about free speech in the form that many of us imagine as the ideal on a college campus: that is, speech that promotes open inquiry and the free exchange of ideas. After all, based on the survey data alone, the sixth-ranked campus for free speech is Hillsdale College, an institution described in a New York Times article “as much a political and ideological campaign” as it is a college.

But because the survey results are the least arbitrary component of the rankings, I am going to treat those as the most reliable. And here’s the key: When looking at the survey data alone, Harvard isn’t last — in fact, it’s nowhere near last. It ranks 96th out of 254, putting it just outside the top third for free speech. Harvard is largely similar to peer institutions. Looking at the survey-based free speech in the 13 “Ivy-Plus” institutions (the Ivy League plus Stanford and Duke Universities, the University of Chicago, Massachusetts Institute of Technology, and Caltech), eight rank in the top half of surveyed institutions on free speech and only one ranks in the bottom third (the University of Pennsylvania at 236). These high-profile colleges, which have come under attack for supposedly suppressing speech, have an average survey-based rank of 100, meaning they are, on average, better for free speech than most colleges included in FIRE’s survey.

So how did these institutions end up with such abysmal final rankings, with an average ranking of 175th and only three ranking in the top half (Chicago at 13th, Brown University at 69th, and Duke at 124th)? Specifically, how did Harvard land in the bottom spot in the rankings? Because FIRE put a thumb on the scale in a way that penalizes high-profile colleges.

This thumb comes in the form of the arbitrary deductions FIRE applies to the survey data — the second and third ingredients of the scores. These deductions attempt to quantify the “campus climate for free speech” by using media reports to tally incidents of speakers, students, or scholars who were supported (bonus points) or sanctioned (penalties). Colleges have one point added for each incidence of supporting those under criticism for speech and one to three points deducted for sanctions, depending on their severity. The penalties lose a quarter-point of their severity for every year that has passed since the events occurred.

Assuming we accept that FIRE collected and coded these events correctly, there are two problems with including these tallies as ingredients in these scores: First, it’s not at all clear why incidents should be assigned the points that FIRE assigns. For example, why is a speaker who is being supported valued at one point? Is one point the right bonus to reward free-speech culture? In statistics, a standard deviation is a common way to think of the size of a change. If something changes by one standard deviation, the change is usually considered substantial. FIRE sets the standard deviation of the survey scores to be 10 points, so by FIRE’s scoring system, a speaker being supported increases the rating of a campus’s free-speech culture by 1/10th of a standard deviation of free speech.

Why is this? (I’m not even clear how to think about what 1/10th of a standard deviation of free speech means.) Or why is a speaker invitation being revoked a three-point penalty, while a speaker voluntarily withdrawing is only a one-point penalty? Is it three times more harmful to free speech? Why do these penalties decline in severity by a quarter-point every year? Are they 1/40th of a standard deviation less harmful to speech if they happened in the previous calendar year? I could go on.

FIRE makes a similar arbitrary decision when it takes away points for its Spotlight ratings of the written policies around speech. FIRE awards one standard deviation (10 points) for a “green light” rating (the best rating), takes away one standard deviation for a “yellow light” (the middle rating), and takes away two standard deviations (20 points) for a “red light” (the worst rating). This ensures a 30-point difference in the final scores between every “red light” and “green light” institution. Why 30 points? Does 30 points represent a meaningful number in measuring free speech at a college? It’s hard for me to think of an argument for why this would be the case.

Incidentally, the survey responses for the colleges in these different Spotlight categories might also give us pause. FIRE puts six institutions in a special “warning” category because, according to FIRE, they are so bad at speech that they are worse than the “red light” institutions. These are institutions, which include Hillsdale College and Liberty University, that FIRE considers to have policies that prioritize “other values over a commitment to freedom of speech.” But the very same colleges had better results on the survey. The average survey score from these “warning” colleges (57.9), is better than the scores for the red-light colleges (53.9), yellow-light colleges (54.3), and even green-light colleges (54), all of which also have scores that are basically indistinguishable from each other — making one wonder if either the Spotlight ratings, the survey, or both, fail to accurately reflect the speech climate on campus.

The decision to count these incidents and ratings is almost entirely arbitrary; had FIRE made a different decision, the penalties and final rankings would have been different. While there is value in tracking these incidents, the inclusion of these tallies in combination with survey data to craft a numeric score creates a ranking that shouldn’t be treated as anything real.

Moreover, we must consider whether simply counting these incidents teaches us anything about differences in free speech across campuses. Consider first that a simple count of scholars and speakers, rather than a percentage of total scholars and speakers at an institution, penalizes large institutions because they simply have more opportunity for speakers, scholars, and students to be penalized. Michigan Tech, the No. 1 overall college by FIRE’s rankings and number three based on the tally of “campus indicators,” has about 7,000 students. Harvard has over 21,000. Harvard also has more than five times more faculty than Michigan Tech (not to mention the thousands of other scholars affiliated with Harvard in some way). Harvard, I am guessing, also has far more speakers. The opportunity for students, speakers, and scholars to come under criticism is simply greater at Harvard than it is at Michigan Tech or most other institutions, and so a count of these incidents will naturally show more at Harvard, even if there is no baseline difference in protections of free speech.

Now, one may respond that the FIRE rankings also reward colleges when individuals are supported, balancing out the greater opportunity for sanctioning found at larger colleges. But FIRE takes away more points for sanctioning than it adds for supporting, so even if a college supports as many scholars as it sanctions, it will end up in the negative (only 37 colleges in total, or about 15 percent, have a positive campus-indicator score). Does canceling a speaker hurt a campus’s free-speech climate more than supporting a speaker helps it? Perhaps, but it’s not obvious that this is the case. Even more importantly, incidents where a scholar is supported, because the scholar has not been wronged in the final outcome, are probably less likely to make the news, and less likely to be included in FIRE’s database.

As an example, consider that the FIRE database includes incidents from my own department of government at Harvard, including an unfortunate event where students complained about the conservative political views of an instructor, and the instructor was sanctioned. But there were other incidents of students complaining that didn’t make FIRE’s database, including at least one where students unsuccessfully tried to get a conservative scholar removed from his teaching position. Why is this not in the database? Because when the students brought the complaint, the faculty discussed it and dismissed it. That was the end of the matter. There was no story because there was no aggrieved scholar. Arguably, the most robust and successful defenses of free speech — incidents where faculty stand up for free speech and prevail — do not make it into the FIRE database at all.

Because stories about the dog that didn’t bark don’t make the news, we have no idea how often they happen, and so FIRE’s database is skewed in one direction. Moreover, what about the students, speakers, and scholars who use speech with which people politically disagree, but nobody ever complains, so there is no opportunity for sanctioning or support? Isn’t this a sign of a healthy culture of speech? My colleague, Harvey Mansfield, hosted conservative speakers on campus almost every week for decades and nobody, to my knowledge, ever attempted to cancel one of these speakers. In the past month alone, I can count several high-profile events with pro-Israel and pro-Palestinian speakers at Harvard where speakers delivered their remarks to respectful audiences. According to FIRE, Harvard gets no credit for these speakers because nobody tried to cancel them and, as such, there was nobody to sanction or support.

In certain respects, FIRE is penalizing colleges not because they have particularly hostile climates for speech, but because these colleges are places where free speech on controversial issues is most likely to occur. Ranking colleges by simply counting incidents will naturally bias the scores against the most high-profile universities with the largest stages. Well-known (and often controversial) political figures come to Harvard and other high-profile institutions on a regular basis — far more often than they come to many other institutions. The correlation between endowment size and the penalties counted for FIRE’s “campus indicators” is strongly negative, meaning that institutions with larger endowments, and presumably larger profiles, are being penalized more. FIRE says this is because these institutions are bad on free speech — but another, more likely explanation is that more people who have something controversial to say choose to do so at these institutions.

Consider Georgetown and George Washington Universities, institutions that also fit this pattern but not necessarily because of remarkably large endowments. GW and Georgetown, based only on these “campus indicator” penalties, are poor free-speech institutions (ranked 191st and 248th of 254, respectively). Why is this the case? Are they bastions of hostility to speech? Perhaps, but a simpler explanation is that political figures speak at these universities because they are prominent institutions in Washington, D.C., home to the political class. And Washington is home to the news establishment, so when there is an incident, it is more likely to be covered in the news and, thus, discovered by FIRE’s data-collection process.

Moreover, the large endowments at high-profile institutions afford them the opportunity to be the places where controversial issues are likely to be taught, discussed, and debated by their faculty members and students. Harvard, Georgetown, GW, and many other high-profile universities have large public-policy and law schools — the places where such issues are likely to be discussed and debated. Do places with no such schools, such as Michigan Tech, have a better climate of speech? Consider that, on average, universities with public-policy schools, according to FIRE, have worse rankings on “campus indicators” than those that do not. But is this what we want? Rewarding an institution for not addressing controversial issues in the classroom?

Now, it could be that institutions like Harvard attract penalties from FIRE because, as their critics claim, these places just have bad climates for speech and are full of students wanting to cancel speakers, scholars, and other students. But consider that on average, as measured by the student-survey component of FIRE’s rankings, as a college’s endowment becomes larger, its free-speech climate actually improves. This means that, according to the students, these high-profile institutions, including Harvard, are actually more tolerant of dissenting views than institutions that score higher in FIRE’s final rankings.

Harvard’s free-speech climate is far from perfect. Conservatives are sometimes justifiably aggrieved by how they are treated at colleges. And fears about political homogeneity are not unfounded. (I survey the politics of my students every year, and this year in my undergraduate course, for the first time, zero students reported being Republican — a bad sign for Harvard and for the Republican Party). But it’s important to put speech at Harvard and other institutions in context: All people, not just college students, have a free-speech problem. Humans are not naturally inclined to support speech with which we disagree. That’s why free speech is a protected right in the U.S. Constitution. And, in this light, places like Harvard might be considered success stories.

Speech is easy if nobody disagrees. So, speech in perfectly homogeneous places or where nobody says anything controversial is easy. On the other hand, institutions like Harvard are trying to achieve free speech in situations of nearly unprecedented diversity and where controversial issues rise to the fore.

The reason speech sometimes gets challenged is because we are talking about challenging topics. That we do so imperfectly is far better than a situation where challenging topics are simply off the table or where everyone already agrees. At colleges, being challenged is written into the mission and seeking the truth is a shared ideal. Institutions like Harvard often fail to meet these ideals, but ideals are not supposed to be easily attainable. That in such a wonderfully diverse place, with such charged topics on the table, students seem to do relatively well in their self-reported commitment to speech is something we might learn from, even celebrate, rather than find misguided reason to criticize.