A Rankings Revolution? Hardly.

In a recent Chronicle essay, President Leon Botstein of Bard College asks: “Can We Finally Topple the Tyranny of Rankings?” In posing this question, Botstein echoes Colin Diver, the former college president and law-school dean who recently asked, “Could this be the beginning of the end for college rankings?” in The New York Times. Like many others commenting on law schools’ withdrawal from submitting data for the U.S. News & World Report rankings, Botstein and Diver are hopeful that this may be an opportunity to ditch rankings once and for all. They are not alone. Wishing “death to college rankings,” as two Financial Times journalists recently put it, has become a widely shared sentiment in academe and beyond.

The sentiment is hardly new. In fact, unequivocal dismissals of higher-ed rankings has a tradition that predates the U.S. News era. W. Patrick Dolan’s 1976 book, The Ranking Game: The Power of the Academic Elite, criticized academic rankings produced by the American Council on Education in 1966 and 1970. Dolan was critical of the very idea of a ranking of academic institutions and departments. And while he critiqued the way the ACE approached the task, he was especially hard on the administrators, faculty, and other actors in higher ed who enabled and gave legitimacy to the rankings. In addition to his thorough analysis, historical tidbits, and subtle humor, the book is filled with sharp observations that have a lot to offer to the contemporary reader. It feels as if this almost half-a-century-old work could have been written yesterday.

Since November 16, when Yale Law School announced it would stop sending data to U.S. News, another dozen or so law schools — generally among the highest ranked — have followed its lead. Each law school announced this in the form of a public statement, in most cases signed by the dean and published on the law-school website. Interestingly, statements were issued even by Chicago and Cornell — two law schools that announced they would not join in pulling out of the rankings. Reading through all these statements, we get a glimpse into how law schools think about rankings. Although there’s only so much the law schools are willing to say officially and in public, their messages still reveal how law schools feel about U.S. News and about rankings and evaluations more generally.

The law schools that have decided to withdraw all seem to agree about one thing: The current U.S. News rankings methodology does not reflect their values and creates perverse incentives that are not in the best interests of law schools, students, or the legal profession. “Done well,” writes John Manning of Harvard, rankings “could convey accurate, relevant information” that “may help students and families make informed choices.” Similar reasoning was present in other statements. Some schools were more elaborate in laying out the nuances of their preferred methodological changes, some less, but the core message was essentially the same. A number of schools mentioned that they had approached U.S. News in the past, requesting a change in the methodology, without success — hence their decisions to stop cooperating. In short, the message of the Great Rankings Resistance of 2022 is: We, America’s top-ranked law schools, have no issues with being ranked. We have issues with how we are being ranked.

The tradition of ranking academic departments and colleges in the United States goes back more than a hundred years and long predates U.S. News. Throughout the 20th century, scholars, administrators, and academic organizations experimented with ranking departments and institutions for all sorts of reasons. One thing they were especially wary of was any “outsiders” partaking, in particular those not accountable to the institutions being ranked. In 1963, when the National Science Foundation came to the ACE with the idea to produce rankings of graduate departments, a condition of ACE’s approval was that the job be given to an academic organization. In the end, it was ACE itself that did it. The big question was not one of tactics or method, but rather of how to ensure that this, as much as possible, remained an inside job.

U.S. News has always been aware of the importance of having academics and administrators on board with its rankings in one way or another. The 1983 issue of the U.S. News & World Report that contains its first college rankings offers commentary on the “findings” by various college presidents and notable academics. The data for the rankings was collected by inviting more than a thousand presidents to take part in a reputation survey, essentially replicating what the ACE had done earlier. The reputation survey is still there today, performing more or less the same functions for U.S. News that it performed back in the day. First, it brings original (however questionable) data, which is proprietary and can be used for other purposes. Second, like the rankings commentary, it confers legitimacy onto the rankings: U.S. News is not just any outside actor — it has the buy-in of a significant number of insider faculty members and administrators, and thus deserves the industry’s trust.

It is thus not surprising that U.S. News and institutional researchers regularly meet to discuss data, methods, and other practicalities of participating in the rankings. It is also not surprising that many of the new crop of rankings-resistant law-school deans expressed disappointment over a lack of methodological change — but also a hope that U.S. News would reconsider its position. The deans did not spare many words on discussing, let alone criticizing, the very idea of ranking (whether of law schools or universities), or the profound effect the exercise has on how higher ed operates. The “flawed” methodology that these deans highlight has very little to do with any of that.

Paradoxically, the statement by Jens David Ohlin, dean of Cornell Law, makes the most outright critique of rankings. Unlike most of his colleagues, Ohlin did not seem particularly interested in discussing U.S. News’ s methodology, and conveyed a greater skepticism as to whether the whole affair was having its desired impact. In closing, he wrote:

Is it contradictory to dismiss rankings as a distraction while refusing to join others in a collective boycott of the ranking that seems to matter the most? Perhaps. And yet Ohlin’s call for a “deeper and more searching conversation” is exactly what is needed if rankings are to be toppled. As long as those boycotting a ranking do it because they don’t like how a particular ranking is made, instead of for what it is and the kind of systemic effects it creates, the chances that we are about to witness the “death” of rankings are very slim. Based on what the deans of some of the wealthiest and most prestigious law schools have written, they don’t seem to be particularly interested in ending the “tyranny of rankings.”

The law schools’ “rebellion against rankings,” to use Botstein’s phrase, isn’t really a rebellion against rankings. It’s a minor sortie, a handful of superwealthy schools simply not wanting to play ball — for now — with U.S. News. I would not be surprised if this turns out to even strengthen the rankers’ grip on higher ed. After all, there’s no shortage of well-resourced organizations out there prepared and possibly eager to challenge the dominant position of U.S. News. They might happily provide more methodological give-and-take to the institutions they rank. What could possibly go wrong? Decades of research into the effects of rankings and other kinds of metrics on higher ed would say: A lot.

Ohlin is right that a deeper and more searching conversation about rankings is long overdue. That conversation, however, has to be about the effects rankings have had on higher ed, and not about how to make the calculations more acceptable for elite law schools. Only then can we talk about a rebellion against rankings.