Albert Einstein often gets credit for words he never spoke, including these: “Not everything that can be counted counts, and not everything that counts can be counted.”
In 1963, the line appeared in the sociologist William Bruce Cameron’s text Informal Sociology: A Casual Introduction to Sociological Thinking. Two contemporary sociologists have now brought Cameron’s intuitive wisdom to life. In their new book, Engines of Anxiety: Academic Rankings, Reputation, and Accountability, Wendy Nelson Espeland, of Northwestern University, and Michael Sauder, of the University of Iowa, have added welcome scholarly heft to widespread anecdotal evidence that U.S. News & World Report rankings undermine sound decision making and encourage destructive societal behavior.
Defenders of the rankings argue that they improve transparency and accountability. The authors suggest a more problematic impact: Reducing any institution to a single and supposedly objective numerical slot masks subjectivity inherent in the methodology. Even worse, rankings create incentives that raise profound ethical issues. Espeland and Sauder prove their argument with a case study focused on the leading edge of higher education’s problems: law schools. Deans, professors, and prospective law students should pay close attention. But if past is prologue, most of them won’t.
Who created this mess?
“When Mort Zuckerman acquired U.S. News & World Report in 1984 and became its editor,” the authors write, “it was a lackluster news weekly overshadowed by its more successful rivals, Newsweek and Time.” Zuckerman — a Canadian with a law degree from McGill and master’s of law from Harvard — pursued a business career more consistent with his M.B.A. from Wharton. He decided to “expand the rankings and issue them annually as a way to solidify USN’s reputation as the magazine providing ‘news you can use.’”
The nuts-and-bolts task of developing a rankings methodology fell to Robert Morse. A U.S. News employee since 1976, he was and is strictly a numbers guy. Today, as chief data strategist, he “presides over USN’s rankings empire from a small office in a corner of the magazine’s Washington headquarters.”
The nagging question is how Morse’s arbitrary system achieved its profound and enduring influence. To find the answer, the authors examine the 1980s, when Ronald Reagan and Margaret Thatcher shaped a political landscape that embraced “market logic” for everything. Variable return on a person’s education turned on a college’s reputation or “brand”; prospective students became customers for whom institutions of higher learning competed. The authors sum up the viewpoint: “Colleges and universities are like businesses that can be improved by hiring administrators who adopt business practices and act like CEOs.”
Deans feel trapped in a counterproductive system. They’re riddled with anxiety and despise the impact of the publication on their institutions.
Among those business practices is an affinity for metrics, one of which is a ranking. Crucial decisions turn on a number, however suspect it may be. For example, 40 percent of the basis for a law school’s U.S. News ranking comes from a reputation score based on dubious surveys and thin response rates from a small group of academics who know little or nothing about the vast majority of schools they’re assessing. Even more problematic are rankings criteria that produce law-school gaming, manipulation, and outright deceit.
The authors note the extreme cases of Villanova University and the University of Illinois, which sought to enhance their selectivity component by submitting fraudulent LSAT scores. Most schools use subtler techniques — recruiting transfer students whose LSATs don’t count in the rankings but whose tuition goes straight into the bottom line, or encouraging applications from unqualified students to reduce the school’s acceptance rate.
As a law school’s mission becomes muddied or even lost, society pays the price. Part of that price is a loss of diversity. To attract students with high LSAT scores and thereby enhance the school’s ranking, law schools offer an increasing number of “academic scholarships.” But buying LSAT scores leaves less money for aspiring lawyers who really need it. From 2005 to 2010, the number of law students receiving exclusively need-based scholarships declined by more than 3,000 (from 20,781 to 17,610), while the number receiving non-need-based scholarships increased by more than 8,500 (from 31,265 to 39,845). The dollar amounts are even more staggering. During the same period, non-need-based scholarships went up by $230 million, from $290 million to $520 million. Need-based aid increased by only one-tenth that amount: $23 million (from $120 million to $143 million).
Likewise, the U.S. News methodology rewards a school’s expenditures without regard to the value added to its students’ education. Meanwhile, tuition soars, as does law students’ debt, which can reach six digits. To enhance reported placement rates, career counselors divert time away from networking and toward tracking down the status of their most recent graduates. Meanwhile, within 10 months of receiving their J.D.s, only half of law-school grads are finding full-time long-term jobs requiring bar passage.
Why does the rankings system endure? One reason is consumers’ willingness to use it. Life is simpler when a ranking — or any metric, for that matter — provides the answer to otherwise difficult questions. The number makes independent thought and judgment seem superfluous.
“I have a friend who is waiting for the U.S. News rankings to decide where to send his acceptance deposit,” a student in my advanced undergraduate pre-law course told me a few years ago. “If he gets into a school ranked number 22 and a school ranked number 24, he’s going to number 22. Period.”
Years of interviews have armed Espeland and Sauder with a plethora of similar stories. And the book’s second half describes the impact of the U.S. News rankings methodology on deans and university administrators. Deans feel trapped in a counterproductive system. They’re riddled with rankings anxiety and despise the impact of the publication on their institutions. But their fear of challenging a dysfunctional regime is overpowering. Any single mover opting out of the system risks a severe penalty — fewer applicants — after plummeting into the dreaded “not ranked” category. That translates into lost revenue, budget cuts, and public humiliation. Still, as the authors note, dental schools successfully boycotted and thereby thwarted a rankings incursion in the late 1990s.
What will end the madness? Engines of Anxiety is a good start. It exposes the moral and ethical judgments that underlie a flawed and destructive system. Rankings can be useful, but as the authors observe, “useful is not the same as good.”
The final chapter hints at what should be the subject of continuing scholarly inquiry: “Measures produce unintended consequences, misguided incentives, and misplaced attention. More generally, they transform what they purport to only reflect, altering the social world in unexpected ways. They are constitutive as well as reactive. … They carry with them assumptions about value, merit, and goals, as well as what is good, normal, and right.”
Steven J. Harper is an adjunct professor of law at Northwestern University and a former litigator with Kirkland & Ellis LLP. His most recent book is The Lawyer Bubble: A Profession in Crisis, (Basic Books, paperback edition with new afterword, 2016).
