Every year a special double issue of Times Higher Education arrives with the mail, bearing its “World University Rankings” pullout segment, colloquially known as the “league tables.” Tearing open the plastic wrapper gives the feel of opening an Oscar envelope. Who will win?
Over the past few years that was a pleasant ritual for those of us teaching at the University of Nottingham, a verdant campus whose most famous alumnus is D.H. Lawrence and where Mohandas Gandhi, Albert Einstein, and H.G. Wells all delivered lectures in the 1920s. More to the point, it’s an institution that has generated such widely acknowledged recent research as The Spirit Level: Why Equality Is Better for Everyone (2010), co-written by Richard Wilkinson, a professor of social epidemiology who recently retired.
That last word indicates the challenge of measuring institutional status, for universities shift and shimmy. Research vaporizes with the emeriti who produced it, and distinguished lectures turn to the dust of ancient history. The waters of reputation require constant replenishment.
For years, none of that seemed much a problem at the University of Nottingham. Its trajectory in the rankings was strictly upward in the three years after I arrived here late in 2009, in what I must admit is a case of happy coincidence, not correlation, and certainly not causation.
The progress was so steady and inexorable as to inspire serenity. In the first academic year after my arrival, 2010-11, Times Higher Education ranked the University of Nottingham No. 174 in the world. Not exactly impressive, but it put us in the world’s top 200. In 2011-12, the newspaper revised its methodology, rolling out a new formula, to fanfare. The University of Nottingham promptly climbed to No. 140, leaping ahead 33 places. In 2012-13, we reached No. 120—a full 20 places better.
Life at the University of Nottingham in those years was like owning a tech stock in 1999, or a collateralized debt obligation in 2006. A 31-percent gain in two years! Surely we would soon break into the top 10! Mind the gap, Cambridge!
The university’s managers remained level-headed, ignoring the Times Higher Education rankings altogether, except perhaps when chortling and rubbing their hands together in glee around their mahogany tables behind closed doors. They naturally preferred QS, a rival ranking system that this year puts the University of Nottingham at No. 75, more or less as it has been for four years running, a number that is posted at our campus’s front entrance. (We have also done well in a survey run from China, the Shanghai Jiao Tong University list, which currently puts Nottingham at No. 83, but no one outside Shanghai seems to go by the Shanghai list, so never mind.)
As for Times Higher Education, the triumphant march into the light came to an abrupt end in October when the magazine released rankings for the present academic year. The University of Nottingham has plummeted to No. 157. Cue sound of nails grating across chalkboard. The top 100, once within comfortable viewing distance, now induces neck strain to perceive.
Plenty of other British universities declined, too—Manchester, Bristol, Sheffield, Sussex, Edinburgh, Imperial College London, University College London, St. Andrews. But as the magazine noted drily, “The University of Nottingham has dropped the most, falling 37 places to joint 157.”
What in the world could possibly have caused such a precipitous drop? The magazine offers no explanation, leaving the reeling mind to wonder, and presenting plenty of comic possibilities, mostly in thoughts involving certain colleagues in a department once removed, but also, in darker hours of the night, oneself. (Was it that article of mine in Labor History?)
Reputation is one factor in the formula, so perhaps word spread throughout world higher education of our pensions being cut (sorry, restructured). That might account for the large number of British universities declining, but it hardly accounts for Nottingham’s specific plunge. Maybe it is all down to a friend of mine, who absconded irresponsibly, along with his impressive CV and his three books published by Cambridge University Press, to the London School of Economics?
So the mind races, until an even deeper possibility occurs: What if the entire exercise is fallacious? What if that is the only conclusion to be derived from an institution rocketing ahead 33 places in one year and falling 37 places two years later, for no discernible reason?
Even in 2011, my mind was just deviant enough to entertain such forbidden thoughts even as my institution shot toward the top, so I cheekily sent some questions by e-mail to the Times Higher Education editor Ann Mroz and Phil Baty, a staff member who oversees the survey. Mroz has since moved on to other roles, but Baty remains czar of the league tables.
Why, I asked, given that books remain the gold standard in the humanities, do books count nowhere in the research citation measurements, the topmost weighted survey category? Why rely solely on metrics about journal-article citations? And can the survey screen for self-citation, which will otherwise become a corrosive temptation?
Which institution, I asked, gets the credit for publications of scholars who change jobs, given the survey’s six-year data set? Perhaps the credit goes to the institution where the scholars published the articles, but then the institution that hired them gets no credit for having them on staff. Conversely, if it goes to the new institution, the one that provided the leave and made possible the publications gets no credit. How to adjudicate?
Finally, is it right to assume, as Times Higher Education’s formula does, that scholarship is most worthy if it has at least two authors from different countries? This means that if I, in Britain, collaborate on an article with someone from, say, Oxford or Cambridge, it will count for less than if I were to collaborate with someone from any university outside Britain, even those now ranked lower than the University of Nottingham, say Brandeis University, the University of Iowa, Pompeu Fabra University, or Yonsei University. How can that possibly make sense?
I am glad to report that I received, in response to these impertinencies, a very gracious answer from Mr. Baty, who said he was traveling abroad and promised to give me “a fuller response shortly,” while linking to an article on co-authorship that was certainly interesting but did not address my query. He never got around to that more fulsome answer, for which I hardly blame him, since he must live a life consumed with amassing vast data sets and formula refinement.
I am still left to wonder if it isn’t a fool’s game, such a ranking exercise. I recently spent an evening with a Duke University historian whose last book sold 160,000 copies. “I’m through with journal articles,” he told me, immediately rendering himself irrelevant in the Times Higher Education formula.
Scrutinize other nitty-gritty dimensions of the formula and there is plenty of cause for more doubt. The University of Nottingham performs at its worst in the category of “industry income.” That comprises only 2.5 percent of the total score but may very well account for why we are known for a book arguing that societies are better when they are more equal, rather than for, say, innovative accounting tricks under the rubric of “financial engineering.” Might lack of subordination to external private and state interests actually be a positive indicator of intellectual freedom and institutional quality?
I’m firmly with university management on this one. I say stick with QS—or Shanghai—if we show up better there! Or we might declare independence from the entire dodgy enterprise, boast of no number at all, and adopt as a motto this jaunty line from the new hit single “Royals,” by Lorde: “We’ll never be royals. ... And we don’t care. We aren’t caught up in your love affair.”