Skip to content
ADVERTISEMENT
Sign In
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Events
    • Virtual Events
    • Chronicle On-The-Road
    • Professional Development
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
  • More
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Events
    • Virtual Events
    • Chronicle On-The-Road
    • Professional Development
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
    Upcoming Events:
    Student Housing
    Serving Higher Ed
    Chronicle Festival 2025
Sign In
News

International Group Announces Audit of University Rankings

By Aisha Labi October 10, 2010
At a conference in Berlin, higher-education officials are discussing the growth of university rankings, including a new effort to evaluate the ratings systems themselves.
At a conference in Berlin, higher-education officials are discussing the growth of university rankings, including a new effort to evaluate the ratings systems themselves.Stefan Zeitz
Berlin

University rankings organizations could soon find themselves on the receiving end of the kinds of evaluations that have made them so newsworthy and influential. At a conference here last week for academics and institutions focused on rankings, the organizer unveiled a project that would effectively rank the rankers.

To continue reading for FREE, please sign in.

Sign In

Or subscribe now to read with unlimited access for as low as $10/month.

Don’t have an account? Sign up now.

A free account provides you access to a limited number of free articles each month, plus newsletters, job postings, salary data, and exclusive store discounts.

Sign Up

University rankings organizations could soon find themselves on the receiving end of the kinds of evaluations that have made them so newsworthy and influential. At a conference here last week for academics and institutions focused on rankings, the organizer unveiled a project that would effectively rank the rankers.

The IREG Observatory on Academic Ranking and Excellence, which was created a few years ago to develop quality-control mechanisms for rankings, announced that a volunteer trial audit of two or three rankings will soon be under way.

The International Ranking Expert Group, or IREG, first met in Washington in 2004 and two years later came up with a set of principles on the ranking of higher-education institutions.

There has “always been the idea that IREG could evolve into a quality assurance” body, said Gero Federkeil, who oversees the rankings of German institutions by the CHE Centre for Higher Education Development, which co-hosted last week’s conference.

As rankings proliferate around the world, they are increasingly having a direct impact on the decisions of students, academic staff, institutions, and policy makers, but each of those groups differs in its use of rankings and the sophistication it brings to evaluating them.

Less informed groups, such as students, “don’t have a deep understanding of the limitations of rankings,” Mr. Federkeil said, and an audit would provide an assessment tool for users. The rankers themselves also need to be held accountable for possible deficits in their tabulations or methodological flaws, he said.

The audit project, which he is helping to manage, will be based closely on IREG’s principles, which emphasize clarity and openness in the purposes and goals of rankings, the design and weighting of indicators, the collection and processing of data, and the presentation of results.

“We all say that rankings should aim at delivering transparency about higher-education institutions, but we think there should be transparency about rankings too,” Mr. Federkeil said. The audit process could eventually give rise to an IREG quality label, which would amount to an identification of trustworthy rankings, thereby enhancing the credibility of rankings and improving their quality, Mr. Federkeil said.

At the Berlin meeting last week, Mr. Federkeil and Ying Cheng, of the Center for World-Class Universities at Shanghai Jiao Tong University, which produces the best-known and most influential global ranking of universities, outlined the proposed methodology and procedure for the audit. The IREG executive committee will nominate audit teams consisting of three to five people. The chair of each team must not have any formal affiliation with a ranking organization, and at least one member of the audit team must be a member of the IREG executive committee. Audits will be based on self-reported data as well as possible on-site visits, and each full audit is expected to take about five months to complete.

Skepticism and Unease

Whether the audit will actually work remains to be seen. Many of the people who attended the meeting expressed deep skepticism and unease about how effectively a rigorous and independent audit procedure could be applied.

ADVERTISEMENT

“In principle, I think it’s a good thing,” said Ben Sowter, head of the intelligence unit at QS, which produces the QS World University Rankings. But “there is a long way to go before this audit looks like the kind of measure it needs to be.”

Still, if it eventually evolves into a widely accepted and respected quality-assurance mechanism, the audit could become a useful tool “and enable us to counter some of the criticism that we receive,” he added.

Robert J. Morse, director of data research at U.S. News & World Report, said the magazine would most likely participate in the audit, but only “after we fully understand the processes and how it’s going to be scored.”

He agreed that it is important for rankers to be held to standards and to be transparent in their work.

ADVERTISEMENT

“We communicate very frequently with academics, but maybe we would need to also post in more detail about the mathematical processes and quality controls and other steps we take from the academic level, and that’s something that we would consider doing,” he said.

Mr. Morse and others also asked whether there would really be critical distance between the audit committee and IREG’s executive committee.

Ellen Hazelkorn, executive director of the Higher Education Policy Research Unit at the Dublin Institute of Technology and a well-known critic of the growing influence of rankings in shaping institutional and governmental policy, noted that rankings have become an intensely competitive business, and that any audit procedures would need to be clear and open enough to ensure that competitors were not pronouncing on one another’s work.

She also said that auditors should ensure that all constituencies are involved in the process, including academics, policy makers, and students.

ADVERTISEMENT

“I think it could potentially go somewhere,” she said of the audit project. “I’m just not sure as to how it would work and who might subject themselves to it.”

Some people invoked a comparison between the proposed audit and the accreditation process in the United States, in which universities participate voluntarily, but Ms. Hazelkorn emphasized that “universities have a compulsion to participate in accreditation” in order to secure eligibility for such financial benefits as Pell Grants, and that such an incentive is absent in when it comes to rankings.

But Nian Cai Liu, dean of the Graduate School of Education at Shanghai Jiao Tong University, which began producing the Academic Ranking of World Universities in 2003, applauded the effort.

“We need something, we need to start,” he said. “I think there will be more and more rankings, but there will be in a sense be more concentration of rankings,” he predicted. Those with an IREG approval label will grow in influence, but the rest will lose significance.

We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Tags
International
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
ADVERTISEMENT
ADVERTISEMENT

Related Content

International Conference Takes a Fresh Look at University Rankings

More News

UCLA students, researchers and demonstrators rally during a "Kill the Cuts" protest against the Trump administration's funding cuts on research, health and higher education at the University of California Los Angeles (UCLA) in Los Angeles on April 8, 2025.
Scholarship & Research
Trump Proposed Slashing the National Science Foundation’s Budget. A Key Senate Committee Just Refused.
Illustration of a steamroller rolling over a colorful road and leaving gray asphalt in its wake.
Newly Updated
Oregon State U. Will End a Renowned Program That Aimed to Reduce Bias in Hiring
Dr. Gregory Washington, president of George Mason University.
Another probe
George Mason President Discriminated Against White People After George Floyd Protests, Justice Dept. Says
Protesters gather outside the Department of Education headquarters in Washington, D.C., on Feb. 14, 2025 to protest the Trump administrations cuts at the agency.
An Uncertain Future
The Education Dept. Got a Green Light to Shrink. Here Are 3 Questions About What’s Next.

From The Review

Photo-based illustration with repeated images of a student walking, in the pattern of a graph trending down, then up.
The Review | Opinion
7 Ways Community Colleges Can Boost Enrollment
By Bob Levey
Illustration of an ocean tide shaped like Donald Trump about to wash away sandcastles shaped like a college campus.
The Review | Essay
Why Universities Are So Powerless in Their Fight Against Trump
By Jason Owen-Smith
Photo-based illustration of a closeup of a pencil meshed with a circuit bosrd
The Review | Essay
How Are Students Really Using AI?
By Derek O'Connell

Upcoming Events

07-31-Turbulent-Workday_assets v2_Plain.png
Keeping Your Institution Moving Forward in Turbulent Times
Ascendium_Housing_Plain.png
What It Really Takes to Serve Students’ Basic Needs: Housing
Lead With Insight
  • Explore Content
    • Latest News
    • Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Events
    • Chronicle Store
    • Chronicle Intelligence
    • Jobs in Higher Education
    • Post a Job
  • Know The Chronicle
    • About Us
    • Vision, Mission, Values
    • DEI at The Chronicle
    • Write for Us
    • Work at The Chronicle
    • Our Reporting Process
    • Advertise With Us
    • Brand Studio
    • Accessibility Statement
  • Account and Access
    • Manage Your Account
    • Manage Newsletters
    • Individual Subscriptions
    • Group and Institutional Access
    • Subscription & Account FAQ
  • Get Support
    • Contact Us
    • Reprints & Permissions
    • User Agreement
    • Terms and Conditions
    • Privacy Policy
    • California Privacy Policy
    • Do Not Sell My Personal Information
1255 23rd Street, N.W. Washington, D.C. 20037
© 2025 The Chronicle of Higher Education
The Chronicle of Higher Education is academe’s most trusted resource for independent journalism, career development, and forward-looking intelligence. Our readers lead, teach, learn, and innovate with insights from The Chronicle.
Follow Us
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin