> Skip to content
FEATURED:
  • The Evolution of Race in Admissions
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
ADVERTISEMENT
Blog Logo

Percolator: Scientists Offer New Formula to Predict Career Success

Research that matters.

  • Twitter
  • LinkedIn
  • Show more sharing options
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
  • Copy Link URLCopied!
  • Print

Scientists Offer New Formula to Predict Career Success

By  Paul Basken
September 12, 2012

First there was the “impact factor.” Then came the “h-index.” Now, for those who believe that scientific prowess can be measured by statistical metrics, comes the Acuna-Allesina-Kording formula.

The formula,

We’re sorry. Something went wrong.

We are unable to fully display the content of this page.

The most likely cause of this is a content blocker on your computer or network. Please make sure your computer, VPN, or network allows javascript and allows content to be delivered from c950.chronicle.com and chronicle.blueconic.net.

Once javascript and access to those URLs are allowed, please refresh this page. You may then be asked to log in, create an account if you don't already have one, or subscribe.

If you continue to experience issues, contact us at 202-466-1032 or help@chronicle.com

First there was the “impact factor.” Then came the “h-index.” Now, for those who believe that scientific prowess can be measured by statistical metrics, comes the Acuna-Allesina-Kording formula.

The formula, outlined on Wednesday in the journal Nature, is intended to improve upon the h-index—a tally of a researcher’s publications and citations—by adding a few more numerical measures of a scientist’s publishing history to allow for predictions of future success.

The idea, said the paper’s senior author, Konrad P. Kording, an associate professor of physical medicine and rehabilitation at Northwestern University, is to help universities and grant-writing agencies “fund someone who will have high impact in the future.”

Kording readily admitted his method—tweaking the h-index by adding numbers such as years of publication and number of distinct journals—cannot be perfect and shouldn’t be a substitute for thoughtful human analysis of a researcher’s past writings and future goals. But even careful subjective reviews have their limits, especially in the real world of deadline pressures and global competition, Kording said.

“Both ways of evaluating people,” he said, “have advantages and disadvantages.”

ADVERTISEMENT

Despite that caution, the idea put forth by Kording, along with Daniel E. Acuna, a postdoctoral student in his lab, and Stefano Allesina, an assistant professor of ecology and evolution at the University of Chicago, is already generating some of the same divisions that have surrounded the impact factor and the h-index.

“It’s really disturbing,” said Robert H. Austin, a professor of physics at Princeton University, referring to the trend in which he now sees hiring panels requiring candidates to state their h-index, and applicants sometimes offering it at the top of their résumés.

The focus on the h-index, said John M. Drake, an associate professor of ecology at the University of Georgia, is leading researchers to choose popular and established topics that are likely to win citations from other authors. That, he said, “would seem to be the opposite of creativity.”

The forerunner of the h-index, the impact factor, was devised in the 1950s by Eugene Garfield, a librarian pursuing a doctorate in structural linguistics. His simple formula—dividing the number of citations a scientific journal received in the two previous years by the number of articles it published—sparked a revolution in establishing the reputations of the journals and the scientists who write for them.

It also spawned resentment, with scientists complaining of the biases it creates and the gamesmanship it encourages. Among the more blatant cases of alleged abuse, journals eager to raise their impact factor have suggested that prospective writers mention some of their previously published articles.

ADVERTISEMENT

Then came the h-index, created in 2005 by Jorge E. Hirsch, a professor of physics at the University of California at San Diego. It’s also a simple measure, but aimed more at ranking the researcher than the journals. It’s the number of a researcher’s published papers that have been cited at least as many times as that number itself.

While it quickly gained popularity, the h-index suffered from the sense that it mostly confirmed past success. That’s because those with the largest h-index were readily seen as renowned scientists late in their careers.

Hirsch recognized the problem and wrote a follow-up analysis in 2007 in which he tested the ability of several other variants to provide greater predictions of a scientist’s future career prospects. His conclusion, published in PNAS, the Proceedings of the National Academies of Science, was that none did.

Kording and his team are now suggesting otherwise. Their proposal adds to the h-index by including a scientist’s total number of articles, the number of years since the first one was published, the number of journals they’ve appeared in, and the number of “top” journals.

The new formula, Kording said, has proved more than twice as accurate as the h-index for predicting the future success of researchers in the life sciences. He was driven to pursue the formula, he said, out of a basic curiosity about science and how it works.

ADVERTISEMENT

“I’m a scientist,” he said, “so as a scientist I can’t avoid basically asking myself about the future, about my career, about the career of friends, about the careers of people that I know, and which factors drive good science.” He also acknowledged the “basic angst” among scientists wondering if their work is ultimately either useful or worthless to society.

Hirsch is less impressed. After reviewing the Acuna-Allesina-Kording paper, he said the factors added to his h-index appeared to have little meaningful effect. He suggested the additional factors had been devised by “optimizing the coefficients” for a particular set of authors covered by the paper. He said the predictive powers would not hold up for a wider set of test cases.

“I would expect that dropping the h-index from the formula would have a major effect, and dropping any of the other criteria a minor effect,” Hirsch told The Chronicle.

Another critic of the overall use of such statistical measurements, Anurag A. Agrawal, a professor of ecology and evolutionary biology at Cornell University, said he’s gradually realizing that metrics such as impact factor and h-index will soon grow obsolete.

Eventually, Agrawal said, technology will allow better direct measurements of the value of each piece of published research. It won’t be so important to estimate a researcher’s value in five or 10 years, when data can show almost immediately how many people have actually downloaded and used a published journal article, he said.

ADVERTISEMENT

Kording also believes the future will improve the degree to which statistical measures complement subjective human evaluations. He acknowledged that impact factor and h-index can be “gamed” and can create harmful and perverse incentives. But he said similar problems faced Google, and the reason the company’s search engine has survived and prevailed is that its engineers are constantly adjusting it when they learn how people seek to manipulate its results.

“Basically, if you have strong statisticians that build metrics,” he said, “these guys will always be ahead of those people that game it.”

As for the effects of such improved metrics, Kording anticipates a world in which universities and grant-writing agencies work more efficiently, to the betterment of science.

Because better metrics may help all sides recognize young talent, “it might make the rich universities richer and the poor universities poorer,” he said. And rather than punish creative adventurers who dare to tread into areas not yet recognized by other scientists, better systems of talent evaluation might mean a person with the talent to lead an intellectual revolution might actually get the money needed to do it, he said.

“In a system where we have better metrics,” Kording said, “the people who could make most out of the money are more likely to get it.”

Paul Basken
Paul Basken was a government policy and science reporter with The Chronicle of Higher Education, where he won an annual National Press Club award for exclusives.
ADVERTISEMENT
ADVERTISEMENT
  • Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
    Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
  • The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
    The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
  • Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
    Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
  • Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
    Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
1255 23rd Street, N.W. Washington, D.C. 20037
© 2023 The Chronicle of Higher Education
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin