> Skip to content
FEATURED:
  • Public Perception of College
Sign In
  • News
  • Advice
  • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Technology
    • The Workplace
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Technology
    • The Workplace
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
  • News
  • Advice
  • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Technology
    • The Workplace
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Technology
    • The Workplace
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
  • News
  • Advice
  • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Technology
    • The Workplace
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Technology
    • The Workplace
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
ADVERTISEMENT
Research
  • Twitter
  • LinkedIn
  • Show more sharing options
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
  • Copy Link URLCopied!
  • Print

Twitter’s Value as Measure of Scientific Impact Encounters New Doubt

By  Paul Basken
December 10, 2013

Evaluating scientists by their journal-citation counts is a much-despised shorthand at many universities. And so, when a study came out two years ago suggesting Twitter mentions as a supplement or even an alternative to citation counts, it got a fair bit of attention.

Now, according to a much more exhaustive analysis of the question, it appears that any correlation between tweets and citations was greatly overstated.

The previous study, from the University of Toronto, found that highly tweeted articles were almost 11 times as likely as less-tweeted articles to be highly cited. But the study involved only 55 articles in a single journal. The new study, by scientists at the University of Montreal, involved some 1.4 million articles from more than 5,000 journals. It found no meaningful correlation.

We're sorry. Something went wrong.

We are unable to fully display the content of this page.

The most likely cause of this is a content blocker on your computer or network.

Please allow access to our site, and then refresh this page. You may then be asked to log in, create an account if you don't already have one, or subscribe.

If you continue to experience issues, please contact us at 202-466-1032 or help@chronicle.com

Evaluating scientists by their journal-citation counts is a much-despised shorthand at many universities. And so, when a study came out two years ago suggesting Twitter mentions as a supplement or even an alternative to citation counts, it got a fair bit of attention.

Now, according to a much more exhaustive analysis of the question, it appears that any correlation between tweets and citations was greatly overstated.

The previous study, from the University of Toronto, found that highly tweeted articles were almost 11 times as likely as less-tweeted articles to be highly cited. But the study involved only 55 articles in a single journal. The new study, by scientists at the University of Montreal, involved some 1.4 million articles from more than 5,000 journals. It found no meaningful correlation.

“We found very much contradictory results,” said the lead author, Stefanie Haustein, a postdoctoral researcher at the University of Montreal’s School of Library and Information Science.

Her article, “Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature,” was recently published online by the Journal of the American Society for Information Science and Technology.

ADVERTISEMENT

Ms. Haustein’s review considered all articles and reviews indexed in PubMed and the Web of Science over the three years ending in 2012. Of the 15 most-tweeted articles, each prompting more than 260 tweets, only four had more than a dozen citations by other scientists and four had none at all.

The smaller 2011 study—by Gunther Eysenbach, an associate professor at the University of Toronto’s Institute of Health Policy, Management, and Evaluation—won coverage in major news outlets with the opposite message. But almost immediately, the implied suggestion that researchers pay more attention to Twitter began attracting skeptics.

In addition to their small number, the articles in Dr. Eysenbach’s study all came from the Journal of Medical Internet Research, which itself uses Twitter to promote its articles. Some reviews suggested that his research may have been affected by his ties to the journal—he disclosed in the study that he is editor, publisher, and a shareholder of the journal—and his well-known advocacy of replacing or augmenting journal citations with Twitter and other alternative measures of scientific value.

Indicators of Social Impact

Dr. Eysenbach said on Monday that he saw little in Ms. Haustein’s report to contradict his findings. Both studies, he said, affirm that tweets reflect social impact, “which is exactly what I proposed.”

Ms. Haustein said she agreed that tweets can reflect social impact. Even though she found no evidence that they reflect future article-citation levels, she said, Twitter may provide universities and others with valuable information about how scientific research is valued by the public at large.

ADVERTISEMENT

And Twitter does provide reaction on a much faster time scale than citations—days rather than years—so it might be possible someday to use tweets to help judge the value of research findings to future scientists if confounding factors can be identified and weighted, she said.

Those factors could include the degree to which journals or authors tweet their own work, or the degree to which the public sharing of a research finding is driven by curiosities not strictly related to scientific value. As examples, she noted that some of the 15 most-tweeted articles in her study concerned such topics as the likelihood of penile fracture during stressful sexual encounters, and the case of a semi-cooked squid that injected its sperm bag into the tongue of a Korean diner.

“There’s funny stuff in there that people don’t tweet for the scientific content, but because it’s interesting,” she said.

It’s also hard to tease out the effects of marketing, said a co-author, Cassidy R. Sugimoto, an assistant professor of library and information science at Indiana University at Bloomington. That further discounts the likelihood that tweets reflect some broad societal importance of studies appreciated “outside of the ivory tower,” Ms. Sugimoto said.

Of the 13 journals in which more than 50 percent of the publication’s articles were tweeted, nine had an official Twitter account and the other four were represented by official Twitter accounts of their publishers or affiliated societies, the study found.

ADVERTISEMENT

We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Law & PolicyPolitical Influence & Activism
Paul Basken
Paul Basken was a government policy and science reporter with The Chronicle of Higher Education, where he won an annual National Press Club award for exclusives.
ADVERTISEMENT
ADVERTISEMENT
  • Explore Content
    • Latest News
    • Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Virtual Events
    • Chronicle Store
    • Chronicle Intelligence
    • Find a Job
    • Post a Job
    Explore Content
    • Latest News
    • Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Virtual Events
    • Chronicle Store
    • Chronicle Intelligence
    • Find a Job
    • Post a Job
  • Know The Chronicle
    • About Us
    • Write for Us
    • Work at The Chronicle
    • Our Reporting Process
    • Advertise With Us
    • Brand Studio
    • DEI Commitment Statement
    • Accessibility Statement
    Know The Chronicle
    • About Us
    • Write for Us
    • Work at The Chronicle
    • Our Reporting Process
    • Advertise With Us
    • Brand Studio
    • DEI Commitment Statement
    • Accessibility Statement
  • Account and Access
    • Manage Your Account
    • Manage Newsletters
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    Account and Access
    • Manage Your Account
    • Manage Newsletters
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
  • Get Support
    • Contact Us
    • Reprints & Permissions
    • User Agreement
    • Terms and Conditions
    • Privacy Policy
    • California Privacy Policy
    • Do Not Sell My Personal Information
    Get Support
    • Contact Us
    • Reprints & Permissions
    • User Agreement
    • Terms and Conditions
    • Privacy Policy
    • California Privacy Policy
    • Do Not Sell My Personal Information
1255 23rd Street, N.W. Washington, D.C. 20037
© 2023 The Chronicle of Higher Education
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin