> Skip to content
FEATURED:
  • The Evolution of Race in Admissions
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Find a Job
    • Post a Job
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Find a Job
    • Post a Job
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Find a Job
    • Post a Job
Sign In
ADVERTISEMENT
2015 Influence List
  • Twitter
  • LinkedIn
  • Show more sharing options
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
  • Copy Link URLCopied!
  • Print

Research Watchdog: Brian Nosek

By  Tom Bartlett
December 13, 2015
16Influencers-Nosek-1
Casey Templeton

It only seems as if Brian Nosek is trying to destroy psychology.

Really, he’s trying to save it. Mr. Nosek, who is 43 and executive director of the Center for Open Science, is reckoning with deep problems not just in his discipline — he’s a professor of psychology at the University of Virginia — but in science generally. While the problems can sound arcane, with talk of p-hacking and the file-drawer effect, the upshot is simple: Too much of what gets published in respected, peer-reviewed journals cannot be replicated. And when a finding can’t be replicated, that casts doubt on its reliability. And if what gets published in scientific journals is unreliable, then science is in trouble.

We’re sorry. Something went wrong.

We are unable to fully display the content of this page.

The most likely cause of this is a content blocker on your computer or network. Please make sure your computer, VPN, or network allows javascript and allows content to be delivered from c950.chronicle.com and chronicle.blueconic.net.

Once javascript and access to those URLs are allowed, please refresh this page. You may then be asked to log in, create an account if you don't already have one, or subscribe.

If you continue to experience issues, contact us at 202-466-1032 or help@chronicle.com

16Influencers-Nosek-1
Casey Templeton

It only seems as if Brian Nosek is trying to destroy psychology.

Really, he’s trying to save it. Mr. Nosek, who is 43 and executive director of the Center for Open Science, is reckoning with deep problems not just in his discipline — he’s a professor of psychology at the University of Virginia — but in science generally. While the problems can sound arcane, with talk of p-hacking and the file-drawer effect, the upshot is simple: Too much of what gets published in respected, peer-reviewed journals cannot be replicated. And when a finding can’t be replicated, that casts doubt on its reliability. And if what gets published in scientific journals is unreliable, then science is in trouble.

Before Mr. Nosek came along, no one knew the extent of that trouble. Oh, sure, there were signs — a failed replication here, a retracted paper there — but it felt scattershot. What was needed was a close examination of the system, something rigorous and hard to dismiss. In late 2011, Mr. Nosek started the Reproducibility Project, which aimed to replicate findings in 100 studies from three top psychology journals.

He forced a reckoning with deep problems in science.

It was an enormous undertaking, bringing together more than 270 researchers. It wasn’t easy: There were administrative headaches, and the project took considerably longer than originally predicted. But somehow they pulled it off.

The results? Only 39 percent of the studies could be replicated.

Terrible news, for sure. Talk to psychologists now, after the numbers have had time to sink in, and it’s clear that many of them still don’t know what to think. The project didn’t discover that a few flawed studies managed to somehow slip past the peer-review filter. Instead it showed that findings that can be replicated are the exception. Consequently, it’s hard to look at the results of an exciting new study these days without raising an eyebrow.

ADVERTISEMENT

Mr. Nosek’s own reaction to this dire verdict has been measured. “I would have loved for the reproducibility rate to have been higher,” he said when the project’s findings were released. That’s a diplomatic way of putting it.

He’s left it for others to speculate on which subdisciplines may have to be scrapped entirely, or whether entire high-profile careers are built on statistically shaky foundations. Mr. Nosek is trying to reform science without entirely alienating fellow scientists.

And he’s not finished. In a recently published paper, Mr. Nosek and his colleagues conducted an experiment to see if it’s possible to sniff out dubious studies in advance. They gave a few dozen researchers $100 each and asked them to bet on whether certain studies in the Reproducibility Project would be successfully replicated (obviously, they placed their bets before the project’s findings were known). It turns out that they bet correctly most of the time. In other words, scientists kind of already know which studies are bunk. That should make it easier to decide which studies should receive extra scrutiny.

It’s a sign that maybe there’s some hope amid all the discouragement. Or, as Mr. Nosek put it recently: “Wow, we could really do better.”

Tom Bartlett is a senior writer who covers science and other things. Follow him on Twitter @tebartl.

ADVERTISEMENT

A version of this article appeared in the December 18, 2015, issue.
Read other items in this The 2015 Influence List package.
We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Tom Bartlett
Tom Bartlett is a senior writer who covers science and ideas. Follow him on Twitter @tebartl.
ADVERTISEMENT
ADVERTISEMENT

Related Content

  • Coalition Builder: James G. Nondorf
  • Silence Breakers: Concerned Student 1950
  • Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
    Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
  • The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
    The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
  • Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
    Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
  • Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
    Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
1255 23rd Street, N.W. Washington, D.C. 20037
© 2023 The Chronicle of Higher Education
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin