> Skip to content
FEATURED:
  • Student Success Resource Center
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
ADVERTISEMENT
Science
  • Twitter
  • LinkedIn
  • Show more sharing options
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
  • Copy Link URLCopied!
  • Print

When Researchers State Goals for Clinical Trials in Advance, Success Rates Plunge

By  Paul Basken
August 5, 2015

Apparently, requiring scientists to state their objectives ahead of time makes a big difference.

Around 2000, the U.S. government ordered researchers conducting clinical trials with federal money to announce ahead of time which medical question they were hoping to answer.

Before then, 57 percent of large-budget trials for cardiovascular disease attributed a positive effect to a drug or dietary supplement, according to a study published on Wednesday. After the new requirement, the success rate dropped to just 8 percent, the study found.

We’re sorry. Something went wrong.

We are unable to fully display the content of this page.

The most likely cause of this is a content blocker on your computer or network. Please make sure your computer, VPN, or network allows javascript and allows content to be delivered from c950.chronicle.com and chronicle.blueconic.net.

Once javascript and access to those URLs are allowed, please refresh this page. You may then be asked to log in, create an account if you don't already have one, or subscribe.

If you continue to experience issues, contact us at 202-466-1032 or help@chronicle.com

Apparently, requiring scientists to state their objectives ahead of time makes a big difference.

Around 2000, the U.S. government ordered researchers conducting clinical trials with federal money to announce ahead of time which medical question they were hoping to answer.

Before then, 57 percent of large-budget trials for cardiovascular disease attributed a positive effect to a drug or dietary supplement, according to a study published on Wednesday. After the new requirement, the success rate dropped to just 8 percent, the study found.

The difference reflects a shift away from what had been a common practice for scientists: combing through their data after a trial to find correlations between drugs and patient outcomes. Often those correlations might just randomly occur, said one of the study’s authors, Robert M. Kaplan, who is the chief science officer at the federal Agency for Healthcare Research and Quality.

“It was actually pretty common for people to just measure a lot of things” and then pick through the data, Mr. Kaplan said. “If you measure 20 things, one of them is going to be statistically significant by chance.”

ADVERTISEMENT

The findings were published on Wednesday in the journal PLOS ONE by Mr. Kaplan and Veronica L. Irvin, an assistant professor of public health and human sciences at Oregon State University. They began their project in 2012, when both worked at the National Institutes of Health’s Office of Behavioral and Social Sciences Research.

The Irvin-Kaplan analysis was based on 55 large medical trials — with annual budgets of at least $500,000 — financed by the NIH’s National Heart, Lung, and Blood Institute. Seventeen of the 30 trial summaries published from 1970 to 2000 showed positive results for an intervention, while only two of the 25 studies published thereafter produced a positive outcome.

It’s been a longstanding problem that researchers tend to avoid reporting negative outcomes of trials. Many experts believe that tendency is driven by factors including a desire to produce results that can be published in journals. Failing to report negative results, however, deprives other medical professionals and patients of important safety and efficacy information, the authors of the study said.

Null and negative findings need to become destigmatized, Mr. Kaplan said. He cited the example of a trial at the Women’s Health Initiative that demonstrated that postmenopausal estrogen-replacement therapy did not help most women. Because of that negative result, millions fewer women now use those ineffective medicines, he said.

The problem with random findings that seem to show effectiveness, Mr. Kaplan said, is that they are often statistical fluctuations that wouldn’t be repeated in a subsequent test. In those cases, drugs or other interventions might not actually be effective.

ADVERTISEMENT

The pre-registration of clinical trials through websites — primarily through the government’s ClinicalTrials.gov database — is part of a continuing process in which “the way we do science has really matured” over the last couple of decades, Mr. Kaplan said.

Mr. Kaplan and Ms. Irvin chose the National Heart, Lung, and Blood Institute for their study because it was one of the earliest adopters of trial-registration requirements. They are now working on a much larger analysis involving a variety of trials beyond heart disease, over a 50-year span, to see if the same trends can be found.

Paul Basken covers university research and its intersection with government policy. He can be found on Twitter @pbasken, or reached by email at paul.basken@chronicle.com.

We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Scholarship & Research
Paul Basken
Paul Basken was a government policy and science reporter with The Chronicle of Higher Education, where he won an annual National Press Club award for exclusives.
ADVERTISEMENT
ADVERTISEMENT
  • Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Virtual Events
    • Chronicle Store
    • Find a Job
    Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Virtual Events
    • Chronicle Store
    • Find a Job
  • The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
    The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
  • Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
    Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
  • Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
    Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
1255 23rd Street, N.W. Washington, D.C. 20037
© 2023 The Chronicle of Higher Education
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin