A new center at Stanford University seeks to encourage scientists to look to a new source for research material: the mirror.
At the Meta-Research Innovation Center at Stanford, or Metrics, John P.A. Ioannidis and Steven N. Goodman, both professors of medicine at Stanford, plan to study how research is done, and how it can be done better.
Dr. Ioannidis is perhaps best known outside epidemiology circles for a 2005 paper provocatively titled “Why Most Published Research Findings Are False.”
Small sample sizes and financial conflicts of interest were among several red flags that Dr. Ioannidis, then a professor of medicine at Greece’s University of Ioannina School of Medicine, highlighted as potential indicators of dubious findings.
As the Stanford center’s directors, he and Dr. Goodman hope to bring together like-minded researchers working across many disciplines to share their findings informally and, potentially, at conferences every couple of years. The program has received a $6-million grant from the Laura and John Arnold Foundation.
“We’re all interested in advancing excellence in research,” says Dr. Ioannidis, who is 48 and joined Stanford in 2010. Through their combined efforts he hopes the center’s researchers can improve the efficiency of scientific research by identifying common design errors and biases that can lead to flawed research that isn’t reproducible.
He cautions, though, that meta-researchers can be prone to the same mistakes as the researchers whose work they hope to improve.
Dr. Ioannidis, who plans to devote roughly half of his time to the center, says he was first drawn to the field of meta-analysis as he struggled with mistakes in his own work. As he turned to prior research for answers, he found that much of it suffered from the same problems.
“It became apparent to me that these issues occurred so frequently that, maybe instead of trying to answer a single question, one should take a bird’s-eye view,” he says. “What’s happening in the scientific literature in general?”
A meeting with meta-analysis pioneers Thomas C. Chalmers and Joseph Lau during Dr. Ioannidis’s residency at Harvard Medical School persuaded him to devote more of his research efforts to the still-young field.
Identifying common flaws is one goal, but Dr. Ioannidis says it’s also important to decide how to fix them. Should a review of research methods occur before grants are issued? By institutional review boards, or at the pre-publication level?
Part of the answer is persuading researchers to share their data with others, so that their results can be reproduced. But many researchers are resistant to the idea, particularly if they’re working with a unique data set.
“They feel that their databases are personal gold mines,” he says.
Despite pockets of resistance, Dr. Ioannidis says, most researchers have been receptive to his work. By applying the same scrutiny to his own work and drawing conclusions about wide swaths of research, rather than particular papers, he says, he has avoided making enemies in the ivory tower.
“If I were to say, ‘That person’s study is miserable,’” he says, “I would probably start to get some death threats.”
