This Questionable Study Caught Fire in Anti-Vaccine Circles. How Did It Get Through Peer Review?
It was an eye-popping figure: the number of deaths caused by Covid vaccines “may be as high as 278,000,” according to a study published in January.
Right away, some readers noted seemingly disqualifying flaws in the methods used to produce the estimate, which included a survey funded by an anti-vaccine advocate. One medical expert called it “antivax propaganda disguised as a survey.” But their concerns were largely drowned out by Twitter and Substack accounts with collective followings in the millions who cheered the finding. Steve Kirsch, a veteran tech entrepreneur who founded an anti-vaccine group, pointed out the study had the ivory tower’s stamp of approval: It was published in a peer-reviewed scientific journal and written by a professor at Michigan State University.
We're sorry. Something went wrong.
We are unable to fully display the content of this page.
The most likely cause of this is a content blocker on your computer or network.
If you continue to experience issues, please contact us at 202-466-1032 or firstname.lastname@example.org
It was an eye-popping figure: The number of deaths caused by Covid vaccines “may be as high as 278,000,” according to a study published in January.
Right away, some readers noted seemingly disqualifying flaws in the methods used to produce the estimate, which included a survey funded by an anti-vaccine advocate. One medical expert called it “antivax propaganda disguised as a survey.” But their concerns were largely drowned out by Twitter and Substack accounts with collective followings in the millions, who cheered the finding. Steve Kirsch, a veteran tech entrepreneur who founded an anti-vaccine group, pointed out that the study had the ivory tower’s stamp of approval: It had been published in a peer-reviewed scientific journal and written by a professor at Michigan State University.
After two months of going viral, the study is set to be retracted by BMC Infectious Diseases, the author and the journal told The Chronicle. Three years into a pandemic that has killed more than 6.8 million people, the episode is the latest illustration of the pressure on scholarly journals to vet Covid research that may be built on falsehoods — a high-stakes job that they routinely botch, observers say.
“The maintenance of scientific integrity in journals — the correction and retraction process — is fundamentally ill-equipped to deal with the pandemic,” said Gideon Meyerowitz-Katz, an epidemiologist and Ph.D. student at the University of Wollongong, in Australia. Having dug into dozens of shoddy Covid studies, he called the soon-to-be-retracted paper “among the worst things I’ve ever seen published.”
But Mark Skidmore, the Michigan State economist who wrote the paper, said that he objected to the retraction. “I stand by the methods, analysis, and conclusions of the study,” he said by email.
Citing a desire to let readers decide for themselves, Skidmore compiled a PDF file of what he said was his post-publication correspondence with the journal’s top editor. According to the document, the correspondence led the journal’s editorial board to conclude that he had used “inappropriate” methodology that could not prove that vaccines had caused deaths, that “limitations of the study were not adequately described,” and that there were “critical issues” with “the accuracy of data collection,” among other problems.
Maria Hodges, executive editor of the group of journals that includes BMC Infectious Diseases and is published by Springer Nature, confirmed that the study would be retracted, but said she was unable to provide other details, including a timeline, until the process was complete. As of Wednesday morning, the study was still posted with an editor’s note, added two days after it had been published, saying that criticisms of it were being reviewed.
Severe reactions to Covid vaccines are rare, according to the Centers for Disease Control and Prevention, and any potential risks are outweighed by the benefits of being vaccinated. Over the time that more than 672 million doses have been given out in the United States, the agency has identified nine deaths caused by Johnson & Johnson’s vaccine.
Still, misinformation about the vaccines (and masks, hydroxychloroquine, ivermectin, and many other things) has flown fast and furious, and some of it has borne the imprimatur of peer review. More than 300 Covid-related papers have been retracted, according to Retraction Watch.
A Journal’s Most-Viewed Paper Ever
In Skidmore’s study, a hired company administered an online survey to an anonymous group of 2,800 people representative of the U.S. population in late 2021, and asked them about their experiences during the pandemic. The study found that those who knew someone who’d had a health problem from Covid were more likely to be vaccinated, while those who knew someone who’d experienced a health problem after being vaccinated were less likely to be vaccinated themselves.
Skidmore then went further and took the number of vaccine-caused deaths that the respondents reported knowing about — 57, according to the study — and used them to estimate the total number of people who had died for the same reason. To flesh out the estimate, he counted deaths reported to a federal database called the Vaccine Adverse Event Reporting System, known as VAERS, and arrived at the figure 278,000.
This methodology for calculating vaccine-induced deaths was rife with problems, observers noted, chiefly that Skidmore did not try to verify whether anyone counted in the death toll actually had been vaccinated, had died, or had died because of the vaccine.
When participants were asked to write about the health problem of the vaccinated person they knew “best,” answers like “death from a heart attack after vaccination by a few weeks,” “they passed away from Covid,” and “cancer retu[r]ned in his body spread all over” all counted as vaccine-caused deaths, according to study materials posted online. In addition, the VAERS database is open to reports from anyone and does not verify any of them. (Skidmore also did not cite the official VAERS website but an anonymously run WordPress site that posts about the need to “defund the CDC.”)
When questioned by the journal, Skidmore noted that his paper had stated its limitations, according to the emails posted on his website. Meyerowitz-Katz said that caveat was not sufficient: “It’s a bit like saying, ‘I have a million dollars,’ and when people ask you to prove you have a million dollars, you say, ‘The limitation is I don’t have a million dollars.’”
When Skidmore’s study was published, on January 24, it quickly became the journal’s most-viewed paper ever. One of its biggest promoters was Kirsch, the tech entrepreneur who invented the optical mouse. He wrote about it on his Substack site, which has more than 223,000 subscribers, and on his Twitter account, where he has almost 300,000 followers. (Last month he tweeted about offering $100,000 to a woman sitting next to him on a plane to remove her mask.) Kirsch has touted unproven Covid cures and founded the Vaccine Safety Research Foundation, which has disseminated ads about alleged vaccine injuries.
In a sympathetic interview with Skidmore, Kirsch noted that the study had been peer-reviewed. “The journal picks the peer reviewers … so how can they complain?” he said.
‘What I Should Be Doing’
BMC Infectious Diseases takes the unusual step of posting an article’s peer reviews online, a policy intended to increase transparency. Between being submitted in July and accepted in January, Skidmore’s manuscript was evaluated by two people, one of whom was disclosed to be Yasir Ahmed Mohammed Elhadi of Alexandria University, in Egypt. (Elhadi did not respond to a request for comment.) Neither reviewer questioned Skidmore’s method for counting vaccine-caused deaths.
David Gorski, a cancer surgeon and researcher at Wayne State University’s School of Medicine who called Skidmore’s calculations “fractal pseudoscience,” wrote on his blog that neither reviewer “appeared to know what they were talking about.” In the emails with the journal posted by Skidmore, the editor does not assign any blame to the peer reviewers.
Gorski noted that the online survey was disclosed to have been funded by Catherine Austin Fitts. According to The Washington Post, Fitts appeared in a viral video full of Covid misinformation and falsely suggested that the vaccines would “modify your DNA and for all we know make you infertile.” Gorski also observed that Skidmore has published multiple blog posts skeptical of the vaccines.
The economist said by email that he had published more than 80 peer-reviewed articles, and “in my 28 years as a professor, I have never experienced this particular situation.”
“I have been willing to ask important questions and try to answer them,” he added. “In my assessment, I have done what I should be doing.”
Hodges, the BMC editor, said in a statement: “We take our responsibility to maintain the accuracy of the scientific record very seriously, particularly when concerns are raised about findings that can influence health-related behaviors and attitudes.”
She said that the journal had begun investigating the paper’s claims as soon as public concerns were raised. “While we endeavor to proceed with investigations as swiftly and efficiently as possible, it can take some time to complete the process, and it is not always possible to update our readers as quickly as we would prefer,” she wrote. She declined to comment on the paper’s peer-review process, citing confidentiality reasons, but said that “we keep our editorial processes and policies under continuous review.”
Meyerowitz-Katz said BMC Infectious Diseases owes the public an explanation of how the paper got approved in the first place, including who on the BMC staff was responsible for vetting it, why the editors initially accepted the peer reviews but later backtracked, and if the episode will lead to changes in the journal’s review system. “The fact it’s being retracted is entirely appropriate,” he said. “That being said, the process of retraction is just as opaque and frustrating as the process at every other journal.”
Meanwhile, Meyerowitz-Katz is still waiting on multiple other journals to acknowledge problems he’s flagged. For two separate Covid studies, including one potentially linking heart problems to vaccines, he’s waited almost a year for a response.
In situations that aren’t public-health emergencies, it might be more acceptable for scientists to debate among themselves for a while after a study is published. But “when you’ve got something that’s been published that’s clearly incorrect and is impacting policy on a day-to-day basis,” he said, “you need a more rapid correction than the journals traditionally allow.”