W hat can anonymous commenters do, other than sling mud with impunity?
Fix the scientific record, that’s what.
That was Brandon Stell’s thought when he started PubPeer, a website where faceless Internet critics can pick apart scientific papers in academic journals. Four years later, PubPeer has grown to be a proof-of-concept for a new way to vet scientific work — one that subjects authors to the scrutiny of hundreds or even thousands of experts, rather than just the handful who reviewed it before publication. As a result, Mr. Stell and his website have become key figures in the debate over how best to restore public confidence in academic science.
The idea was to capture the sort of second-guessing that already happens in academic departments across the country in gatherings called “journal clubs.” Journal clubs are like wonky book clubs: Prominent scientists and their protégés discuss recently published research papers, sometimes raising critical points about their validity.
Mr. Stell, 41, an American neuroscientist who now runs a brain-physiology lab in Paris, started attending journal clubs as an undergraduate at the University of Colorado at Boulder and kept up during graduate school at the University of California at Los Angeles. He was impressed by the discussions but nonplussed that the issues they raised remained confined to those rooms. “I was always like, ‘Why doesn’t everybody know about the issues?’” he says.
He decided that any points worth making about a published paper in a journal-club meeting were worth making in public, where other experts could weigh in anonymously.
It was a risky move. Academics can be prickly about having their work questioned, especially when the critiques occur outside the normal scholarly channels. Scientific subfields are vulnerable to small-town politics. “My grants and my publications are reviewed by my peers,” says Mr. Stell, “and if one of them received a comment on PubPeer, it would be very easy for them to put pressure on me to remove that comment or change that comment.”
So he and the site’s two co-founders, Richard Smith and George Smith, decided to stay anonymous — at least in the beginning. (The Smith brothers have not revealed their institutional affiliations, except to say that Richard was formerly a graduate student working in Mr. Stell’s lab and George is a programmer who coded PubPeer’s website.)
Promoting a new website can be difficult enough when the founders can put their names behind it. Mr. Stell and his collaborators faced the challenging task of evangelizing for PubPeer without giving away that it was their own project. The neuroscientist hoped people would buy his explanation, which was always some version of, “Check it out, I just stumbled upon this cool new website.”
It was, at times, an awkward secret. Sometime in the first year of PubPeer, somebody commented on a paper written by the head of Mr. Stell’s department. Somebody else commented on a paper written by the person who was reviewing a funding application for Mr. Stell’s brain-physiology lab. “If those guys knew that I was the guy behind the site,” he says, “it would have, at the very least, created an uncomfortable situation.”
PubPeer slowly gained popularity, helped along by some high-profile successes. In 2013 an anonymous critique on the site forced the authors of a stem-cell paper published in the prestigious journal Cell to acknowledge a series of errors. In 2014, PubPeer commenters began pulling loose threads on a pair of groundbreaking research papers, also on stem cells, published in Nature; an investigation later revealed that some images had been manipulated and some text plagiarized. The site’s commenters also played a key role in last year’s downfall of Olivier Voinnet, a well-known plant biologist who now acknowledges numerous errors in his papers over the years.
PubPeer attracted more than 55,000 visitors last month, and users have contributed nearly 50,000 comments.
S ome see the anonymity that PubPeer affords in a more sinister light. “PubPeer is unquestionably an opportunity to vent spleen at the imperfections of colleagues,” wrote Michael R. Blatt, editor of the journal Plant Physiology, in “Vigilante Science,” an editorial, last fall. “For some, it is also an invitation to undermine competitors with innuendo and the implicit threat of uncovering something perfidious.”
The moderators of PubPeer say they vet comments for personal attacks and require users to base any criticisms on “publicly verifiable information.” Not only does anonymity encourage people to raise concerns about published research without fear of retribution, says Mr. Stell, but it also forces researchers to focus on the content of a criticism rather than the source. “Frankly, a few ruffled academic feathers pale into insignificance,” he wrote in a response to Mr. Blatt, “when patients’ lives, taxpayer billions, and young researchers’ careers are at stake.”
At least one academic argues that PubPeer has done undue damage to his career. In 2014, Fazlul Sarkar, a cancer researcher, took legal action to make PubPeer reveal the identities of several users after comments on the site prompted the University of Mississippi to rescind a lucrative job offer it had made to him. A Michigan judge last year ordered the website to reveal the identity of one of the commenters, a decision PubPeer is appealing with help from the American Civil Liberties Union.
Late last summer Mr. Stell registered the PubPeer Foundation as a nonprofit public-benefit corporation, a move that required him and his collaborators to reveal their identities. He says he has not lost any friends, although one French academic did confront him in a hallway at a conference and accuse him of encouraging something akin to the practice in Nazi-occupied France of whispering that neighbors were Jews, even if it were untrue. (So much for the measure and civility of face-to-face criticism.)
Mr. Stell believes that his own hypothesis — that open, anonymous commenting makes science better — can withstand even its most rational critics. His formula is simple: More anonymity means more scrutiny for published papers, and more scrutiny means more errors are caught. If that means more authors have to tolerate questions from indiscernible figures up in the cheap seats, then so be it.
“Science proceeds by the falsification and refinement of hypotheses,” Mr. Stell wrote last year in a response to critics. “Thus, for most papers, the only way is down.”
