Back in January, a paper published in the Journal of Personality and Social Psychology appeared to prove that ESP is real, that in certain circumstances (involving, as it happens, erotic pictures) people really can predict the future. Naturally, this got more attention than your average academic publication. At the time I talked to the author of the paper, Daryl J. Bem, who was reeling from all the media attention.
Now that nearly a year has passed, I wanted to see if any replications had been published. I e-mailed Stuart Ritchie, a doctoral student in psychology at the University of Edinburgh, who, along with two colleagues, ran Bem’s experiments but didn’t get the same results. Their subjects couldn’t predict the future. This has been noted on blogs but, according to Ritchie, he and his colleagues haven’t had any success getting their paper published.
Considering all the attention Bem’s paper received -- and the enormous implications if his findings are indeed valid -- follow-up research seems pretty important. Ritchie wrote in an e-mail about his and co-authors’ struggles over recent months to get their paper published. I thought it was worth it to quote his account at length:
Here’s the story: we sent the paper to the journal that Bem published his paper in, and they said ‘no, we don’t ever accept straight replication attempts’. We then tried another couple of journals, who said the same thing. We then sent it to the British Journal of Psychology, who sent it out for review. For whatever reason (and they have apologised, to their credit), it was quite badly delayed in their review process, and they took many months to get back to us.
When they did get back to us, there were two reviews, one very positive, urging publication, and one quite negative. This latter review didn’t find any problems in our methodology or writeup itself, but suggested that, since the three of us (Richard Wiseman, Chris French and I) are all skeptical of ESP, we might have unconsciously influenced the results using our own psychic powers. The story behind this is that Richard has co-authored two papers where he and a believer in psi both did the same experiment, and the believer found positive results but he didn’t. However, the most recent time they did this - which was the best-controlled and largest size - neither found results. This doesn’t exactly give hugely compelling evidence for an ‘experimenter effect’ in psi research, in our opinion. Here’s that last paper, if you’re interested.
Anyway, the BJP editor agreed with the second reviewer, and said that he’d only accept our paper if we ran a fourth experiment where we got a believer to run all the participants, to control for these experimenter effects. We thought that was a bit silly, and said that to the editor, but he didn’t change his mind. We don’t think doing another replication with a believer at the helm is the right thing to do, for the reason above, and for the reason that Bem had stated in his original paper that his experimental paradigms were designed so that most of the work is done by a computer and the experimenter has very little to do (this was explicitly because of his concerns about possible experimenter effects). So, after this very long and unproductive delay, we’re off to another journal to try again. How frustrating.
Obviously this is Ritchie’s take, and others involved may see it differently. But it made me think of the last paragraph of the New York Times story from last year about Bem’s paper:
And, of course, the unwritten rule that failed studies — the ones that find no effects — are far less likely to be published than positive ones. What are the odds, for instance, that the journal would have published Dr. Bem’s study if it had come to the ho-hum conclusion that ESP still does not exist?
Maybe Ritchie’s experience is the answer to that question.
