A Much-Debated Neuroscience Paper Has Lost Its Voodoo

April 01, 2009

One of the most-debated psychology papers of the last few years is finally about to enjoy its official print publication — but without the provocative title that helped spark the debate in the first place.

Last year four scholars at the Massachusetts Institute of Technology and the University of California at San Diego began to circulate a preprint version of “Voodoo Correlations in Social Neuroscience.”

The paper argues that many recent studies in social neuroscience — and, by implication, in several other types of neuroscience — suffer from a severe statistical flaw.

At issue are the brain-imaging studies that appear in the newspaper almost every day: studies that attempt to find correlations between, say, jealousy or gambling and specific regions of the brain. To oversimplify, the “Voodoo” paper argues that most such studies report spuriously high correlations because neuroscientists don’t adequately guard against the possibility that the brain patterns they see are simply random.

The preprint version generated a barrage of commentary in the psycho-blogosphere and the science press.

Critics have attacked not only the “Voodoo” authors’ findings but also their methods. In particular, some scholars have complained about a questionnaire that the authors sent to dozens of neuroscientists about their statistical techniques. Some have said that the questions were ambiguously phrased, and others have said that the authors should have been more candid about how critical they planned to be. (Here is one of the authors’ recent rejoinders to their critics.)

The “Voodoo” paper is now finally about to roll off the presses, in the May issue of Perspectives on Psychological Science. The issue also includes seven replies and commentaries — most of which, it turns out, are broadly sympathetic to the paper’s argument.

In an introduction, the journal’s editor, Ed Diener, a professor of psychology at the University of Illinois at Urbana-Champaign, suggests that he takes seriously the criticisms of the questionnaire. “When inquiring about another researcher’s methods and statistics, how much are we required to tell them?” he writes. “How much should we tell them in order to foster scientific collegiality?”

Mr. Diener also insisted on giving the paper a more sedate title. It will probably always be talked about as the “voodoo correlations” paper, but for official purposes it is now “Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition.” —David Glenn

(Voodoo-doll photo by the Flickr user Juha-Matti. Used under a Creative Commons license.)