Mark Abramson for The Chronicle Review
John Bargh rocked the world of social psychology with experiments that showed the power of unconscious cues over our behavior.
A framed print of “The Garden of Earthly Delights” hangs above the moss-green, L-shaped sectional in John Bargh’s office on the third floor of Yale University’s Kirtland Hall. Hieronymus Bosch’s famous triptych imagines a natural environment that is like ours (water, flowers) yet not (enormous spiked and translucent orbs). What precisely the 15th-century Dutch master had in mind is still a mystery, though theories abound. On the left is presumably paradise, in the middle is the world, and on the right is hell, complete with knife-faced monster and human-devouring bird devil.
By Bosch’s standard, it’s too much to say the past year has been hellish for Bargh, but it hasn’t been paradise either. Along with personal upheaval, including a lengthy child-custody battle, he has coped with what amounts to an assault on his life’s work, the research that pushed him into prominence, the studies that Malcolm Gladwell called “fascinating” and Daniel Kahneman deemed “classic.” What was once widely praised is now being pilloried in some quarters as emblematic of the shoddiness and shallowness of social psychology. When Bargh responded to one such salvo with a couple of sarcastic blog posts, he was ridiculed as going on a “one-man rampage.” He took the posts down and regrets writing them, but his frustration and sadness at how he’s been treated remain.
Psychology may be simultaneously at the highest and lowest point in its history. Right now its niftiest findings are routinely simplified and repackaged for a mass audience; if you wish to publish a best seller sans bloodsucking or light bondage, you would be well advised to match a few dozen psychological papers with relatable anecdotes and a grabby, one-word title. That isn’t true across the board. Researchers engaged in more technical work on, say, the role of grapheme units in word recognition must comfort themselves with the knowledge that science is, by its nature, incremental. But a social psychologist with a sexy theory has star potential. In the last decade or so, researchers have made astonishing discoveries about the role of consciousness, the reasons for human behavior, the motivations for why we do what we do. This stuff is anything but incremental.
At the same time, psychology has been beset with scandal and doubt. Formerly high-flying researchers like Diederik Stapel, Marc Hauser, and Dirk Smeesters saw their careers implode after allegations that they had cooked their results and managed to slip them past the supposedly watchful eyes of peer reviewers. Psychology isn’t the only field with fakers, but it has its share. Plus there’s the so-called file-drawer problem, that is, the tendency for researchers to publish their singular successes and ignore their multiple failures, making a fluke look like a breakthrough. Fairly or not, social psychologists are perceived to be less rigorous in their methods, generally not replicating their own or one another’s work, instead pressing on toward the next headline-making outcome.
Much of the criticism has been directed at priming. The definitions get dicey here because the term can refer to a range of phenomena, some of which are grounded in decades of solid evidence—like the “anchoring effect,” which happens, for instance, when a store lists a competitor’s inflated price next to its own to make you think you’re getting a bargain. That works. The studies that raise eyebrows are mostly in an area known as behavioral or goal priming, research that demonstrates how subliminal prompts can make you do all manner of crazy things. A warm mug makes you friendlier. The American flag makes you vote Republican. Fast-food logos make you impatient. A small group of skeptical psychologists—let’s call them the Replicators—have been trying to reproduce some of the most popular priming effects in their own labs.
What have they found? Mostly that they can’t get those results. The studies don’t check out. Something is wrong. And because he is undoubtedly the biggest name in the field, the Replicators have paid special attention to John Bargh and the study that started it all.
As in so many other famous psychological experiments, the researcher lies to the subject. After rearranging lists of words into sensible sentences, the subject—a New York University undergraduate—is told that the experiment is about language ability. It is not. In fact, the real test doesn’t begin until the subject exits the room. In the hallway is a graduate student with a stopwatch hidden beneath her coat. She’s pretending to wait for a meeting but really she’s working with the researchers. She times how long it takes the subject to walk from the doorway to a strip of silver tape a little more than 30 feet down the corridor. The experiment hinges on that stopwatch.
The words the subject was asked to rearrange were not random, though they seemed that way (this was confirmed in postexperiment interviews with each subject). They were words like “bingo” and “Florida,” “knits” and “wrinkles,” “bitter” and “alone.” Reading the list, you can almost picture a stooped senior padding around a condo, complaining at the television. A control group unscrambled words that evoked no theme. When the walking times of the two groups were compared, the Florida-knits-alone subjects walked, on average, more slowly than the control group. Words on a page made them act old.
It’s a cute finding. But the more you think about it, the more serious it starts to seem. What if we are constantly being influenced by subtle, unnoticed cues? If “Florida” makes you sluggish, could “cheetah” make you fleet of foot? Forget walking speeds. Is our environment making us meaner or more creative or stupider without our realizing it? We like to think we’re steering the ship of self, but what if we’re actually getting blown about by ghostly gusts?
John Bargh and his co-authors, Mark Chen and Lara Burrows, performed that experiment in 1990 or 1991. They didn’t publish it until 1996. Why sit on such a fascinating result? For starters, they wanted to do it again, which they did. They also wanted to perform similar experiments with different cues. One of those other experiments tested subjects to see if they were more hostile when primed with an African-American face. They were. (The subjects were not African-American.) In the other experiment, the subjects were primed with rude words to see if that would make them more likely to interrupt a conversation. It did.
The researchers waited to publish until other labs had found the same type of results. They knew their finding would be controversial. They knew many people wouldn’t believe it. They were willing to stick their necks out, but they didn’t want to be the only ones.
Since that study was published in the Journal of Personality and Social Psychology, it has been cited more than 2,000 times. Though other researchers did similar work at around the same time, and even before, it was that paper that sparked the priming era. Its authors knew, even before it was published, that the paper was likely to catch fire. They wrote: “The implications for many social psychological phenomena ... would appear to be considerable.” Translation: This is a huge deal.
When he was 9 or 10, Bargh decided to become a psychologist. He was in the kitchen of his family’s house in Champaign, Ill., when this revelation came to him. He didn’t know everything that would entail, of course, or what exactly a psychologist did, but he wanted to understand more about human emotion because it was this “mysterious powerful influence on everything.” His dad was an administrator at the University of Illinois, and so he was familiar with university campuses. He liked them. He still does. When he was in high school, he remembers arguing about B.F. Skinner. Everyone else in the class thought Skinner’s ideas were ridiculous. Bargh took the other side, not so much because he embraced the philosophy of radical behaviorism or enjoyed Skinner’s popular writings. It was more because he reveled in contrarianism. “This guy is thinking something nobody else agrees with,” he says now. “Let’s consider that he might be right.”
I met Bargh on a Thursday morning a couple of weeks before Christmas. He was dressed in cable-knit and worn jeans with hiking boots. At 58 he still has a full head of dark, appropriately mussed-up hair. Bargh was reclining on the previously mentioned moss-green sectional while downing coffee to stay alert as he whittled away at a thick stack of finals papers. He rose to greet me, sat back down, and sighed.
The last year has been tough for Bargh. Professionally, the nadir probably came in January, when a failed replication of the famous elderly-walking study was published in the journal PLoS ONE. It was not the first failed replication, but this one stung. In the experiment, the researchers had tried to mirror Bargh’s methods with an important exception: Rather than stopwatches, they used automatic timing devices with infrared sensors to eliminate any potential bias. The words didn’t make subjects act old. They tried the experiment again with stopwatches and added a twist: They told those operating the stopwatches which subjects were expected to walk slowly. Then it worked. The title of their paper tells the story: “Behavioral Priming: It’s All in the Mind, but Whose Mind?”
The paper annoyed Bargh. He thought the researchers didn’t faithfully follow his methods section, despite their claims that they did. But what really set him off was a blog post that explained the results. The post, on the blog Not Exactly Rocket Science, compared what happened in the experiment to the notorious case of Clever Hans, the horse that could supposedly count. It was thought that Hans was a whiz with figures, stomping a hoof in response to mathematical queries. In reality, the horse was picking up on body language from its handler. Bargh was the deluded horse handler in this scenario. That didn’t sit well with him. If the PLoS ONE paper is correct, the significance of his experiment largely dissipates. What’s more, he looks like a fool, tricked by a fairly obvious flaw in the setup.
Bargh responded in two long, detailed posts on his rarely updated Psychology Today blog. He spelled out the errors he believed were made in the PLoS ONE paper. Most crucially, he wrote, in the original experiment there was no way for the graduate student with the stopwatch to know who was supposed to walk slowly and who wasn’t. The posts were less temperate than most public discourse in science, but they were hardly mouth-foaming rants. He referred to “incompetent or ill-informed researchers,” clearly a shot at the paper’s authors. He mocked the journal where the replication was published as “pay to play” and lacking the oversight of traditional journals. The title of the post, “Nothing in Their Heads,” while perhaps a reference to unconscious behavior, seemed less than collegial.
He also expressed concern for readers who count on “supposedly reputable online media sources for accurate information on psychological science.” This was a dig at the blog post’s author, Ed Yong, who Bargh believes had written an unfair piece. “I was hurt by the things that were said, not just in the article, but in Ed Yong’s coverage of it,” Bargh says now. Yong’s post was more, though, than a credulous summary of the study. He interviewed researchers and provided context. The headline, “Why a classic psychology experiment isn’t what it seemed,” might benefit from softening, but if you’re looking for an example of sloppy journalism, this ain’t it.
While Bargh was dismayed by the paper and the publicity, the authors of the replication were equally taken aback by the severity of Bargh’s reaction. “That really threw us off, that response,” says Axel Cleeremans, a professor of cognitive science at the Université Libre de Bruxelles. “It was obvious that he was so dismissive, it was close to frankly insulting. He described us as amateur experimentalists, which everyone knows we are not.” Nor did they feel that his critique of their methods was valid. Even so, they tried the experiment again, taking into account Bargh’s concerns. It still didn’t work.
Bargh took his blog posts down after they were criticized. Though his views haven’t changed, he feels bad about his tone. In our conversations over the last month or so, Bargh has at times vigorously defended his work, pointing to a review he published recently in Trends in Cognitive Sciences that marshals recent priming studies into a kind of state-of-the-field address. Short version: Science marches on, priming’s doing great.
He complains that he has been a victim of scientific bullying (and some sympathetic toward Bargh use that phrase, too). There are other times, though, when he just seems crushed. “You invest your whole career and life in something, and to have this happen near the end of it—it’s very hard to take,” he says. Priming is what Bargh is known for. When he says “my name is a symbol that stands for these kinds of effects,” he’s not being arrogant. That’s a fact. Before the 1996 paper, he had already published respected and much-cited work on unconscious, automatic mental processes, but priming has defined him. In an interview on the Web site Edge a few years ago, back before the onslaught, he explained his research goals: “We have a trajectory downward, always downward, trying to find simple, basic causes and with big effects. We’re looking for simple things—not anything complicated—simple processes or concepts that then have profound effects.” The article labeled him “the simplifier.”
When I ask if he still believes in these effects, he says yes. They have been replicated in multiple labs. Some of those replications have been exact: stopwatch, the same set of words, and so on. Others have been conceptual. While they explore the same idea, maybe the study is about handwriting rather than walking. Maybe it’s about obesity rather than elderly stereotypes. But the gist is the same. “It’s not just my work that’s under attack here,” Bargh says. “It’s lots of people’s research being attacked and dismissed.” He has moments of doubt. How could he not? It’s deeply unsettling to have someone scrutinizing your old papers, looking for inconsistencies, even if you’re fairly confident about what you’ve accomplished. “Maybe there’s something we were doing that I didn’t realize,” he says, explaining the thoughts that have gone through his head. “You start doing that examination.”
So why not do an actual examination? Set up the same experiments again, with additional safeguards. It wouldn’t be terribly costly. No need for a grant to get undergraduates to unscramble sentences and stroll down a hallway. Bargh says he wouldn’t want to force his graduate students, already worried about their job prospects, to spend time on research that carries a stigma. Also, he is aware that some critics believe he’s been pulling tricks, that he has a “special touch” when it comes to priming, a comment that sounds like a compliment but isn’t. “I don’t think anyone would believe me,” he says.
Harold Pashler wouldn’t. Pashler, a professor of psychology at the University of California at San Diego, is the most prolific of the Replicators. He started trying priming experiments about four years ago because, he says, “I wanted to see these effects for myself.” That’s a diplomatic way of saying he thought they were fishy. He’s tried more than a dozen so far, including the elderly-walking study. He’s never been able to achieve the same results. Not once.
This fall, Daniel Kahneman, the Nobel Prize-winning psychologist, sent an e-mail to a small group of psychologists, including Bargh, warning of a “train wreck looming” in the field because of doubts surrounding priming research. He was blunt: “I believe that you should collectively do something about this mess. To deal effectively with the doubts you should acknowledge their existence and confront them straight on, because a posture of defiant denial is self-defeating,” he wrote.
Strongly worded e-mails from Nobel laureates tend to get noticed, and this one did. He sent it after conversations with Bargh about the relentless attacks on priming research. Kahneman cast himself as a mediator, a sort of senior statesman, endeavoring to bring together believers and skeptics. He does have a dog in the fight, though: Kahneman believes in these effects and has written admiringly of Bargh, including in his best seller Thinking, Fast and Slow.
On the heels of that message from on high, an e-mail dialogue began between the two camps. The vibe was more conciliatory than what you hear when researchers are speaking off the cuff and off the record. There was talk of the type of collaboration that Kahneman had floated, researchers from opposing sides combining their efforts in the name of truth. It was very civil, and it didn’t lead anywhere.
In one of those e-mails, Pashler issued a challenge masquerading as a gentle query: “Would you be able to suggest one or two goal priming effects that you think are especially strong and robust, even if they are not particularly well-known?” In other words, put up or shut up. Point me to the stuff you’re certain of and I’ll try to replicate it. This was intended to counter the charge that he and others were cherry-picking the weakest work and then doing a victory dance after demolishing it. He didn’t get the straightforward answer he wanted. “Some suggestions emerged but none were pointing to a concrete example,” he says.
One possible explanation for why these studies continually and bewilderingly fail to replicate is that they have hidden moderators, sensitive conditions that make them a challenge to pull off. Pashler argues that the studies never suggest that. He wrote in that same e-mail: “So from our reading of the literature, it is not clear why the results should be subtle or fragile.”
Bargh contends that we know more about these effects than we did in the 1990s, that they’re more complicated than researchers had originally assumed. That’s not a problem, it’s progress. And if you aren’t familiar with the literature in social psychology, with the numerous experiments that have modified and sharpened those early conclusions, you’re unlikely to successfully replicate them. Then you will trot out your failure as evidence that the study is bogus when really what you’ve proved is that you’re no good at social psychology.
Pashler can’t quite disguise his disdain for such a defense. “That doesn’t make sense to me,” he says. “You published it. That must mean you think it is a repeatable piece of work. Why can’t we do it just the way you did it?”
That’s how David Shanks sees things. He, too, has been trying to replicate well-known priming studies, and he, too, has been unable to do so. In a forthcoming paper, Shanks, a professor of psychology at University College London, recounts his and his several co-authors’ attempts to replicate one of the most intriguing effects, the so-called professor prime. In the study, one group was told to imagine a professor’s life and then list the traits that brought to mind. Another group was told to do the same except with a soccer hooligan rather than a professor.
The groups were then asked questions selected from the board game Trivial Pursuit, questions like “Who painted ‘Guernica’?” and “What is the capital of Bangladesh?” (Picasso and Dhaka, for those playing at home.) Their scores were then tallied. The subjects who imagined the professor scored above a control group that wasn’t primed. The subjects who imagined soccer hooligans scored below the professor group and below the control. Thinking about a professor makes you smart while thinking about a hooligan makes you dumb. The study has been replicated a number of times, including once on Dutch television.
Shanks can’t get the result. And, boy, has he tried. Not once or twice, but nine times.
The skepticism about priming, says Shanks, isn’t limited to those who have committed themselves to reperforming these experiments. It’s not only the Replicators. “I think more people in academic psychology than you would imagine appreciate the historical implausibility of these findings, and it’s just that those are the opinions that they have over the water fountain,” he says. “They’re not the opinions that get into the journalism.”
Like all the skeptics I spoke with, Shanks believes the worst is yet to come for priming, predicting that “over the next two or three years you’re going to see an avalanche of failed replications published.” The avalanche may come sooner than that. There are failed replications in press at the moment and many more that have been completed (Shanks’s paper on the professor prime is in press at PLoS ONE). A couple of researchers I spoke with didn’t want to talk about their results until they had been peer reviewed, but their preliminary results are not encouraging.
Ap Dijksterhuis is the author of the professor-prime paper. At first, Dijksterhuis, a professor of psychology at Radboud University Nijmegen, in the Netherlands, wasn’t sure he wanted to be interviewed for this article. That study is ancient news—it was published in 1998, and he’s moved away from studying unconscious processes in the last couple of years, in part because he wanted to move on to new research on happiness and in part because of the rancor and suspicion that now accompany such work. He’s tired of it.
The outing of Diederik Stapel made the atmosphere worse. Stapel was a social psychologist at Tilburg University, also in the Netherlands, who was found to have committed scientific misconduct in scores of papers. The scope and the depth of the fraud were jaw-dropping, and it changed the conversation. “It wasn’t about research practices that could have been better. It was about fraud,” Dijksterhuis says of the Stapel scandal. “I think that’s playing in the background. It now almost feels as if people who do find significant data are making mistakes, are doing bad research, and maybe even doing fraudulent things.”
In the e-mail discussion spurred by Kahneman’s call to action, Dijksterhuis laid out a number of possible explanations for why skeptics were coming up empty when they attempted priming studies. Cultural differences, for example. Studying prejudice in the Netherlands is different from studying it in the United States. Certain subjects are not susceptible to certain primes, particularly a subject who is unusually self-aware. In an interview, he offered another, less charitable possibility. “It could be that they are bad experimenters,” he says. “They may turn out failures to replicate that have been shown by 15 or 20 people already. It basically shows that it’s something with them, and it’s something going on in their labs.”
Joseph Cesario is somewhere between a believer and a skeptic, though these days he’s leaning more skeptic. Cesario is a social psychologist at Michigan State University, and he’s successfully replicated Bargh’s elderly-walking study, discovering in the course of the experiment that the attitude of a subject toward the elderly determined whether the effect worked or not. If you hate old people, you won’t slow down. He is sympathetic to the argument that moderators exist that make these studies hard to replicate, lots of little monkey wrenches ready to ruin the works. But that argument only goes so far. “At some point, it becomes excuse-making,” he says. “We have to have some threshold where we say that it doesn’t exist. It can’t be the case that some small group of people keep hitting on the right moderators over and over again.”
Cesario has been trying to replicate a recent finding of Bargh’s. In that study, published last year in the journal Emotion, Bargh and his co-author, Idit Shalev, asked subjects about their personal hygiene habits—how often they showered and bathed, for how long, how warm they liked the water. They also had subjects take a standard test to determine their degree of social isolation, whether they were lonely or not. What they found is that lonely people took longer and warmer baths and showers, perhaps substituting the warmth of the water for the warmth of regular human interaction.
That isn’t priming, exactly, though it is a related unconscious phenomenon often called embodied cognition. As in the elderly-walking study, the subjects didn’t realize what they were doing, didn’t know they were bathing longer because they were lonely. Can warm water alleviate feelings of isolation? This was a result with real-world applications, and reporters jumped on it. “Wash the loneliness away with a long, hot bath,” read an NBC News headline.
Bargh’s study had 92 subjects. So far Cesario has run more than 2,500 through the same experiment. He’s found absolutely no relationship between bathing and loneliness. Zero. “It’s very worrisome if you have people thinking they can take a shower and they can cure their depression,” he says. And he says Bargh’s data are troublesome. “Extremely small samples, extremely large effects—that’s a red flag,” he says. “It’s not a red flag for people publishing those studies, but it should be.”
Even though he is, in a sense, taking aim at Bargh, Cesario thinks it’s a shame that the debate over priming has become so personal, as if it’s a referendum on one man. “He has the most eye-catching findings. He always has,” Cesario says. “To the extent that some of his effects don’t replicate, because he’s identified as priming, it casts doubt on the entire body of research. He is priming.”
That has been the narrative. Bargh’s research is crumbling under scrutiny and, along with it, perhaps priming as a whole. Maybe the most exciting aspect of social psychology over the last couple of decades, these almost magical experiments in which people are prompted to be smarter or slower without them even knowing it, will end up as an embarrassing footnote rather than a landmark achievement.
Then along comes Gary Latham.
Latham, an organizational psychologist in the management school at the University of Toronto, thought the research Bargh and others did was crap. That’s the word he used. He told one of his graduate students, Amanda Shantz, that if she tried to apply Bargh’s principles it would be a win-win. If it failed, they could publish a useful takedown. If it succeeded ... well, that would be interesting.
They performed a pilot study, which involved showing subjects a photo of a woman winning a race before the subjects took part in a brainstorming task. As Bargh’s research would predict, the photo made them perform better at the brainstorming task. Or seemed to. Latham performed the experiment again in cooperation with another lab. This time the study involved employees in a university fund-raising call center. They were divided into three groups. Each group was given a fact sheet that would be visible while they made phone calls. In the upper left-hand corner of the fact sheet was either a photo of a woman winning a race, a generic photo of employees at a call center, or no photo. Again, consistent with Bargh, the subjects who were primed raised more money. Those with the photo of call-center employees raised the most, while those with the race-winner photo came in second, both outpacing the photo-less control. This was true even though, when questioned afterward, the subjects said they had been too busy to notice the photos.
Latham didn’t want Bargh to be right. “I couldn’t have been more skeptical or more disbelieving when I started the research,” he says. “I nearly fell off my chair when my data” supported Bargh’s findings.
That experiment has changed Latham’s opinion of priming and has him wondering now about the applications for unconscious primes in our daily lives. Are there photos that would make people be safer at work? Are there photos that undermine performance? How should we be fine-tuning the images that surround us? “It’s almost scary in lots of ways that these primes in these environments can affect us without us being aware,” he says. Latham hasn’t stopped there. He’s continued to try experiments using Bargh’s ideas, and those results have only strengthened his confidence in priming. “I’ve got two more that are just mind-blowing,” he says. “And I know John Bargh doesn’t know about them, but he’ll be a happy guy when he sees them.”
Latham doesn’t know why others have had trouble. He only knows what he’s found, and he’s certain about his own data. In the end, Latham thinks Bargh will be vindicated as a pioneer in understanding unconscious motivations. “I’m like a converted Christian,” he says. “I started out as a devout atheist, and now I’m a believer.”
Following his come-to-Jesus transformation, Latham sent an e-mail to Bargh to let him know about the call-center experiment. When I brought this up with Bargh, his face brightened slightly for the first time in our conversation. “You can imagine how that helped me,” he says. He had been feeling isolated, under siege, worried that his legacy was becoming a cautionary tale. “You feel like you’re on an island,” he says.
Though Latham is now a believer, he remains the exception. With more failed replications in the pipeline, Dijksterhuis believes that Kahneman’s looming-train-wreck letter, though well meaning, may become a self-fulfilling prophecy, helping to sink the field rather than save it. Perhaps the perception has already become so negative that further replications, regardless of what they find, won’t matter much. For his part, Bargh is trying to take the long view. “We have to think about 50 or 100 years from now—are people going to believe the same theories?” he says. “Maybe it’s not true. Let’s see if it is or isn’t.”
