The Liar’s ‘Tell’

Paul Ekman has spent much of his long career studying emotions as expressed on the face. So it’s fitting that two walls in the entry hallway of his San Francisco apartment are decorated with masks from around the world.

“This is a nice ‘surprise’ mask,” he says, pausing to admire a specimen from Mexico, a male visage with white skin, ruby lips, and strikingly wide eyes. But immediately he starts to quibble, pointing out details that would elude most of humanity: “The upper eye is raised a little too much for it to be pure surprise. See the white showing above the iris? This person is moving from momentary surprise into fear.”

Ekman is renowned for his ability to read faces for signs of what people are thinking and feeling. In his best seller Blink, Malcolm Gladwell writes that “much of our understanding of mind-reading” is owed to Ekman and his collaborators. He relates how Ekman, an emeritus professor of psychology at the University of California at San Francisco, could tell by their faces alone when figures as varied as Bill Clinton and Kim Philby, the infamous British spy, were lying—Clinton in real time, Philby on historical video. Lie to Me, a television show featuring a human lie detector modeled on Ekman, ran from 2009 to 2011 on Fox. His work on lying is one reason the American Psychological Association deemed Ekman one of the 100 most influential psychologists of the 20th century.

Ekman does not rely on the face alone to do what Gladwell calls mind reading, although that’s by far the best-known aspect of his work. His book Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage (Norton), first published in 1985 but updated through 2009, mentions more than 30 cues that can be helpful in detecting deceit, from “slips of the tongue” to “indirect speech” to “facial blanching.” He is much in demand as a consultant, and, at 80, satisfies most of that demand these days by offering online courses and training by surrogates through two companies, the Paul Ekman Group and Paul Ekman International. Business is brisk.

On the public-policy front, Ekman’s work helped inspire an immense federal program in American airports called SPOT, for Screening of Passengers by Observational Techniques. Costing about $200-million annually—$900-million in all since 2007—the program, run by the Transportation Security Administration, deploys more than 3,000 officers to look for behavioral cues in the faces and body language of airline passengers. Those deemed suspicious are pulled aside and, if they display more signs of duplicity during an interview, are referred to law enforcement.

But Ekman’s lie-detection work has recently taken some hard blows. He has long had academic critics (unmentioned in Blink) who say he has not proved that his behavior-based lie-detection techniques actually work. In November 2013, the Government Accountability Office took things up a notch by recommending that Congress cut the funding of the TSA program. The watchdog agency argued that neither scholarship in general nor specific analyses of SPOT offered any proof that malign intent could be divined by looking at body language or facial cues.

Plenty of academics share this negative view of SPOT. “I really don’t think the current program at TSA is doing anything to protect us,” says Charles R. Honts, a professor of psychology at Boise State University, who has consulted with the Department of Defense on behavioral observation.

This is the rare social-science debate in which lives are at stake. If the GAO is to be believed, a 10-year government investment has been a waste, and possibly a dangerous one, if it provided a false sense of security. Is the science behind the SPOT program so misbegotten that it should be abandoned? Or might it be a promising program with a few flaws? Ekman argues that the GAO failed to consider the most up-to-date and pertinent research on the subject, and that pulling behavioral-detection officers out of airports would amount to “open season for terrorists.” But his critics say the world’s most famous lie detector has been stretching the truth a bit himself, offering an exaggerated account of his findings to a credulous press and policy makers.

Ekman’s ideas about why lies are detectable from exterior appearances come by way of Freud, who once wrote, “No mortal can keep a secret. If his lips are silent, he chatters with his finger-tips; betrayal oozes out of him at every pore.” Ekman quoted the line in his first foray into lie detection, a 1969 paper in Psychiatry titled “Nonverbal Leakage and Cues to Deception,” written with Wallace V. Friesen, a UCSF research associate at the time.

At first, Ekman thought the face might be an unpromising place to look for cues. His thinking was that humans had such experience communicating with the face that natural selection would have rewarded the ability to conceal emotion. So it might make more sense to focus on things like “illustrators” (hand gestures while talking; these decreased as a liar concentrated on keeping his facts straight, Ekman found), or the even stronger “emblems” (including the Freudian slip of giving an interviewer the finger absent-mindedly). One exception in which the face proved revealing, he wrote, was “micro affect displays,” or micro-expressions, brief flashes of the emotional truth belying a false face: These lasted between one twenty-fifth and one-fifth of a second and could be detected only by the instinctively gifted or the well trained, or via slow-motion video.

In one exploratory study described in “Nonverbal Leakage and Clues to Deception,” Ekman and Friesen had students watch video of psychiatric patients, including “Mary,” who was lying to a doctor, telling him she was no longer suicidal (she had confessed the truth after the video was made). The people who watched Mary’s hands and body as she spoke were more likely than those who watched her head to identify that she was “tense,” “excitable,” and “fearful.” Yet at one point Mary also let slip a micro-expression of deep despair, Ekman and Friesen wrote. That detail was mentioned more or less in passing; no empirical evidence was provided.

Ekman’s studies into the mid-1970s would affirm his belief that laypeople could identify lies better through bodily tics and movements than through facial expressions, but his claims that trained experts could read micro-expressions grew bolder. In a 1974 study, Ekman and Friesen asked nurses to maintain a happy demeanor while looking at pleasant or distressing pictures. Again, the test subjects did better at perceiving faked reactions when they watched the body. But, the authors wrote, four experienced facial analysts each “accurately judged both the honest and deceptive behavior of almost all of the nurses.” Again, this was mentioned almost as an aside.

In parallel with his work on lie detection, Ekman was developing a compendium of every possible facial expression, dubbed the Facial Action Coding System. He used it to analyze the components of “felt” and “false” smiles. Both kinds involve turning up the corners of the lips (using the zygomaticus major muscle), but true smiles also include distinctive muscle movement around the eyes (orbicularis oculi, pars lateralis), notably a slight downward tug of the space between the eyebrow and the upper eyelid, evidently difficult to fake.

“The face is the most powerful indicator of deception,” Ekman says in the living room of his 25th-floor apartment, which offers sweeping views of the city and the bay, extending to distant Napa (and, on clear days, farther still, to Mount St. Helena). His voice is authoritative, therapeutic, only slightly weakened by age. “But it only gets you to 70-percent accuracy. That’s not a useful figure. In order to get over 90 percent you need to involve gesture, voice, and nuance of the content of speech. And we do train people in all of that.”

But some scholars say the idea that anyone could reach 90-percent lie-detection accuracy by observing behavioral cues visible to the naked eye is pure fantasy. Testifying before Congress in 2011, Maria Hartwig, an associate professor of psychology at the John Jay College of Criminal Justice, of the City University of New York, took on Ekman directly. (He also testified.) “No such finding has ever been reported in the peer-reviewed literature,” she said.

For its report, the GAO leaned heavily on a 2006 analysis of previous research, by Bella DePaulo, a project scientist at the University of California at Santa Barbara, and Charles F. Bond Jr., who has since retired from Texas Christian University. Based on more than 200 published and unpublished studies on interpersonal lie detection, dating back decades, their overall finding was that people could distinguish truth from lies by a very small degree: about 54 percent of the time.

Ekman objects that since the subjects in those studies had received no training in behavioral cues, a comparison of that number with what trained experts can do is apples to oranges. But another meta- analysis, by DePaulo and three other authors, from 2003, looked at evidence of actual behavioral cues to deception—genuine tip-offs. That paper drew on more than 115 studies and 158 supposed cues, and the results were unimpressive. Contradicting Ekman’s focus on the face and other physical indicators, the most potent cues, themselves modest, usually had to do with narrative. Liars used fewer details, displayed general ambivalence, and projected lower perceived plausibility (a vague but apparently useful criterion). A few physical characteristics did emerge as predictive of lying—a nervous demeanor, higher voice pitch—but the effect sizes were small enough to make them of dubious usefulness in the real world. Ending on a skeptical note, the authors said they thought it was unlikely that anxiety caused by lying could ever be distinguished from the anxiety caused by a false accusation of untruthfulness.

Ekman and the TSA say the meta-analysis is worthless because DePaulo lumped together studies using high- and low-stakes lies, and only the former are likely to lead to psychological strain and therefore “leakage.” “Crap in, crap out,” snaps Ekman. (In the few studies DePaulo examined that looked at whether liars were motivated to succeed, detection of high-stakes lies was no better than of other lies.)

It’s true that Ekman has consistently emphasized the importance of high stakes. The nurses in his early studies were told that their careers depended on their being convincingly untruthful, in order, for example, to protect patients or their families. In a 1997 paper, Ekman and Mark Frank, now a professor of communication at the University at Buffalo, put people in a position where they had the option of stealing $50 in cash. They were told that if they did, and subsequently convinced an inquisitor that they had not, they got to keep the money as well as the study-participation fee. If the lie was detected, however, they’d lose all the money, including the fee, and have to sit on a cold chair in a small dark space as they were blasted by white noise. (That last part of the punishment was never doled out.) In a variant of the study, subjects lied about a strongly held political view, with the same promised punishment. The results showed that people who were good at detecting one kind of high-stakes lie were also good at detecting the other. It also found that people who could recognize expressions when flashed very briefly on a screen were the best at detecting deceit.

Separately, facial coders evaluated the expressions of all participants, using videotape. Facial actions denoting fear or disgust distinguished liars from truth-tellers at a rate of roughly 80 percent, Ekman and Frank said.

Those results are hardly trivial. But several of Ekman’s critics argue that they fall well short of nailing down his central claims. There’s no evidence that the successful lie detectors were using micro-expressions to make their judgments, for example. And the facial coders did not make decisions about veracity in anything like real time; those were calculated after the fact, based on ratings the coders had provided while watching video frame by frame.

Among those frustrated by the alleged gap between Ekman and Frank’s bold assertions and the details of their papers is Nicholas Epley, a professor of behavioral science at the University of Chicago’s Booth School of Business and author of Mindwise: How We Understand How Others Think, Feel, Believe, and Want (Knopf). “That’s the consistently maddening thing about this—the claims that the data are there without me being able to find them,” he says.

Some of Ekman’s most famous examples are largely anecdotal, as in the case of “Mary,” the depressed woman who flashed a micro-expression of despair despite her claims to be feeling better.

Making it even harder to check Ekman’s data is a decision he says he has made to protect national security: not to publish all of his findings. “I got caught in a bind,” he says. “How much am I going to publish when the Chinese follow my work very closely?”

He does say some important findings are in the pipeline. In his 2011 testimony before Congress, he made a point of appending summaries of two studies he said would soon be published. Frank was the lead author on both, Ekman a co-author on the first and a consultant on the second. The first study used a mock airport gate, active or retired police officers or federal investigators as questioners, and students as passengers. The students lied about their intent to steal a check on the other side of the gate, and were caught at a 90-percent rate. Three years later, those papers remain unpublished. (Frank concedes he’s been slow in revising the work in response to questions from reviewers.)

Bella DePaulo proposes that Ekman’s charisma has helped to paper over gaps in his arguments. “I have been on panels where every other person has what I would consider better data than he does, and he’s the one who is the most compelling figure,” she says. “He starts talking and you can’t stop listening.” It’s often a balancing act for psychologists to tell a clear story about their work while acknowledging the caveats and “messiness” of social science. But DePaulo says, “When you hear Ekman talk, all you hear is the good, compelling, terrific story.”

Because of SPOT’s secrecy, it’s hard to tell how much the program leans on Ekman’s work. He is certainly one of its intellectual godfathers, and when the Government Accountability Office asked the TSA for data to defend its work, the watchdog agency said, the TSA replied with references to numerous Ekman papers. The GAO, however, pointed out that although some officers had undergone training on facial micro-expressions and other Ekman lie-detection techniques, micro-expressions had not actually made the cut of cues that officers watched out for. How many of the other cues are drawn from his work? Impossible to say, since the criteria are secret. The list, which once included nearly 100 behaviors, is being edited for efficiency and effectiveness, the TSA says.

On the ground, the man most responsible for SPOT is an entrepreneurial TSA agent named Carl Maccario, based at Logan International Airport, in Boston, who was certainly interested in Ekman’s work. He was frustrated by the focus on procedures and banned objects—penknives, nail clippers—that characterized the first months of airport security after 9/11.

It drove Maccario nuts that people might look anxious or shifty in line, and yet the policy was to leave them alone so long as their baggage went through scanners cleanly. He connected with law-enforcement officers who shared his views and contacted Ekman, who flew to Boston to watch passengers, pro bono.

Kip Hawley, a former head of the Transportation Security Administration, tells Maccario’s story in his memoir, Permanent Emergency. (Maccario, who the TSA says is no longer part of the SPOT program, did not return a phone message.) When Hawley visited Logan, Maccario buttonholed him, and Hawley wound up inviting him to Washington to help set up the program nationally. By 2007, it had spread to 42 airports, and by 2012, roughly 3,100 behavioral-detection officers were working in 176 of the more than 450 airports the TSA oversees.

The Department of Homeland Security hasn’t given up the fight. A senior TSA official, Sarah Tauber, said SPOT’s strength lay in its “threat agnostic” nature: It can snag people with evil intent regardless of the technology or tactics they intend to use. The DHS budget request for next year does not include a reduction in funds for SPOT.

The program depends on two premises: First, that behavioral cues exist, and second, that people can be trained to identify them. What does the research say about training? A recent large meta-analysis, by Christian A. Meissner, a professor of psychology at Iowa State, and three co-authors found what they described as “small to medium” benefits: Training might bump accuracy to 60 or 70 percent, from 54 percent. Echoing DePaulo’s findings from a decade ago, however, this study of studies found that training people to analyze the content of speech was more effective than having them look for Ekman-style tip-offs. Indeed, Meissner and his co-authors suggested that lie-detection coaches should basically ignore nonverbal cues.

Despite his skepticism about physical cues, however, Meissner takes a middle position on SPOT itself. Physical cues may not be of much help in detecting lies, he says in an interview, but they may not be a bad way to identify people who could stand to be asked a few questions—at which point a good screening system would switch to analyzing the content of the answers. “Maybe a person is late for their plane, and anxious,” he says. “Maybe it’s hot outside and they ran to the plane, and they are sweating. I can imagine that a TSA agent could say, ‘Hey, you look out of breath, is everything OK?’ "

That description of how SPOT should work mirrors almost exactly Ekman’s own. The GAO, however, found that things did not go so smoothly. Interviews with behavioral-detection officers revealed that many thought the guidelines for pulling someone aside were subjective. And rates of referral by behavioral-detection officers to law enforcement varied significantly by airport, suggesting more than a smidgen of arbitrariness in the standards.

The American Institutes for Research, a nonprofit organization hired by the TSA to evaluate the program, produced a report that remains a point of contention. Only a few details from the report have been made public. It compared the results, at 43 airports, of interviews initiated by behavior-detection officers and interviews with randomly selected passengers. The interviews initiated by officers were many times more likely to end in arrest than those in the control group, the researchers found—nine times more likely, by one estimate.

But the GAO points out, among other problems, that the study was not “blind": Many officers knew which passengers had been selected randomly. Tracy Costigan, a psychologist who oversaw the study for AIR, says the limitations were pointed out in the study itself and were considered acceptable by peer reviewers.

Ekman is a great fan of the study, calling it the largest field examination of deception detection yet attempted, and has urged the TSA to publish it. And he thinks that the National Academy of Sciences should convene an independent panel to analyze this “record breaking” analysis, envisioning that SPOT will be vindicated.

And what about those micro-expressions, so extolled in Blink? Until recently, Ekman and a few researchers in his orbit were just about the only people to have explored them. But there’s been a resurgence of interest, led chiefly by Stephen Porter, a professor of psychology on the University of British Columbia’s Okanagan campus, and Leanne ten Brinke, a postdoctoral fellow at the University of California at Berkeley’s Haas School of Business. Their research has buttressed Ekman’s work in some ways, undercut it in others.

In a 2008 study, they videotaped students as they adopted happy, sad, neutral, or disgusted expressions while looking at neutral, happy, or sad photos. The authors did identify some partial micro-expressions on the upper or lower part of the face that were inconsistent with the intended expression. But in a blow to Ekman’s theory, these happened almost as often when people were displaying their true feelings as when they were not. Inconsistent expressions seemed to be universal. Also, the authors found no complete micro-expressions that conformed to Ekman’s definition—covering the full face, lasting one twenty-fifth to one-fifth of a second. The expressions they spotted lasted longer.

A 2012 paper, however, looked at real-life cases of people pleading on behalf of a missing relative—half were later found to be responsible for the disappearance—and found more-robust evidence that micro-expressions were giveaways. Fleeting expressions provided tip-offs 70 percent of the time. “Brief emotional expressions can definitely reveal covert emotions, but not in the manner Ekman proposes,” Porter says by email. Ekman now admits it was a mistake to define micro-expressions by duration, and describes them now as any facial expression that is very difficult to detect in real time.

Ekman declines to discuss his portrayal in Blink, but he wants to dispel some general misconceptions about his work—for example, that he relies on any single expression to tell that a person is lying, or that he makes his judgments quickly. Micro-expressions and the full range of cues have to be considered in the full context of what a person is saying, Ekman says, and within the full context of a situation.

In recent years, in the late autumn of his career, Ekman has broadened his interests. He has struck up a friendship with the Dalai Lama that has resulted in two books, including Emotional Awareness (Times, 2008).

He plans to write at least two more books, including a scientific autobiography, and he has another project in the works funded by the Dalai Lama: online tutorials to help people better understand their emotions. Asked what it feels like to continue to face criticism of one of his life’s chief projects, he says, “I don’t pay attention to it anymore. … One of my most prominent critics once told me, ‘I love your work, but the only way I can get tenure is by attacking a famous person. So I don’t take it seriously.’ " He calls that attitude “despicable.”

“I’m trying to describe nature,” Ekman says, “and to provide people with tools that will help them in their lives and their jobs.”

Christopher Shea is a contributing writer for The Chronicle.