Magical Thinking on Positive Psychology

Recently, The Chronicle Review published “Positive Psychology Goes to War,” a long article adapted from my book, The Quick Fix: Why Fad Psychology Can’t Cure Our Social Ills. In it, I provided some history about the development and implementation of Comprehensive Soldier Fitness, or CSF, a major Army program designed to prevent PTSD and suicide among soldiers, based in large part on the tenets of positive psychology. Both the article and the book chapter from which it was adapted were highly critical of Martin Seligman, the esteemed University of Pennsylvania psychologist who is a founder of positive psychology and the driving force behind CSF. I argued that he and his Positive Psychology Center have, at times, oversold the evidence underpinning certain positive-psychology interventions, perhaps most importantly the Penn Resilience Program, which came to form the basis for a major component of CSF.

I read Seligman’s response to my article with interest. Unfortunately, it confirmed my view that he is sometimes imprecise when it comes to backing up his claims — and that he sometimes stretches scientific evidence in self-serving ways.

Seligman writes that I claim “that the prevention of PTSD was the overriding goal” of CSF. But, he insists: “It was not. CSF’s goal was to ‘enhance resilience and coping skills,’ enabling soldiers ‘to grow and thrive,’ as opposed to focusing on pathology. This was in keeping with the theme of positive psychology.”

In both my book and article, I indeed write that a major goal of CSF was the prevention of both PTSD and suicide in soldiers, with the suicides the Army sought to prevent mostly being ones associated with PTSD. I really don’t think it’s credible to claim otherwise.

Take, for example, the very CSF mission statement Seligman quotes from. Earlier in his argument, he does so more fully: “CSF marks a new era for the Army by comprehensively equipping and training our Soldiers, Family members and Army Civilians to maximize their potential and face the physical and psychological challenges of sustained operations. We are committed to a true prevention model, aimed at the entire force, which will enhance resilience and coping skills enabling them to grow and thrive in today’s Army.” Of course the idea of a “true prevention model” flies in the face of Seligman’s claim that the program didn’t seek to focus on pathology. What was it trying to prevent, if not psychopathologies such as PTSD and the tragedies that result from them, like suicide?

This is far from the only time those who designed and/or boosted CSF said that its goal was the prevention of PTSD or suicide or both. A 2011 article published in American Psychologist, for example, argued that “the preventive effects of the PRP on depression and anxiety are relevant to one of the aims of the MRT course, preventing posttraumatic stress disorder (PTSD), since PTSD is a nasty combination of depressive and anxiety symptoms.” One of the authors is Seligman.

Or take Gen. George W. Casey’s testimony before a Senate appropriations subcommittee in 2010. “That’s why we instituted the Comprehensive Soldier Fitness program,” he told the subcommittee, “to give the soldiers and family members and civilians the skills they need on the front end to be more resilient and to stay away from suicide to begin with. It’s a long-term program, but I think that is the only way that we are ultimately going to begin to reduce this.”

Then there’s Flourish: A Visionary New Understanding of Happiness and Well-Being, which is written by Seligman himself. It contains a passage on the origins of CSF in which Seligman relates the time that Casey, declaring he had no interest in a pilot study, “thundered”: “We are satisfied with [CSF], and we are ready to bet it will prevent depression, anxiety, and PTSD. This is not an academic exercise, and I don’t want another study. This is war. General [Cornum], I want you to roll this out to the whole Army.”

It seems like a lot of important people in and around the Army somehow came to believe that a key goal of Comprehensive Soldier Fitness was the prevention of PTSD and suicide. (Seligman repeats this story in his article in the Chronicle — but he subtly alters the language of the anecdote, omitting the mention of “depression, anxiety, and PTSD.”) This was the stated goal — stated over and over. I don’t know what Seligman means by an “overriding” goal rather than just a “goal,” but the historical record is crystal clear: The Army reached out to Seligman because it had a PTSD and PTSD-related suicide crisis on its hands, and Seligman, at the very least, disseminated the claim that CSF could potentially prevent PTSD and PTSD-related suicide.

In Seligman’s current version of events, the Army, in the midst of a PTSD and suicide crisis, decided it needed to immediately roll out a program — so urgently there wasn’t time for pilot-testing — that wouldn’t address these issues. A program that was “opposed to focusing on pathology,” and that was untested in an adult military context, and that was to be rolled out to every single soldier in the Army. Why? Again, it’s not a credible claim.

“It is odd,” Seligman writes, “that Singal did not present interviews with any of the thousands of soldiers trained in Comprehensive Soldier Fitness. It is odd because my personal impression is that they loved the program.” Seligman continues: “The spontaneous written comment ‘this is the single best course I have had in 20 years in the Army’ was frequent. The comment ‘if I had had this training years ago, it would have saved my marriage’ stands out for me.”

If we’re going to present anecdotes, I have some of my own. It is true that the Army has produced various PR materials in which soldiers claim that CSF has been great or even life-changing. But at least some soldiers view CSF as a chore or a PR gimmick they need to grind their way through.

That, at least, was the conclusion of Emily Sogn, whose doctoral dissertation was on CSF, and who interviewed many participants. Most of the issues, Sogn told me, stemmed from how soldiers viewed the global assessment tool (GAT), the mandatory survey that soldiers need to fill out as part of CSF. “Soldiers are often very flip, very casual, sometimes very hostile about the effectiveness of these surveys,” she said. “They’re often very strategic about how they take these surveys.” This feeling extended up the command chain. “Mostly what I heard in terms of the kind of command climate was that there was a casualness, that the soldiers sensed that it wasn’t very important and that they could do it fairly quickly — and that what they were doing was just getting it done, not that it was something they were supposed to kind of reflect on and treat in a sincere way,” Sogn explained.

The sentiment that appeared in Sogn’s interviews would later be echoed back to me in a much more colorful way by a retired service member who had been a master resilience trainer, and who had to juggle those responsibilities as he was also training hundreds of soldiers on heavy, complicated equipment in hundred-degree weather in Kuwait in anticipation of getting deployed to Iraq: “The MRT [master resilience training] kept popping up as a fucking requirement. The goddamn MRT kept popping up.”

One of my main critiques of Seligman’s approach is that, when it comes to promoting his work and the work of other positive psychologists, he overgeneralizes and overclaims on the basis of rather weak or narrow or only borderline-relevant research findings. For instance, in response to my assertion that the evidentiary base for CSF and the Penn Resilience Program was weak, he shifts the goalposts, arguing that there is evidence for positive-psychology interventions more broadly, citing a metastudy published last year in The Journal of Positive Psychology. But that’s a different question: I don’t believe I ever claimed that there’s no evidence for such interventions. Moreover, more than half of the studies in this meta-analysis came from self-help or individual-level settings, which are very different from the group setting in which the meat of CSF, the master resilience training sessions, are delivered.

Even if we assume this meta-analysis is relevant to the present dispute, in certain ways it supports my point that interventions that (at least arguably) work in one setting might not translate to the setting in which CSF takes place. In championing the 2020 study, Seligman quotes more or less verbatim from the article’s abstract. But he neglects to quote a crucial following sentence: “Individuals in non-western countries with clinical problems, who engaged in longer individual or group therapy programs containing multiple [positive psychology interventions] benefited most.” Since CSF is a preventive program for a general population, not a therapy program for an afflicted population, and since its participants are (obviously) from a western country, it stands to reason that these already-modest effects might be even less impressive in a CSF-like context.

This distinction between preventive contexts (for general populations) and therapeutic contexts (for those already experiencing symptoms, or who have already gotten a diagnosis), is crucial to my argument. Seligman tends to ignore it. And that’s setting aside the large step between this research and CSF — reducing anxiety and depression symptoms is not the same as preventing PTSD and suicide, and programs geared at at the former might have little to no impact on the latter.

After explaining that he was unmoved by the first of a series of non-peer-reviewed evaluations conducted by the Army due to its methodological weaknesses, Seligman claims a second study as a relatively impressive evaluation of CSF:

The second study, in contrast, reported diagnostic outcomes. The evaluators found that soldiers who had undergone master resilience training had better diagnostic outcomes. Diagnosed substance abuse subsequently occurred in 1.16 percent of master resilience-trained soldiers, as compared with 2.85 percent of soldiers in the untrained control. MRT halved diagnosed substance abuse. This was a substantial, clinically meaningful difference.

The same study showed that 4.44 percent of soldiers in the resilience-trained group were later diagnosed with PTSD or panic disorder or depression, as compared with 5.07 percent of the controls.

These last differences were small but statistically significant — and, projected to the 1.1 million soldiers then in the Army, they are meaningful. With master resilience training, we might anticipate 16,900 fewer cases of substance abuse and 6,300 fewer cases of PTSD, panic, and depression. This comes out to savings many times the cost of the entire program, to say nothing of the immense amount of suffering prevented.

Master resilience training “halved diagnosed substance abuse” — that is a very bold causal claim, given that it comes from a non-peer-reviewed study. Seligman further argues that I “omit[ed] any mention of these data,” as though I’m trying to hide the evidence of CSF’s success. That’s false. In fact, I quoted from an Institute of Medicine report that came away unimpressed by CSF, and which mentioned this exact study: “In addition, the institute’s report notes, the one attempt the Army made to evaluate CSF on the basis of actual diagnoses among service members found ‘no difference in diagnosis among those receiving the [CSF] intervention’ and those who had not participated in it.”

I plucked that “no difference in diagnosis” bit from an excerpt that reads, in full, “Findings revealed no change in the GAT factors and no difference in diagnosis among those receiving the intervention. Therefore, the subsequent mediation analysis performed by the authors cannot be interpreted as evidence of intervention/program impact.” Or, as the psychology researcher and data sleuth Nicholas J.L. Brown put it in his invaluable article in The Winnower:

Although the report by Harms et al. runs to 24 pages, excluding appendices, and reports the analysis methods used in considerable detail, the reader will search in vain for an answer to the straightforward question of whether the two groups differed significantly on the outcome variables, as listed in Table 3 on p. 17. A few moments with a chi-square calculator reveals that for mental health problems, there was no significant difference between the “resilience training” and “no-training” groups, χ² (1, N = 7,230) = 1.428, two-tailed p = .23. For substance abuse, the difference was statistically significant, χ² (1, N = 7,230) = 26.482, two-tailed p < .0001, but the effect size was extremely small (Φ = 0.06)

If this is the best Seligman can do, it is very weak tea. In my newsletter, I respond at more length to Seligman’s claims about other meta-analyses. The short version is that on multiple occasions Seligman points readers toward broader examinations of positive-psychology interventions that don’t show what he wants them to show about the Penn Resilience Program specifically. That is the program under scrutiny here, because it was the one adapted to form a core component of Comprehensive Soldier Fitness.

Finally: I can’t resist responding to Seligman’s claim that the Army “asked for my help, and I have given it to them on a pro bono basis for many years.” Seligman’s Positive Psychology Center has been the recipient of many millions of military dollars. That Seligman himself apparently doesn’t charge the military for his services doesn’t change the fact that he and the academic institution he birthed have clearly benefited, from both a material and a public-relations standpoint, from this relationship. This isn’t exactly a partner at a white-shoe firm devoting some free hours to a neighborhood legal-services office.