In 2008 and 2009, in the wake of combat in Iraq and Afghanistan, the United States Army had a mental-health crisis on its hands. Soldiers were afflicted by anxiety, depression, PTSD, drug abuse, divorce, and suicide.
In response, the Army announced the creation of Comprehensive Soldier Fitness. Here is its mission statement: “CSF marks a new era for the Army by comprehensively equipping and training our Soldiers, Family members and Army Civilians to maximize their potential and face the physical and psychological challenges of sustained operations. We are committed to a true prevention model, aimed at the entire force, which will enhance resilience and coping skills enabling them to grow and thrive in today’s Army.”
The program taught soldiers coping skills that had been shown to be beneficial in prior research. These skills included disputing catastrophic thoughts, putting bad events in perspective, active constructive responding, gratitude, and building signature strengths, among others. CSF was created by assembling these exercises into one package.
In an endorsement of an article by Jesse Singal, Len Gutkin suggests that positive psychology, which, along with cognitive-behavioral therapy, is at the heart of Comprehensive Solder Fitness, is “positively BS.” Gutkin describes the components of the program, which The Chronicle dubs “evidence free,” as a “series of dubious treatments for combat trauma.” For his part, Singal’s argument suggests that the interventions from the CSF curriculum depend on evidence that is very weak to nonexistent.
Singal makes three central charges: 1) Positive-psychology interventions are ineffective, 2) Such interventions are not relevant to PTSD, and 3) the evidence for the success of the Penn Resilience Program is nonexistent. (The Penn Resilience Program is a package of positive-psychology interventions, many derived from cognitive-behavioral therapy, intended to increase resilience to adversity and raise well-being. ) All three of these charges are false.
Singal selectively picks his way through the literature on the effectiveness of positive-psychology interventions, telling the reader only about negative evidence. He fails to share the scores of controlled studies showing that these interventions work. Thankfully there is a method, meta-analysis, for evaluating all existing studies together. Singal does not tell the reader about the most comprehensive meta-analysis.
In 2020, The Journal of Positive Psychology published the meta-analysis of the positive-psychology interventions, or PPIs, that went into the CSF package. The authors reviewed 347 studies involving over 72,000 participants from clinical and nonclinical child and adult populations in 41 countries. The effect of PPIs, with an average of 10 sessions over six weeks, offered in multiple formats and contexts, was evaluated. The positive-psychology interventions had a significant small to medium effect on well-being (g = 0.39), strengths (g = 0.46), quality of life (g = 0.48), depression (g = −0.39), anxiety (g = −0.62), and stress (g = −0.58). (G is the effect size; 0.39 is midway between small [0.20] and medium [0.5]). Gains were maintained at three months’ follow-up.
The evidence, contra Gutkin and Singal, is not BS. It is massive, state-of-the-art, and frequently replicated.
The lay reader may not be familiar with the terms “small,” “medium,” and “large” to describe effect sizes. An effect size is the mean difference between two populations divided by the standard deviation of the whole population. These are statistical descriptors of the percentage of people who benefit from a treatment versus a control. Effects in therapy are usually “small” or “medium” — we celebrate when “medium” occurs, and large effects of medications or psychotherapy are rare. And effect sizes in prevention are small (very rarely medium), or more usually nonexistent. So, a “small” effect size in the prevention of psychological problems is not a pejorative. It is a good result; the best that can be anticipated.
This meta-analysis is state-of-the-art, and it flatly contradicts characterizations of an “evidence free” “series of dubious treatments,” and Singal’s claim that positive psychology is “controversial.” There is extensive, replicated evidence that PPIs are effective against anxiety, depression, and stress, and that they increase well-being.
Why did the Army choose to use positive-psychology interventions in the creation of Comprehensive Soldier Fitness? Evidence like the above suggested that positive interventions adapted from well-documented cognitive behavioral therapy increased resilience; raised coping skills; and lowered depression, stress, and anxiety. The Army reasoned that these interventions would be useful to improve thriving and growth, as well as to reduce depression, anxiety, and stress, frequent problems after combat trauma. The Army chose this course in August 2008. They asked for my help, and I have given it to them on a pro bono basis for many years.
The reader might have the impression from Singal’s article that I was trying to rush the Army into a full program without pilot work. The reverse is true. When the Army proposed setting up a prevention program, I advocated explicitly for controlled pilot studies before going ahead. I was overruled.
Brig. Gen. Rhonda Cornum and I presented our proposal for prevention to the general staff at the Pentagon in February 2009. We wanted to go slowly and carefully, and we suggested running a pilot study with 100 drill sergeants and 5,000 soldiers. (Cornum, who retired from the military in 2012, is a physician and holds a doctorate in biochemistry and nutrition.)
That proposal was rejected by Gen. George Casey, chief of staff. “We are at war, General Cornum. I don’t want a pilot study. I want you to teach and measure resilience and positive psychology to the whole Army. Now. Move out, soldier.”
General Cornum and I then planned to get controlled data on whether Comprehensive Soldier Fitness training worked: untreated control groups and random assignment to training or no training. We originally designed the rollout of the training so it would be gradual. We would have 10,000 soldiers undergo resilience training and 10,000 controls without resilience training. A nice, clean experiment.
But the reality of war defeated us. The soldiers’ initial response to this course was so overwhelmingly positive that the generals sped up the rollout, wiping out our neat control group.
So, there were only two explicit evaluations done with the untrained control group, both of which were done early, before the program was rolled out to everyone. They were not peer-reviewed, because the Army uses independent evaluators rather than academic peer review.
The first looked at subjective self-reports of depression, coping, catastrophizing, friendship, and stress, among other outcomes. The evaluators reported statistically significant, small beneficial effects, and concluded that the “skills are having a positive effect on Soldier self-reported resilience.” I was not impressed by this report because, statistical significance aside, effect sizes were probably tiny, and the outcomes depended so heavily on mere self-reports.
The second study, in contrast, reported diagnostic outcomes. The evaluators found that soldiers who had undergone master resilience training had better diagnostic outcomes. Diagnosed substance abuse subsequently occurred in 1.16 percent of master resilience-trained soldiers, as compared with 2.85 percent of soldiers in the untrained control. MRT halved diagnosed substance abuse. This was a substantial, clinically meaningful difference.
The same study showed that 4.44 percent of soldiers in the resilience-trained group were later diagnosed with PTSD or panic disorder or depression, as compared with 5.07 percent of the controls.
These last differences were small but statistically significant — and, projected to the 1.1 million soldiers then in the Army, they are meaningful. With master resilience training, we might anticipate 16,900 fewer cases of substance abuse and 6,300 fewer cases of PTSD, panic, and depression. This comes out to savings many times the cost of the entire program, to say nothing of the immense amount of suffering prevented.
Singal omits any mention of these data, even as he concentrates his attack on the lack of evidence that positive-psychology interventions prevent PTSD. He asserts that the prevention of PTSD was the overriding goal of Comprehensive Soldier Fitness. It was not. CSF’s goal was to “enhance resilience and coping skills,” enabling soldiers “to grow and thrive,” as opposed to focusing on pathology. This was in keeping with the theme of positive psychology, which is that increasing well-being is valuable in its own right and can buffer against pathology.
Anxiety and depression were salient problems for the Army, along with PTSD and suicide, and I knew from our studies that we could very likely prevent anxiety and depression with PPIs. Preventing anxiety and depression is a good end in itself for the Army. And preventing PTSD is an important ongoing research goal for me personally. But preventing PTSD was not a core goal of Comprehensive Soldier Fitness. Throughout all our years working with CSF, we repeatedly published the following disclaimer: “The Penn Resilience Program is not intended for trauma or crisis. This strengths-based prevention program is designed to build resilience and well-being by teaching practical skills to navigate adversity and to grow and thrive in” a soldier’s “professional life.”
It is nevertheless worth looking at positive psychology’s and CSF’s relation to PTSD, since positive-psychology interventions do prevent many of the symptoms of PTSD.
PTSD contains a bulk of depressive, stress, and anxiety symptoms in addition to others. This is the reason that investigators like P.D. Harms et al. (2013) combined the diagnoses of PTSD with diagnosed panic and diagnosed depression. And this is indeed why the 347 meta-analyzed studies of Alan Carr et al. (2020) show that PPIs are not “a series of dubious treatments” but have significant effects reducing depression, anxiety, and stress. This is highly relevant to why CSF training might prevent PTSD.
Singal fails to mention a recent predictive study, of which I am the lead author, that is highly relevant to the prevention of PTSD. Using the Global Assessment Tool — a test that Singal dismisses without bothering to review the evidence justifying its use — we attempted to predict who would develop PTSD after combat in Iraq or Afghanistan between 2009 and 2013. We studied the complete cohort, not a mere sample. About 5 percent of these soldiers developed diagnosed PTSD. Soldiers who were worst on GAT catastrophic thinking were 29 percent more likely to develop PTSD than soldiers with average catastrophic thinking, whereas soldiers lowest on the catastrophic-thinking scale were 25 percent less likely to develop PTSD. Soldiers high in catastrophic thinking and experiencing high combat intensity were 274 percent more likely to develop PTSD than those low on both. This suggests a major way to prevent PTSD: Keep catastrophizers away from intense combat. Reducing catastrophization is an explicit target of positive-psychology interventions — and this study showed that reducing it would likely prevent PTSD.
Recall that the explicit aim of CSF was to improve the growth and thriving of soldiers. Another relevant study we conducted, now under review, tried to predict heroism and exemplary job performance over the course of four years. The Global Assessment Tool-measured high positive affect, low negative affect, and high optimism predicted awards for performance and awards for heroism in a sample of 908,096 soldiers, in which 114,443 soldiers (12.6 percent) received an award. These GAT variables predicted almost fourfold greater awards. This showed that the variables that PPIs improve are major predictive factors for exemplary Army job performance and for battlefield heroism.
Statistics aside, it is odd that Singal did not present interviews with any of the thousands of soldiers trained in Comprehensive Soldier Fitness. It is odd because my personal impression is that they loved the program. By the end of the last day of training, three-quarters of the trainers (who first take the course material and then learn how to become trainers) volunteered for the next level of teaching. Their modal ratings of the whole program were 4.8 and 4.9 out of 5.0, and the spontaneous written comment “this is the single best course I have had in 20 years in the Army” was frequent. The comment “if I had had this training years ago, it would have saved my marriage” stands out for me.
Singal concentrates his attack on the fact that some of the evidence that the Army evaluated in adopting the Penn Resilience Program came from children and adolescents in school. This was too remote from adult soldiers in combat, Singal claims, even though the targets of PRP are preventing anxiety and depression
Singal tells the reader about one early meta-analysis that concluded that no evidence was found that these programs reduce anxiety and depression, and another that has both positive effectiveness evidence and no effectiveness evidence, depending on the variable analyzed. So he indicts me for a “tendency to overclaim.” But he fails to tell the reader about the three more complete and more recent meta-analyses that clearly show that these programs work.
The most recent and most comprehensive meta-analysis, published in 2020 in the Journal of Affective Disorders, finds these programs to be effective. The authors reviewed 38 controlled studies, including 24,135 individuals. At postintervention, the mean effect size was significant, and subgroup analyses revealed significant effect sizes for programs administered to both universal and targeted samples, programs both with and without homework, and programs led by teachers. The mean effect size was maintained at six months’ follow-up, and subgroup analyses indicated significant effect sizes for programs administered to targeted samples, programs based on the Penn Resiliency Program, programs with homework, and programs led by professional interventionists.
Similarly, a 2015 meta-analysis in The Journal of Primary Prevention examined 30 peer-reviewed, randomized or cluster-randomized trials of universal interventions for anxiety and depressive symptoms in school-age children. There were small but significant effects regarding anxiety and depressive symptoms as measured at immediate posttest. At follow-up, which ranged from three to 48 months, effects were significantly larger than zero regarding depressive (but not anxiety) symptoms.
A 2017 meta-analysis in the Journal of the American Academy of Child and Adolescent Psychiatry reviewed 49 studies. For all trials, resilience-focused interventions were effective relative to a control in reducing four of seven outcomes: depressive symptoms, internalizing problems, externalizing problems, and general psychological distress. For child trials (meta-analyses for six outcomes), interventions were effective for anxiety symptoms and general psychological distress. For adolescent trials (meta-analyses for five outcomes), interventions were effective for internalizing problems.
Singal ends rhetorically, with a story in which Patricia A. Resick, a respected PTSD investigator, loses a hypothetical Comprehensive Soldier Fitness contract to an “overheated story-teller” who alleges that he has quick fix for PTSD. There is no quick fix for PTSD, and I have certainly never suggested that positive psychology is a quick fix.
Why would a CSF contract not have gone to Resick? The CSF contract search was not at all about PTSD. Rather, what was called for, according to the contract between the Army and Penn, was a program “to develop an Army resiliency training program that is built on concepts taught as part of an existing resiliency program developed for teachers … emphasizing training for positive psychology skill sets including flexible, accurate thinking, ‘de-catastrophizing’ … problem solving, taking initiative, and coping with difficult situations and emotions.” We had exactly such a program, well-documented. That is why Penn was chosen.
Singal’s three charges are grossly false. Positive-psychology interventions are highly effective, they prevent anxiety and depression, and therefore perhaps PTSD, and the Penn Resilience Program is supported by lots of scientific evidence. This evidence reliably shows reductions in depression, anxiety, and stress and increases in well-being in adults and children. These findings emerge from many studies, in many settings including the military, and with very large samples. This evidence, contra Gutkin and Singal, is not BS. It is massive, it is scientifically state-of-the-art, and it has been replicated frequently.
I wanted the Army to undertake extensive pilot work and to do controlled outcome studies of CSF, but understandably the Army felt that outcome studies were not a high priority in the middle of two wars. That said, judging from all the evidence, the Army made a reasonable judgment — and the programs created have very likely relieved the suffering and raised the mental health of American soldiers.
