College students’ levels of clinically significant mental-health symptoms have doubled during the last decade, with more than 60 percent of students meeting the criteria for one or more psychological problems. Counseling centers have been overwhelmed by demand, and mental-health apps have helped pick up the slack.
But how effective are those apps? How many students are using them, and for how long? Good questions — but the answers are scarce.
Sarah K. Lipson, an associate professor of health law, policy, and management at Boston University, says there is a remarkable lack of evidence about the use of digital mental-health interventions and their effectiveness. With an estimated 10,000 to 20,000 such offerings in the health-tech market today, many of which are targeted toward higher education, college administrators are often overwhelmed trying to pick one.
A team of seven researchers hopes to start changing that with a new report, released Thursday. The authors review in some detail available evidence for nine of the digital mental-health interventions most often purchased by colleges for their students. In many of those cases they found little, no, or outdated evidence of effectiveness. More broadly, the researchers analyzed services at 200 randomly selected colleges and interviewed administrators and other experts at 20 institutions. In that wider look, too, the researchers found surprisingly little evidence on the apps’ use and effectiveness.
The authors hope the report will help guide administrators as to “what they should be asking, even demanding, from these companies,” says Lipson. The authors also hope the report will spur contributions by companies and colleges to a shared, public data bank the researchers hope to set up within about a year.
The authors recommend that colleges consider how best to align the digital tools with a college’s existing counseling services, suicide-prevention and peer-support initiatives, and other wellness programs.
In issuing the report, says Daniel Eisenberg, a professor of health policy and management at the University of California at Los Angeles, “I see our role as trying to set the bar as high as we reasonably can, because we think these programs are so important.” That’s because, for all the murkiness and gaps in the data, the apps show promise.
The authors cite a 2019 review of 89 digital mental-health services in higher education that showed that “47 percent were effective at improving all primary outcomes,” measured by clinical-symptom scores, “while 34 percent were partially effective.” The 24/7, on-the-go convenience of the apps is popular. One survey, the report notes, showed that seven out of 10 respondents ages 14 to 22 had used a mobile health app, often for sleep or stress problems, or for meditation.
The apps are also scalable to meet colleges’ high demand for help. Services with clinicians across the country, and sometimes around the world, help solve state-licensing obstacles for students during vacations, internships, study abroad, or travel for athletics. They also provide services in a range of languages and with clinicians representing a variety of cultural and ethnic backgrounds.
Several representatives of digital mental-health companies who were familiar with the research praised the report’s general goals, even if they had reservations about some aspects of its industry critique.
“What they’re doing is very important and I 100-percent support it,” says Ben Locke, chief clinical officer for the online peer-support platform Togetherall. “This is a first run at a very, very difficult problem, and we just need to acknowledge that and work toward more accurate analyses and comparisons over time.”
He has some qualms, however. One is that although the study excludes apps that focus primarily on traditional one-on-one clinical therapy sessions, Locke says that there is still a huge range of services between and within the platforms that the report does consider. The companies the report examines might offer a crisis hotline, peer support, self-guided mindfulness or behavior-therapy modules, asynchronous coaching, or combinations of those and other approaches.
So, says Locke, the researchers are really comparing “apples, oranges, and mangos, and probably a potato.”
In addition, some online mental-health services, Locke says, have exclusionary clauses and therefore don’t serve students with histories of severe mental-health problems or who have major eating or other disorders. Comparing outcomes from those services against more-inclusive platforms is, again, he says, an apples-to-oranges exercise.
The authors’ focus on engagement and student adoption of services is also problematic, says Locke. It’s true, as the report’s authors note, that a student might in many cases only use an app once. But that might be all the student needs to resolve an issue or to be guided to other resources. And the same usage patterns are true of counseling centers, which are often used by students only once.
Carla D. Chugani, vice president of clinical programs for Mantra Health — which offers students a range of services, including self-guided modules, emotional wellness coaching, and telepsychiatry — also applauds the study’s overall goals.
Chugani points out, however, that evaluating programs aimed at prevention of suicide and other self-harming behaviors is tricky. “It’s very hard to prove that you prevented something from occurring,” she says. She urges colleges, companies, and researchers to work together on studies of effectiveness. “Companies who are really interested in doing what’s right for their students will be the ones who will welcome this participation,” she says.
“We really applaud the efforts of anyone in the industry to take a deeper look at what’s beyond the curtain,” says Ed Gaussen, Mantra’s co-founder and president.
UCLA’s Eisenberg and Boston University’s Lipson say that the report is just a first phase. While the research team seeks to bring more shared data to colleges’ digital-wellness efforts, it next plans to better measure the effectiveness of campus mental-health training and peer-support programs.
“Transparency,” says Lipson, “will go a long way in starting to inform decision making.”
In addition to Eisenberg and Lipson, the report’s authors include Sara Abelson from Temple University, Ashley Johnston from UCLA, Michelle Liu and Stephen M. Schueller from the University of California at Irvine, and Shannon N. Ogden from Boston University. The report was commissioned by the Ruderman Family Foundation and compiled in partnership with the Healthy Minds Network, Temple University’s Hope Center for Student Basic Needs, and Boston University.
