Massive review shows ‘science’ behind most mental health apps is hugely flawed


The Covid-19 pandemic damaged the mental health of countless Americans. To make matters worse, the necessary home care measures hinder access to care. In the meantime, some people have turned to mental health apps for solace: some claim to provide everything from cognitive behavioral therapy to guided meditation. But a new study published in Digital Health PLOS finds that these apps aren’t backed by the rigorous evidence that their claims require – they don’t seem to work very well either.

What’s up – The meta-review examined 145 studies of telephone interventions in mental health. Taken together, the authors say the methods used to study these applications are flawed. The research “did not find compelling evidence to support a cellphone-based intervention on any outcome,” the report said.

“Across the literature, we found a general trend of weakening evidence…and decreasing effect sizes as the comparison condition became more stringent.”

In general, applications were more effective than no treatment at all. But their usefulness declined when compared to other mental health treatments or interventions. This was true whether the apps targeted anxiety, stress, feelings of depression, tobacco or alcohol addiction, or general well-being.

“If you ask me for my personal opinion, from reading the literature, I think there’s pretty strong evidence, in fact, that these apps provide on average low to moderate benefits compared to nothing,” Simon Goldberg, assistant professor in the Department of Counseling Psychology at the University of Wisconsin-Madison, says Reverse.

Why is it important – Mental health apps are a hot market. There are between 10,000 and 20,000 such apps available for download on your smartphone, according to expert opinion. These include everything from mood logs to quit smoking programs to one-on-one text messaging with a licensed clinical therapist.

It’s a big deal: Ads for Talkspace and BetterHelp are inevitable, especially if you like podcasts; BetterHelp was the largest podcast advertiser in 2021. Worldwide, people are expected to spend $500 million on mental health apps in 2022. Yet these apps have varying degrees of scientific backing and there’s little to help you discern which ones are backed by real data.

Anyone can create a mental health app and start selling it on platforms. A study published in the journal Anxiety and depression in 2017 found that of 52 proposed apps for anxiety, two-thirds had no healthcare professional involved in their development. Less than four percent have been rigorously tested. For apps that undergo clinical testing, the research is often performed by researchers employed by the company selling the app.

There are also no robust government regulators verifying these claims. The US Food and Drug Administration has jurisdiction over health apps, but the risks of using them are so low and the apps so numerous that the agency rarely intervenes unless a particularly egregious app arises.

A surprising number of mental health apps use chatbots.WANG ZHAO/AFP/Getty Images

What they found – Goldberg’s team finds that researchers trying to assess the effectiveness of mental health apps are also inconsistent in how they go about it. When the researchers compared the apps to other treatments, they rarely compared them to in-person therapy, for example — a strange oversight.

The meta-review revealed a number of other issues. For example, research has rarely addressed how people interact with the apps studied. In one study, participants used the app for the duration of the study. But when people use mental health apps in the real world, the dropout rate is high.

“It’s a huge problem,” says Goldberg. “Even in a study, people often don’t stay with these things.”

He notes another problem with the interpretation of meta-analyses in general. When studies show no effect, researchers may not publish the data, leaving an “overly optimistic estimate of effect”.

On a more positive note, mental health apps consistently show evidence of “small-scale effects” – some relief for many users – on mental health issues.

Goldberg says that’s not surprising; there must be a reason behind their popularity, after all.

“I’m sure some of the claims are exaggerated,” he says, “but I also think it’s likely that a lot of the products that are out there are useful for people.”

“If you take a look at [the mediation app]Headspace, for example, millions of people use their products,” he says, “and I think for meditation apps to be used by millions of people, people have to benefit somewhat.

“Meditation isn’t much fun for most of us,” Goldberg adds.

How they did it — Goldberg and his co-authors conducted a meta-analysis, which is sort of a meta-analysis of meta-analyses. A meta-analysis systematically merges the results of several independent research trials to get a bigger picture of what they all studied. A meta-review does the same thing, but for meta-analyses.

They collected 14 meta-analyses, each involving at least four studies. In total, they included 145 randomized controlled trials of mental health apps, as well as some of text-message-based inventions that didn’t use any single app. In total, the research included 47,940 participants.

Some have reviewed research on the effectiveness of apps for relieving depression or anxiety. Others have studied the ability of apps to help people quit smoking or improve their general mental well-being. They also looked at different types of apps: guided meditation, texting with a person, texting with a chatbot, and DIY programs.

All the apps had one thing in common: you needed a smartphone; telehealth or in-person therapies were excluded from this analysis.

Headspace meditation app has over two million paid subscribersGetty Images/Getty Images Entertainment/Getty Images

And after – Goldberg says the disparate results for each app may have to do with the newness of the technology itself — they may not have been around long enough or may not have been used enough to get a good sense of their effectiveness in the laboratory or in the real world. . But time will tell.

“I think there is fantastic meta-analytical work and there is fantastic work [random clinical trials]happening,” he said. “I think literature is just young.”

“If you think about it, we’ve been studying psychotherapy for a hundred years. We have been studying mobile phone interventions for maybe 20 years,” he adds. It’s some catching up to do.


About Author

Comments are closed.