ArticlePDF AvailableLiterature Review

Efficacy and Conflicts of Interest in RCTs Evaluating Headspace and Calm Apps: A Systematic Review (Preprint)

Authors:

Abstract

Background: Although there are thousands of mental health apps, 2 apps, Headspace and Calm, claim a large percentage of the marketplace. These two mindfulness and meditation apps have reached tens of millions of active users. To guide consumers, clinicians, and researchers, we performed a systematic review of randomized controlled trials (RCTs) of Headspace and Calm. Objective: Our study aimed to evaluate intervention efficacy, risk of bias, and conflicts of interest (COIs) in the evidence base for Headspace and Calm, the two most popular mental health apps at the time of our search. Methods: To identify studies, we searched academic databases (Google Scholar, MEDLINE, and PsycINFO) and the websites of Headspace and Calm in May 2021 for RCTs of Headspace and Calm testing efficacy via original data collection, published in English in peer-reviewed journals. For each study, we coded (1) study characteristics (eg, participants, sample size, and outcome measures), (2) intervention characteristics (eg, free vs paid version of the app and intended frequency of app usage), (3) all study outcomes, (4) Cochrane risk of bias variables, and (5) COI variables (eg, presence or absence of a preregistration and the presence or absence of a COI statement involving the company). Results: We identified 14 RCTs of Headspace and 1 RCT of Calm. Overall, 93% (13/14) of RCTs of Headspace and 100% (1/1) of RCTs of Calm recruited participants from a nonclinical population. Studies commonly measured mindfulness, well-being, stress, depressive symptoms, and anxiety symptoms. Headspace use improved depression in 75% of studies that evaluated it as an outcome. Findings were mixed for mindfulness, well-being, stress, and anxiety, but at least 40% of studies showed improvement for each of these outcomes. Studies were generally underpowered to detect "small" or "medium" effect sizes. Furthermore, 50% (7/14) of RCTs of Headspace and 0% (0/1) of RCTs of Calm reported a COI that involved Headspace or Calm (the companies). The most common COI was the app company providing premium app access for free for participants, and notably, 14% (2/14) of RCTs of Headspace reported Headspace employee involvement in study design, execution, and data analysis. Only 36% (5/14) of RCTs of Headspace were preregistered, and the 1 RCT of Calm was not preregistered. Conclusions: The empirical research on Headspace appears promising, whereas there is an absence of randomized trials on Calm. Limitations of this study include an inability to compare Headspace and Calm owing to the dearth of RCTs studying Calm and the reliance on author reports to evaluate COIs. When determining whether or not mental health apps are of high quality, identification of high-quality apps and evaluation of their effectiveness and investigators' COIs should be ensured.
Records identified from:
PsycInfo (n = 19)
MEDLINE (n = 59)
Google Scholar (n= 34,310)
Records removed before
screening:
Duplicate records removed
(n = 16,328)
Records screened
(n = 18,060)
Records excluded:
Not studying Headspace or
Calm (n = 17,997)
Reports assessed for eligibility
(n = 63)
Reports excluded:
Non-RCT (n = 48)
Records identified from:
Headspace Website (n = 25)
Calm Website (n = 22)
Reports assessed for eligibility
(n = 47)
Reports excluded:
Non-RCT (n = 38)
Studies included in review
(n = 15)
Identification of studies via databases and registers
Identification of studies via other methods
Identification
Screening
Included
... A sizeable proportion within our sample were not willing to pay anything for a DMHI (39.2%) and the next category (14.2%) would not pay more than $0.99. DMHIs can be significantly more expensive; for example, Headspace, a popular commercial application (O'Daffer et al., 2022), uses a subscription model costing $69.99 annually or $12.99 per month (at the time of writing), reflecting a gap between cost and willingness or ability to pay among many individuals with unmet mental health needs. Unfortunately, DMHIs also face difficulties with other avenues of reimbursement. ...
Article
Full-text available
Unmet mental health needs have a negative impact and digital mental health interventions (DMHIs) offer a low-cost accessible way to receive care. Despite potential benefits, engagement with these strategies outside of research settings remains limited. Investigating the perspectives of persons with unmet mental health needs may offer insight into addressing this gap in care, particularly following changes of the COVID-19 pandemic. The current study investigated experiences with DMHIs, attitudes towards these strategies, and their impact on intentions to use such approaches among 296 individuals with unmet mental health needs. Only a small proportion of participants were using DMHIs (11.1%), although a larger part of the sample had done so in the past (37.5%). Participants endorsed greater comfort with DMHI functionalities they have more control over, such as tracking symptoms and habits and less comfort with those that were more intrusive, such as monitoring location or social media use. Higher educational attainment (β = 0.14, p = 0.01), greater perceived behavioral control (β = 0.18, p = 0.002), and more positive attitudes towards DMHIs (β = 0.39, p < 0.001) were predictive of intentions to use these tools. Intentions to use DMHIs were higher among those who had past or present engagement (d = 0.55, p < 0.001) supporting a link between attitude and behavior. Implications of findings and considerations for future research are discussed.
... Unfortunately, this lack of regulation may also result in less transparency for consumers around the efficacy of any given mHealth intervention. Many of the most popular mHealth apps, such as Headspace, Calm, and others, have been developed with clinical guidance and their effectiveness has been evaluated through research (Mani et al. 2015;O'Daffer et al. 2022). Yet these are intermixed in app stores with mHealth apps that lack evidence and may have questionable privacy policies (Rosenfeld et al. 2017;Tangari et al. 2021). ...
Article
An ongoing mental healthcare crisis has been exacerbated by COVID-19, particularly for youth. However, one unexpected effect of this pandemic is that access to digital mental healthcare has rapidly expanded. We argue that eHealth interventions, including telehealth and mobile interventions (mHealth), can effectively address mental health challenges, reduce costs for individuals and institutions, and broadly expand access to mental healthcare. For the promise of eHealth to be fully realized, FDA regulation must thoughtfully balance the promotion of evidence-based interventions with broad public accessibility. Furthermore, youth involvement throughout the design process and consideration of youth-specific needs when establishing regulations are critical to the success of eHealth in addressing youth mental health in the United States.
... Music therapy apps show promise in aiding gait training in Parkinson's, treating episodic migraine headaches, and rehabilitation following stroke [46][47][48]. Headspace and Calm are two mind-body health apps that help decrease depression [49]. Similar to telemedicine, apps provide increased access and may generate more interest in complementary medicine and ExAT. ...
Article
Full-text available
Art as a way of healing is primordial in many cultures. Expressive Arts Therapy (ExAT) uses art, music, dance, and writing to help individuals navigate their healing journey. Patient self-expression as a mode of recovery has been observed in patients with Parkinson’s disease, epilepsy, Amyotrophic Lateral Sclerosis (ALS) and cancer. Complementary medical approaches such as acupuncture and mindfulness have also demonstrated benefits in patients suffering from neurological injury. Complementary medicine and ExAT are not mainstream approaches following neurosurgical procedures. There are very few systematic studies evaluating the benefits of expressive arts in neurosurgery. Advances in telemedicine and mobile applications may facilitate the incorporation of complementary medicine and ExAT into patient recovery. The purpose of our study is to explore the use of complementary medicine and ExAT in neurosurgical recovery. We start with a brief introduction of ExAT followed by available treatments offered. We discuss the benefits of multidisciplinary care and emerging technologies and how they can facilitate incorporation of complementary medicine and ExAT in neurosurgery. Finally, we review several clinical studies that have demonstrated success in using complementary medicine. Our goal is to provide alternative approaches to neurosurgery recovery so that patients may receive with the best care possible.
Article
Full-text available
Transcranial direct current stimulation (tDCS) is an emerging treatment for major depression. We recruited participants with moderate-to-severe major depressive episodes for an observational clinical trial using Soterix Medical's tDCS telehealth platform as a standard of care. The acute intervention consisted of 28 sessions (5 sessions/week, 6 weeks) of the left anodal dorsolateral prefrontal cortex (DLPFC) tDCS (2.0 mA × 30 min) followed by a tapering phase of weekly sessions for 4 weeks (weeks 7–10). The n = 16 completing participants had a significant reduction in depressive symptoms by week 2 of treatment [Montgomery–Åsberg Depression Rating Scale (MADRS), Baseline : 28.00 ± 4.35 vs. Week 2: 17.12 ± 5.32, p < 0.001] with continual improvement across each biweekly timepoint. Acute intervention responder and remission rates were 75 and 63% and 88 and 81% following the taper period (week 10).
Article
Full-text available
Commercially available smartphone apps represent an ever-evolving and fast-growing market. Our review systematically surveyed currently available commercial sleep smartphone apps to provide details to inform both providers and patients alike, in addition to the healthy consumer market. Most current sleep apps offer a free version and are designed to be used while awake, prior to sleep, and focus on the enhancement of sleep, rather than measurement, by targeting sleep latency using auditory stimuli. Sleep apps could be considered a possible strategy for patients and consumers to improve their sleep, although further validation of specific apps is recommended.
Article
Full-text available
In this perspective, we aim to bring together research on mobile assessments and interventions in the context of mental health care in youth. After the COVID-19 pandemic, one out of five young people is experiencing mental health problems worldwide. New ways to face this burden are now needed. Young people search for low-burden services in terms of costs and time, paired with high flexibility and easy accessibility. Mobile applications meet these principles by providing new ways to inform, monitor, educate, and enable self-help, thus reinventing mental health care in youth. In this perspective, we explore the existing literature reviews on mobile assessments and interventions in youth through data collected passively (e.g., digital phenotyping) and actively (e.g., using Ecological Momentary Assessments—EMAs). The richness of such approaches relies on assessing mental health dynamically by extending beyond the confines of traditional methods and diagnostic criteria, and the integration of sensor data from multiple channels, thus allowing the cross-validation of symptoms through multiple information. However, we also acknowledge the promises and pitfalls of such approaches, including the problem of interpreting small effects combined with different data sources and the real benefits in terms of outcome prediction when compared to gold-standard methods. We also explore a new promising and complementary approach, using chatbots and conversational agents, that encourages interaction while tracing health and providing interventions. Finally, we suggest that it is important to continue to move beyond the ill-being framework by giving more importance to intervention fostering well-being, e.g., using positive psychology.
Article
Full-text available
Information and communication technology are promising channels in delivering the mindfulness-based intervention. A common problem in these interventions is the lack of study of treatment adherence. The current article summarizes the state of measuring and reporting adherence, the rate of adherence, and possible predictors of adherence in the form of a systematic review following the PRISMA statement. A database search of PubMed, Web of Science, Medline, Scopus, and PsychINFO identified 3104 potentially relevant articles published up to the end of December 2019. The review focuses only on the SARS-CoV-2 pre-pandemic time because the accelerated development of the eHealth Mindfulness-based interventions and a vast amount of newly published interventions in the following years was expected. A total of 69 studies met all the eligibility criteria. Out of these studies, 61% reported some measure of adherence and 36% reported the adherence rate. The adherence rate in clinical populations ranged from 41% to 92%; in non-clinical populations, it ranged from 1% to 85%. Predictors of adherence were investigated in 48% of the studies; however, the majority of assessed variables had mixed findings about predicting adherence. There was a huge variety in the definitions and measurements of adherence. A standardized system of measuring and reporting adherence and further investigation of its predictors is needed.
Article
Full-text available
Background The digital health sector has experienced rapid growth over the past decade. However, health care technology stakeholders lack a comprehensive understanding of clinical robustness and claims across the industry. Objective This analysis aimed to examine the clinical robustness and public claims made by digital health companies. Methods A cross-sectional observational analysis was conducted using company data from the Rock Health Digital Health Venture Funding Database, the US Food and Drug Administration, and the US National Library of Medicine. Companies were included if they sell products targeting the prevention, diagnosis, or treatment phases of the care continuum. Clinical robustness was defined using regulatory filings and clinical trials completed by each company. Public claims data included clinical, economic, and engagement claims regarding product outcomes made by each company on its website. Results A total of 224 digital health companies with an average age of 7.7 years were included in our cohort. Average clinical robustness was 2.5 (1.8 clinical trials and 0.8 regulatory filings) with a median score of 1. Ninety-eight (44%) companies had a clinical robustness score of 0, while 45 (20%) companies had a clinical robustness score of 5 or more. The average number of public claims was 1.3 (0.5 clinical, 0.4 economic, and 0.4 engagement); the median number of claims was 1. No correlation was observed between clinical robustness and number of clinical claims (r2=0.02), clinical robustness and total funding (r2=0.08), or clinical robustness and company age (r2=0.18). Conclusions Many digital health companies have a low level of clinical robustness and do not make many claims as measured by regulatory filings, clinical trials, and public data shared online. Companies and customers may benefit from investing in greater clinical validation efforts.
Article
Full-text available
Background: The mental disorders included in the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2019 were depressive disorders, anxiety disorders, bipolar disorder, schizophrenia, autism spectrum disorders, conduct disorder, attention-deficit hyperactivity disorder, eating disorders, idiopathic developmental intellectual disability, and a residual category of other mental disorders. We aimed to measure the global, regional, and national prevalence, disability-adjusted life-years (DALYS), years lived with disability (YLDs), and years of life lost (YLLs) for mental disorders from 1990 to 2019. Methods: In this study, we assessed prevalence and burden estimates from GBD 2019 for 12 mental disorders, males and females, 23 age groups, 204 countries and territories, between 1990 and 2019. DALYs were estimated as the sum of YLDs and YLLs to premature mortality. We systematically reviewed PsycINFO, Embase, PubMed, and the Global Health Data Exchange to obtain data on prevalence, incidence, remission, duration, severity, and excess mortality for each mental disorder. These data informed a Bayesian meta-regression analysis to estimate prevalence by disorder, age, sex, year, and location. revalence was multiplied by corresponding disability weights to estimate YLDs. Cause-specific deaths were compiled from mortality surveillance databases. The Cause of Death Ensemble modelling strategy was used to estimate death rate by age, sex, year, and location. The death rates were multiplied by the years of life expected to be remaining at death based on a normative life expectancy to estimate YLLs. Deaths and YLLs could be calculated only for anorexia nervosa and bulimia nervosa, since these were the only mental disorders identified as underlying causes of death in GBD 2019. Findings: Between 1990 and 2019, the global number of DALYs due to mental disorders increased from 80·8 million (95% uncertainty interval [UI] 59·5–105·9) to 125·3 million (93·0–163·2), and the proportion of global DALYs attributed to mental disorders increased from 3·1% (95% UI 2·4–3·9) to 4·9% (3·9–6·1). Age-standardised DALY rates remained largely consistent between 1990 (1581·2 DALYs [1170·9–2061·4] per 100 000 people) and 2019 (1566·2 DALYs [1160·1–2042·8] per 100 000 people). YLDs contributed to most of the mental disorder burden, with 125·3 million YLDs (95% UI 93·0–163·2; 14·6% [12·2–16·8] of global YLDs) in 2019 attributable to mental disorders. Eating disorders accounted for 17 361·5 YLLs (95% UI 15 518·5–21 459·8). Globally, the age-standardised DALY rate for mental disorders was 1426·5 (95% UI 1056·4–1869·5) per 100 000 population among males and 1703·3 (1261·5–2237·8) per 100 000 population among females. Age-standardised DALY rates were highest in Australasia, Tropical Latin America, and high-income North America. Interpretation: GBD 2019 showed that mental disorders remained among the top ten leading causes of burden worldwide, with no evidence of global reduction in the burden since 1990. The estimated YLLs for mental disorders were extremely low and do not reflect premature mortality in individuals with mental disorders. Research to establish causal pathways between mental disorders and other fatal health outcomes is recommended so that this may be addressed within the GBD study. To reduce the burden of mental disorders, coordinated delivery of effective prevention and treatment programmes by governments and the global health community is imperative.
Article
Full-text available
Background Youper is a widely used, commercially available mobile app that uses artificial intelligence therapy for the treatment of anxiety and depression. Objective Our study examined the acceptability and effectiveness of Youper. Further, we tested the cumulative regulation hypothesis, which posits that cumulative emotion regulation successes with repeated intervention engagement will predict longer-term anxiety and depression symptom reduction. Methods We examined data from paying Youper users (N=4517) who allowed their data to be used for research. To characterize the acceptability of Youper, we asked users to rate the app on a 5-star scale and measured retention statistics for users’ first 4 weeks of subscription. To examine effectiveness, we examined longitudinal measures of anxiety and depression symptoms. To test the cumulative regulation hypothesis, we used the proportion of successful emotion regulation attempts to predict symptom reduction. Results Youper users rated the app highly (mean 4.36 stars, SD 0.84), and 42.66% (1927/4517) of users were retained by week 4. Symptoms decreased in the first 2 weeks of app use (anxiety: d=0.57; depression: d=0.46). Anxiety improvements were maintained in the subsequent 2 weeks, but depression symptoms increased slightly with a very small effect size (d=0.05). A higher proportion of successful emotion regulation attempts significantly predicted greater anxiety and depression symptom reduction. Conclusions Youper is a low-cost, completely self-guided treatment that is accessible to users who may not otherwise access mental health care. Our findings demonstrate the acceptability and effectiveness of Youper as a treatment for anxiety and depression symptoms and support continued study of Youper in a randomized clinical trial.
Article
Full-text available
Background: In an oversaturated market of publicly available mobile apps for psychosocial self-care and stress management, health care providers, patients, and consumers interested in mental health-related apps may wonder which, if any, are efficacious. Readily available metrics for consumers include user popularity and media buzz rather than scientific evidence. Objective: This systematic review aimed to (1) examine the breadth of therapeutic contents and features of psychosocial wellness and stress management apps available to self-help seekers for public download and (2) determine which of these apps have original research support. Methods: First, we conducted a systematic review of publicly available apps on the iPhone App Store (Apple Inc) and Android Google Play (Google LLC) platforms using conventional self-help-seeking search terms related to wellness and stress. The results were limited to English-language apps available for free download. In total, 2 reviewers independently evaluated all apps and discussed the findings to reach 100% consensus regarding inclusion. Second, a literature review was conducted on the included apps to identify supporting studies with original data collection. Results: We screened 3287 apps and found 1009 psychosocial wellness and stress management apps. Content varied widely. The most common evidence-based strategy was mindfulness-meditation, followed by positive psychology and goal setting. Most apps were intended to be used as self-help interventions, with only 1.09% (11/1009) involving an electronic therapist and 1.88% (19/1009) designed as a supplement to in-person psychotherapy. Only 4.66% (47/1009) of apps targeted individuals with psychological disorders, and less than 1% of apps (6/1009, 0.59%) targeted individuals with other chronic illnesses. Approximately 2% (21/1009, 2.08%) were supported by original research publications, with a total of 25 efficacy studies and 10 feasibility studies. The Headspace mindfulness app had the most evidence, including 8 efficacy studies. Most other scientifically backed apps were supported by a single feasibility or efficacy study. Conclusions: Only 2.08% (21/1009) of publicly available psychosocial wellness and stress management mobile apps discoverable to self-help seekers have published, peer-reviewed evidence of feasibility and/or efficacy. Clinicians and investigators may use these findings to help patients and families navigate the volume of emerging digital health interventions for stress management and wellness.
Article
Full-text available
Anxiety and trauma related disorders are highly prevalent, causing suffering and high costs for society. Current treatment strategies, although effective, only show moderate effect-sizes when compared to adequate control groups demonstrating a need to develop new forms of treatment or optimize existing ones. In order to achieve this, an increased understanding of what mechanisms are involved is needed. An emerging literature indicates that mindfulness training (MFT) can be used to treat fear and anxiety related disorders, but the treatment mechanisms are unclear. One hypothesis, largely based on findings from neuroimaging studies, states that MFT may improve extinction retention, but this has not been demonstrated empirically. To investigate this question healthy subjects either completed a 4-week MFT- intervention delivered through a smart-phone app (n = 14) or were assigned to a waitlist (n = 15). Subsequently, subjects participated in a two-day experimental protocol using pavlovian aversive conditioning, evaluating acquisition and extinction of threat-related responses on day 1, and extinction retention on day 2. Results showed that the MFT group displayed reduced spontaneous recovery of threat related arousal responses, as compared to the waitlist control group, on day 2. MFT did not however, have an effect on either the acquisition or extinction of conditioned responses day 1. This clarifies the positive effect of MFT on emotional functioning and could have implications for the treatment of anxiety and trauma related disorders.
Article
Full-text available
Smartphone applications for the treatment of depression and anxiety have acquired millions of users, yet little is known about whether they include evidence-based therapeutic content. We examined the extent to which popular mental health applications (MH apps) for depression and anxiety contain treatment elements found in empirically supported psychotherapy protocols (i.e., "common elements"). Of the 27 MH apps reviewed, 23 included at least one common element, with a median of three elements. Psychoeducation (in 52% of apps), relaxation (44%), meditation (41%), mindfulness (37%), and assessment (37%) were the most frequent elements, whereas several elements (e.g., problem solving) were not found in any apps. We also identified gaps between app content and empirically supported treatments. Cognitive restructuring was more common in depression protocols than in depression apps (75% of protocols vs. 31% of apps), as was problem solving (34% vs. 0%). For anxiety, exposure (85%, 12%), cognitive restructuring (60%, 12%), and problem solving (25%, 0%) were more common in protocols than apps. Overall, our findings highlight empirically supported treatment elements that are poorly represented in current MH apps. The absence of several core treatment elements underscores the need for future research, including randomized trials testing the effectiveness of popular MH apps.
Article
Objective This article analyzes the changes in downloads and activity of users of select popular mental health mobile applications (mHealth apps) during coronavirus disease 2019 (COVID-19). Background The outbreak of the COVID-19 crisis has shown a negative impact on public mental health. Mobile health has the potential to help address the psychological needs of existing and new patients during the pandemic and beyond. Method Downloads data of 16 widely used apps were analyzed. The quality of apps was reviewed using the Mobile Application Rating Scale (MARS) framework. Correlation analysis was conducted to investigate the relationship between app quality and app popularity. Results Among the 16 apps, 10 were meditational in nature, 13 showed increased downloads, with 11 apps showing above 10% increase in the downloads after the pandemic started. The popular apps were satisfactory in terms of functionality and esthetics but lacked clinical grounding and evidence base. There exists a gap between app quality and app popularity. Conclusion This study provided evidence for increased downloads of mental mHealth apps (primarily meditation apps) during the COVID-19 pandemic but revealed several gaps and opportunities to address deficiencies in evidence-based design, usability and effective assessment, and integration into current workflows. Application The COVID-19 pandemic is a potential turning point for mHealth applications for mental health care. Whereas the evidence suggests a need for alternative delivery of care, human factors and ergonomics methods should be utilized to ensure these tools are user-centered, easy to use, evidence-based, well-integrated with professional care, and used sustainably.
Article
In this article, we (1) discuss the reasons why pre‐registration is a good idea, both for the field and for individual researchers, (2) respond to arguments against pre‐registration, (3) describe how to best write and review a pre‐registration, and (4) comment on pre‐registration’s rapidly accelerating popularity. Along the way, we describe the (big) problem that pre‐registration can solve (i.e., false‐positives caused by p‐hacking), while also offering viable solutions to the problems that pre‐registration cannot solve (e.g., hidden confounds or fraud). Pre‐registration does not guarantee that every published finding will be true, but without it you can safely bet that many more will be false. It is time for our field to embrace pre‐registration, while taking steps to ensure that it is done right.