This project has results

Comparing a Smartphone Program with a Peer-Led Program to Help People with Serious Mental Illness Manage Their Symptoms

Sign Up for Updates to This Study  

Results Summary and Professional Abstract

Results Summary
Download Summary Español (pdf) Audio Recording (mp3)

Results Summary

What was the research about?

People with a serious mental illness, or SMI, may have difficulty with daily activities like working or taking care of themselves. Programs at mental health clinics can help people with SMI manage their symptoms and improve their health. But it can be hard for people with SMI to get to the clinic to use these programs, and some may not feel comfortable at a clinic.

In this study, the research team compared two programs to help people with SMI manage their symptoms. In the FOCUS program, people used a smartphone app. In the Wellness Recovery Action Plan, or WRAP, program, people went to group sessions at a clinic.

What were the results?

Using the programs. The research team studied how often people used the programs. To do this, the team looked at whether people used FOCUS at least once a day for five days every week. For WRAP, the team looked at whether people went to at least 60 minutes of a 90-minute session every week. More people in FOCUS started the program than people in WRAP. After eight weeks, more people in FOCUS used their assigned program than those in WRAP. When the programs ended after 12 weeks, the team found no difference between people’s use of the two programs.

Program satisfaction. The study found no difference in how satisfied people were with their assigned program. People in both programs reported high satisfaction.

Recovery and quality of life. The study found no difference in how well the programs helped people recover or in people’s quality of life.

Depression and other symptoms. After the study, people in both programs had less depression, anxiety, and other symptoms than before the study. The study found no difference between the two groups.

Who was in the study?

The study included 163 people with SMI getting care at a large health agency in the Chicago area. Of these, 65 percent were African American. The average patient age was 49, and 59 percent were men. The patients had schizophrenia or schizoaffective disorder, bipolar disorder, or major depressive disorder.

What did the research team do?

The research team assigned people to either FOCUS or WRAP by chance. Each program lasted 12 weeks.

  • People in FOCUS used a smartphone app. They got daily prompts to remind them to answer questions about their symptoms and functioning. They could also access information to help them manage their symptoms. A support specialist called people weekly to talk about their symptoms and any problems using the smartphone.
  • People in WRAP went to a mental health clinic every week for a two-hour group session. People with lived experience with mental illness led the sessions. In the group sessions, people talked about challenges. They also put together plans to help them get well.

To compare the effects of FOCUS and WRAP, the research team gave surveys to people at the end of the programs and again three months later. The team also interviewed people in both programs.

What were the limits of the study?

The research team gave people in the FOCUS program a smartphone with a data plan. People who don’t have smart phones with data plans wouldn’t be able to use this program. The study took place in one urban area; results may not apply to other areas.

Because the team didn’t compare the FOCUS and WRAP groups with a group that didn’t get help with their symptoms, the team doesn’t know if the changes were because of FOCUS and WRAP or something else.

Future research could see if the FOCUS program works for people in rural or other areas where access to clinic services is difficult.

How can people use the results?

Compared with the WRAP program, more people in the FOCUS group were willing to start the program and more people engaged with it during the first eight weeks of treatment. Mental health clinics can use the study results when looking at how to engage and support people with SMI.

Professional Abstract

Professional Abstract

Objective

To compare the effectiveness of a smartphone-based versus a clinic-based self-management intervention in helping people with serious mental illness (SMI) manage their symptoms

Study Design

Design Elements Description
Design Randomized controlled trial
Population 163 patients with SMI receiving community-based treatment or recovery-support services
Interventions/
Comparators
  • FOCUS: smartphone app to help patients self-manage symptoms and a support specialist who called patients weekly
  • WRAP: in-person weekly 2-hour personal wellness sessions led by 2 trained facilitators with lived experience of mental illness to help patients self-manage symptoms
Outcomes

Primary: patient engagement, patient satisfaction, general psychopathology

Secondary: depression, psychosis, recovery, quality of life
Timeframe 3-month follow-up for primary outcomes

This randomized controlled trial compared the effects of two self-management interventions on patient engagement, patient satisfaction, and clinical outcomes in patients with SMI.

In the first intervention, called FOCUS, participants received a smartphone app that included daily self-assessment prompts and written, video, and audio content for self-management of symptoms. Clinicians could view patients’ responses to daily assessment prompts and reports on patients’ use of the smartphone system via a dashboard. A support specialist also reviewed the dashboards and called patients weekly to check in about symptoms and patients’ use of the technology.

In the second intervention, Wellness Recovery Action Plan (WRAP), trained facilitators with lived experience of mental illness led weekly two-hour wellness sessions. They helped patients develop a list of ways to stay well, identify potentially upsetting events, and develop written action plans for dealing with crises.

The study included 163 patients from a large mental health agency in the Chicago area. Of these, 65% were African American. The average patient age was 49, and 59% were male. The patients had schizophrenia or schizoaffective disorder, bipolar disorder, or major depressive disorder.

Each intervention lasted 12 weeks. Researchers followed up with all participants at the end of the intervention and again three months later. Participants completed surveys to measure engagement in and satisfaction with treatment. They also completed a mental health assessment to assess general psychopathology. The assessment used a measure with domains such as anxiety, somatization, hostility, paranoid thinking, and psychoticism. Researchers also interviewed participants to learn about their experiences with FOCUS and WRAP and their perceptions about the impact of the two interventions.

People with SMI, patient advocates, mental health advocates, and mental health clinic managers provided input on research procedures and materials development.

Results

Patient engagement. Following randomization, more participants assigned to FOCUS started their program than participants assigned to WRAP (p<0.001). Participants assigned to FOCUS were more likely to remain fully engaged in treatment after eight weeks (p=0.03) compared with participants assigned to WRAP. However, after 12 weeks, the two interventions did not differ in the number of participants who were fully engaged.

Satisfaction with treatment. At 12 weeks, participants’ satisfaction with treatment did not differ between the two interventions. Participants in both interventions reported high overall satisfaction.

Clinical outcomes. General psychopathology, depression, psychosis, recovery, and quality of life did not differ between participants in FOCUS and WRAP. In both interventions, participant-reported general psychopathology (FOCUS, p<0.001; WRAP, p=0.005) and depression (FOCUS, p=0.01; WRAP, p=0.03) decreased. For WRAP participants, recovery improved when the intervention ended (p=0.03); for FOCUS participants, recovery and quality of life improved three months postintervention (p=0.01).

Limitations

Patients in FOCUS received a smartphone with a data plan. People outside of the study may not have smartphones with data plans. Because the study did not include a usual care group, researchers could not determine whether improvements in outcomes, such as patient-reported depression and general psychopathology, resulted from the interventions or something else.

Conclusions and Relevance

More people were willing to start and work with FOCUS than WRAP over the first eight weeks of the intervention. FOCUS worked as well as WRAP to help people with SMI manage their symptoms. Smartphone-based interventions for people with SMI may support self-management without the potential barriers that occur with clinic-based interventions. Mental health clinics can use these results when considering how best to engage and support patients with SMI.

Future Research Needs

Future research could compare these interventions in rural areas, where access to clinics may be more limited.

Final Research Report

View this project's final research report.

Journal Articles

Results of This Project

Psychiatric Services

Mobile Health (mHealth) Versus Clinic-Based Group Intervention for People With Serious Mental Illness: A Randomized Controlled Trial

Related Articles

JMIR Mental Health

Transdiagnostic Mobile Health: Smartphone Intervention Reduces Depressive Symptoms in People With Mood and Psychotic Disorders

Psychiatric Services

Effect of Mobile Health on In-person Service Use Among People With Serious Mental Illness

Psychiatric Rehabilitation Journal

Life with FOCUS: A qualitative evaluation of the impact of a smartphone intervention on people with serious mental illness

More on this Project  

Facebook Live

Does Telehealth Improve Outcomes for Patients with Serious Mental Illness?
On November 1, 2018, Dror Ben-Zeev, PhD, participated in a PCORI Facebook Live session for a conversation about how telehealth may help improve outcomes for patients living with serious mental illness. Mark Ishaug, MA, a study stakeholder partner, also participated in the session.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented, and the researchers made changes or provided responses. Those comments and responses included the following:

  • Reviewers pointed out that the researchers could not establish that the new intervention, FOCUS, was in fact beneficial because the study had been designed as a head-to-head comparison. The researchers added a study limitation indicating that because they tested FOCUS against another intervention rather than against no treatment or usual care, the advantages seen in FOCUS outcomes could potentially be artifacts of time passing or other unmeasured events.
  • Reviewers expressed concern that the study sample could be biased because study staff recruited only those patients who reached out to them, rather than trying to recruit all patients at the target clinics. The researchers acknowledged that this could lead to a more-activated, engaged study sample but noted that because of clinic restrictions they were unable to approach potential participants directly. They further noted that this problem is common in research in the real world.
  • Reviewers noted several differences between the two interventions under study other than the mobile health component that was a feature of the FOCUS intervention. The researchers acknowledged this, stating that they could only speculate about whether the mobile health feature of FOCUS led to any outcome differences. The reason researchers did not use a comparator intervention that was more similar to FOCUS except for the mobile health component was that they wanted to compare FOCUS to an intervention that was evidence based and widely used.

Conflict of Interest Disclosures

View the COI disclosure form.

Project Details

Principal Investigator
Dror Ben-Zeev, PhD
Project Status
Completed; PCORI Public and Professional Abstracts, and Final Research Report Posted
Project Title
Comparing Mobile Health and Clinic-Based Self-Management Interventions for Serious Mental Illness: Patient Engagement, Satisfaction, and Outcomes
Board Approval Date
September 2014
Project End Date
September 2018
Organization
University of Washington^
Year Awarded
2014
State
Washington
Year Completed
2018
Project Type
Research Project
Health Conditions  
Mental/Behavioral Health
Schizophrenia and Psychotic Disorders
Serious Mental Illness
Depression
Bipolar Disorder
Intervention Strategies
Technology Interventions
Training and Education Interventions
Populations
Racial/Ethnic Minorities
Funding Announcement
Assessment of Prevention, Diagnosis, and Treatment Options
Project Budget
$1,895,609
DOI - Digital Object Identifier
10.25302/4.2019.CER.140311403IC
Study Registration Information
HSRP20152198
NCT02421965

^Dror Ben-Zeev, PhD was affiliated with Dartmouth College in New Hampshire when this project was initially awarded.

Page Last Updated: 
February 20, 2020