Results Summary

What was the research about?

People with a serious mental illness, or SMI, may have difficulty with daily activities like working or taking care of themselves. Programs at mental health clinics can help people with SMI manage their symptoms and improve their health. But it can be hard for people with SMI to get to the clinic to use these programs, and some may not feel comfortable at a clinic.

In this study, the research team compared two programs to help people with SMI manage their symptoms. In the FOCUS program, people used a smartphone app. In the Wellness Recovery Action Plan, or WRAP, program, people went to group sessions at a clinic.

What were the results?

Using the programs. The research team studied how often people used the programs. To do this, the team looked at whether people used FOCUS at least once a day for five days every week. For WRAP, the team looked at whether people went to at least 60 minutes of a 90-minute session every week. More people in FOCUS started the program than people in WRAP. After eight weeks, more people in FOCUS used their assigned program than those in WRAP. When the programs ended after 12 weeks, the team found no difference between people’s use of the two programs.

Program satisfaction. The study found no difference in how satisfied people were with their assigned program. People in both programs reported high satisfaction.

Recovery and quality of life. The study found no difference in how well the programs helped people recover or in people’s quality of life.

Depression and other symptoms. After the study, people in both programs had less depression, anxiety, and other symptoms than before the study. The study found no difference between the two groups.

Who was in the study?

The study included 163 people with SMI getting care at a large health agency in the Chicago area. Of these, 65 percent were African American. The average patient age was 49, and 59 percent were men. The patients had schizophrenia or schizoaffective disorder, bipolar disorder, or major depressive disorder.

What did the research team do?

The research team assigned people to either FOCUS or WRAP by chance. Each program lasted 12 weeks.

  • People in FOCUS used a smartphone app. They got daily prompts to remind them to answer questions about their symptoms and functioning. They could also access information to help them manage their symptoms. A support specialist called people weekly to talk about their symptoms and any problems using the smartphone.
  • People in WRAP went to a mental health clinic every week for a two-hour group session. People with lived experience with mental illness led the sessions. In the group sessions, people talked about challenges. They also put together plans to help them get well.

To compare the effects of FOCUS and WRAP, the research team gave surveys to people at the end of the programs and again three months later. The team also interviewed people in both programs.

What were the limits of the study?

The research team gave people in the FOCUS program a smartphone with a data plan. People who don’t have smart phones with data plans wouldn’t be able to use this program. The study took place in one urban area; results may not apply to other areas.

Because the team didn’t compare the FOCUS and WRAP groups with a group that didn’t get help with their symptoms, the team doesn’t know if the changes were because of FOCUS and WRAP or something else.

Future research could see if the FOCUS program works for people in rural or other areas where access to clinic services is difficult.

How can people use the results?

Compared with the WRAP program, more people in the FOCUS group were willing to start the program and more people engaged with it during the first eight weeks of treatment. Mental health clinics can use the study results when looking at how to engage and support people with SMI.

Final Research Report

View this project's final research report.

More to Explore...

Facebook Live

Does Telehealth Improve Outcomes for Patients with Serious Mental Illness?
On November 1, 2018, Dror Ben-Zeev, PhD, participated in a PCORI Facebook Live session for a conversation about how telehealth may help improve outcomes for patients living with serious mental illness. Mark Ishaug, MA, a study stakeholder partner, also participated in the session.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented, and the researchers made changes or provided responses. Those comments and responses included the following:

  • Reviewers pointed out that the researchers could not establish that the new intervention, FOCUS, was in fact beneficial because the study had been designed as a head-to-head comparison. The researchers added a study limitation indicating that because they tested FOCUS against another intervention rather than against no treatment or usual care, the advantages seen in FOCUS outcomes could potentially be artifacts of time passing or other unmeasured events.
  • Reviewers expressed concern that the study sample could be biased because study staff recruited only those patients who reached out to them, rather than trying to recruit all patients at the target clinics. The researchers acknowledged that this could lead to a more-activated, engaged study sample but noted that because of clinic restrictions they were unable to approach potential participants directly. They further noted that this problem is common in research in the real world.
  • Reviewers noted several differences between the two interventions under study other than the mobile health component that was a feature of the FOCUS intervention. The researchers acknowledged this, stating that they could only speculate about whether the mobile health feature of FOCUS led to any outcome differences. The reason researchers did not use a comparator intervention that was more similar to FOCUS except for the mobile health component was that they wanted to compare FOCUS to an intervention that was evidence based and widely used.

Conflict of Interest Disclosures

Project Information

Dror Ben-Zeev, PhD
University of Washington^
$1,895,609
10.25302/4.2019.CER.140311403IC
Comparing Mobile Health and Clinic-Based Self-Management Interventions for Serious Mental Illness: Patient Engagement, Satisfaction, and Outcomes

Key Dates

September 2014
September 2018
2014
2018

Study Registration Information

^Dror Ben-Zeev, PhD was affiliated with Dartmouth College in New Hampshire when this project was initially awarded.

Tags

Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
Intervention Strategy Intervention Strategies PCORI funds comparative clinical effectiveness research (CER) studies that compare two or more options or approaches to health care, or that compare different ways of delivering or receiving care. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: November 30, 2022