PCORI funds implementation projects to promote the use of findings from PCORI-funded studies. This project focuses on implementing findings from the completed PCORI-funded research project: Comparing a Smartphone Program with a Peer-Led Program to Help People with Serious Mental Illness Manage Their Symptoms.
1. What were the results from the original PCORI-funded research study?
The project team conducted a comparative effectiveness trial of a smartphone-delivered mHealth intervention (FOCUS) versus a widely-used clinic-based intervention. FOCUS participants had significantly better treatment commencement rates following randomization (90% vs. 58%) and better continuous engagement in treatment. FOCUS participants reported high treatment satisfaction ratings and experienced significant improvement in clinical outcomes: general psychopathology (SCL-9 Cohen’s d=.44), depression (BDI-2 Cohen’s d=.31), and recovery (RAS Cohen’s d=.38). FOCUS clinical effects are in the medium range and are clinically meaningful. Reduction in severity of depressive symptoms was found trans-diagnostically. During treatment, FOCUS treatment responders (participants who experienced recovery gains and maintained them at follow-up) reduced service use more than non-responders. Postintervention, service use by mHealth treatment responders continued to drop (by an additional 11%). The study showed that: a) FOCUS was more accessible than clinic-based care; b) FOCUS was more engaging than clinic-based care; c) FOCUS accomplished the same clinically as an in-person, clinic-based intervention more intensive in time, labor, and resources, and reduced the need for additional services.
2. Why is this research finding important?
The project team’s original PCORI study was the first to involve a direct head-to-head comparison of a smartphone-based mHealth intervention for SMI and another active evidence-based clinic intervention. The literature consistently describes a gap in services for people with SMI and significant unmet need, in part due to limited access to evidence-based treatment options at clinics. Before the project team’s PCORI study it was unclear whether mHealth (with its strong potential for very broad accessibility and remote engagement) could produce outcomes that are comparable to more complex and resource-intensive, person-delivered treatments for SMI. The project team’s study demonstrated conclusively that the FOCUS smartphone intervention produces robust engagement, satisfaction and clinical improvements. The project team concluded that for people with SMI seeking illness management support, FOCUS is a viable option because they are more likely to access, start, and stay in mHealth treatment without sacrificing satisfaction and improvement in key clinical outcomes. The project team’s findings have become particularly relevant in the new COVID-19 mental health services landscape; social distancing mandates have hampered agencies’ ability to deliver clinic-based care, prompting widescale interest in telehealth technologies.
3. What is the goal of this project?
The project team proposes to use a multi-faceted technology-assisted strategy to support systematic implementation of FOCUS at community agencies that provide services to people with SMI in two US states. The project team will provide the training, resources, and expertise necessary to expand the scope, reach, impact, and sustainability potential of an intervention recommended based on the original PCORI trial, broadening the impact of this important work.
4. What is the project team doing?
The project team will employ a multi-component implementation strategy guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework. The core FOCUS intervention components will be unchanged from the project team’s PCORI study. the project team’s implementation strategy will target the adaptable periphery of these components to increase knowledge about the FOCUS application, improve scalability of mHealth support specialist training, and to make the delivery of FOCUS data to clinicians more efficient. The project team’s approach will employ evidence-based implementation strategies: development of an advisory board, improved training materials and delivery of clinical data to clinicians, and facilitation practice to support FOCUS implementation. In the exploration phase the project team engage agency leadership in discussions about mHealth alignment with agency mission, identification of needs related to use of mHealth, and development of an implementation strategy to meet staff training and service delivery needs. During the preparation phase the project team will use lessons learned from work during the COVID-19 pandemic and from the exploration phase to adapt in-person implementation and evaluation materials to be administered remotely. Existing FOCUS training and fidelity monitoring materials will be digitized and clinical data from FOCUS will be delivered via interactive learning management system (LMS). In the implementation phase of the project, the project team’s partner agencies will use the adapted materials to implement FOCUS while members from the project team’s group will provide remote practice facilitation by monitoring provider engagement with FOCUS training and clinical data, conducting remote fidelity assessments, providing feedback to clinicians, and tailoring training materials to ensure practitioners are competent in the delivery of FOCUS. In the sustainment phase agencies will take on the independent implementation of FOCUS with the project team’s reducing support and monitoring implementation outcomes. The study will be conducted in collaboration with the adult outpatient programs at community mental health agencies in New Hampshire and Missouri that provide care to people with SMI: The Mental Health Center of Greater Manchester (MHCGM), Community Partners New Hampshire (CPNH), and Places for People in St. Louis County. Leaders and frontline staff have been advised of the activities and have agreed to integrate FOCUS implementation into their workflows.
5. How is the team evaluating this project?
Patient-level symptom and recovery measures will be collected at baseline and post-intervention. Data will be analyzed using mixed effects linear regression with a fixed effect for time-point and random effect for individual patient, accounting for repeated observations within individual. Implementation outcomes will be collected via assessments with agency staff at three timepoints: 8, 24, and 32 months after study initiation, allowing assessment of implementation outcomes shortly after training: 1 year into the implementation phase, and 6 months into the sustainment phase.
6. How is the team involving patients and others in making sure the findings reach people who can use them?
Stakeholders include multiple people with lived experience of SMI and family members; community mental health leaders; professionals who provide services to people with SMI; and mental health advocates at the local, regional, and national levels. The project team is partnering with the National Alliance on Mental Illness' New Hampshire (www.naminh.org) and St Louis (www.namistl.org) branches, and the St. Louis Empowerment Center of the Depression and Bipolar Support Alliance (DBSA) ( www.dbsalliance.org). The project team engaged the New Hampshire Behavioral Health Association (www.nhcbha.org), the professional organization of all community mental health centers in the state.
7. How will this project help ensure future uptake and use of findings from patient-centered outcomes research?
At the conclusion of the project, two key partners will support the widespread dissemination of the findings and uptake. The National Council for Mental Wellbeing (www.thenationalcouncil.org/), the nation’s premier community mental health treatment association, will disseminate findings among its nearly 3,500 agency members serving 10 million people in 40 states. The Mental Health Technology Transfer Network (MHTCC) (https://mhttcnetwork.org/) will adopt the study products (training curriculums, learning management system, and intervention support digital resources) and have agreed to make them freely available through the MHTCC resource catalogue.
Initial PCORI-Funded Research Study
This implementation project focuses on putting findings into practice from this completed PCORI-funded research study: Comparing a Smartphone Program with a Peer-Led Program to Help People with Serious Mental Illness Manage Their Symptoms