Tailor, Iterate, and Adapt to Context
Several strategies have been developed and used so that evidence-based practice (EBP) efforts better fit the particular and sometimes evolving context and circumstances of the sites where they occur. These strategies are often dynamic and can be focused on the practice to be implemented or the implementation strategy. We include evidence related to six strategies under this category. Select any of the strategies in the box below to learn more.
Purposefully Reexamine and Adjust the Implementation
Strategy Definition
This is a strategy that monitors progress and adjusts effective practices and implementation strategies to continuously improve the quality of care. (Source)
This strategy underscores the value of using a concrete schedule for monitoring implementation progress rather than as-needed reevaluation. This should include time-sensitive benchmarks or milestone dates to anticipate when adjustments might be needed.
Strength of Evidence Supporting Its Effectiveness: Inconclusive
Based on direct and indirect evidence from included studies and contextual systematic review evidence identified outside of this review, we judged evidence to be either supportive, moderate supportive, limited supportive, indirect supportive only, or inconclusive. We determined the strength of evidence supporting the contribution of this strategy to successful implementations was inconclusive.
Health Care Areas Where the Purposefully Reexamine and Adjust the Implementation Strategy Was Used
Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.
| Area | Percentage of studies | Number of studies |
|---|---|---|
| Mental health | 12% | 2 |
| Multiple Settings | 6% | 1 |
| Primary care | 25% | 4 |
| Public health | 38% | 6 |
| Specialty care | 12% | 2 |
| Other | 6% | 1 |
Results From Studies That Included Purposefully Reexamine and Adjust the Implementation Strategy
We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.
How Often Were Studies That Included the "Purposefully Reexamine and Adjust the Implementation" Strategy Effective?
| Outcome | Percentage of studies | Number of studies |
|---|---|---|
| Conducting the EBP at alcohol | 86% | 6/7 |
| Performing the EBP well (i.e., with fidelity) | 60% | 6/10 |
| Sustaining the EBP | 50% | 4/8 |
| Achieving health outcomes | 33% | 3/9 |
The tables below show the recorded outcomes of each study that used this strategy by setting.
Studies That Used Purposefully Reexamine and Adjust the Implementation and Their Outcomes, by Setting
Primary care
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Bravata 2020) | Not tested | Not tested | Not tested | Positive finding |
| (Chinman 2021) | Positive finding | Not tested | Null finding | Null finding |
| (Ornstein 2010) | Positive finding | Not tested | Not tested | Not tested |
| (Quanbeck 2018) | Positive finding | Positive finding | Positive finding | Not tested |
Specialty care
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Williams 2016) | Not tested | Not tested | Null finding | Positive finding |
| (Naar 2022) | Not tested | Not tested | Null finding | Not tested |
Mental health
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Chinman 2017) | Positive finding | Null finding | Not tested | Not tested |
| (Chokron Garneau 2022) | Positive finding | Positive finding | Positive finding | Null finding |
Public Health
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Acosta 2020) | Not tested | Positive finding | Positive finding | Null finding |
| (Chinman 2016) | Not tested | Positive finding | Not tested | Positive finding |
| (Fagan 2012) | Positive finding | Positive finding | Positive finding | Not tested |
| (Luecking 2021) | Not tested | Null finding | Not tested | Not tested |
| (Smelson 2018) | Not tested | Positive finding | Not tested | Not tested |
| (Chinman 2014) | Not tested | Null finding | Not tested | Null finding |
Multiple
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Morton 2020) | Not tested | Not tested | Not tested | Null finding |
Other
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Resnick 2021) | Null finding | Null finding | Null finding | Null finding |
References
Bravata D. Assessment of the Protocol-Guided Rapid Evaluation of Veterans Experiencing New Transient Neurological Symptoms (PREVENT) Program for Improving Quality of Care for Transient Ischemic Attack: A Nonrandomized Cluster Trial. Published online 2020.
Chinman M, Goldberg R, Daniels K, et al. Implementation of peer specialist services in VA primary care: a cluster randomized trial on the impact of external facilitation. Implement Sci. 2021;16(1):60. doi:10.1186/s13012-021-01130-2
Ornstein S, Nemeth LS, Jenkins RG, Nietert PJ. Colorectal Cancer Screening in Primary Care Translating Research Into Practice. Med Care. 2010;48(10). doi:10.1097/MLR.0b013e3181ec5591
Quanbeck A, Brown RT, Zgierska AE, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for Opioid prescribing in primary care. Implement Sci. 2018;13(1):21. doi:10.1186/s13012-018-0713-1
Williams L, Daggett V, Slaven JE, et al. A cluster-randomised quality improvement study to improve two inpatient stroke quality indicators. BMJ Qual Saf. 2016;25(4). doi:10.1136/bmjqs-2015-004188
Naar S, MacDonell K, Chapman J, et al. Tailored Motivational Interviewing in Adolescent HIV Clinics: Primary Outcome Analysis of a Stepped Wedge Implementation Trial. J Acquir Immune Defic Syndr. 2022;90(S1):S74-s83. doi:10.1097/qai.0000000000002967
Chinman M, McCarthy S, Hannah G, Byrne TH, Smelson DA. Using Getting To Outcomes to facilitate the use of an evidence-based practice in VA homeless programs: a cluster-randomized trial of an implementation support strategy. Implement Sci. 2017;12(1):34. doi:10.1186/s13012-017-0565-0
Chokron Garneau H, Assefa MT, Jo B, Ford JH, Saldana L, McGovern MP. Sustainment of Integrated Care in Addiction Treatment Settings: Primary Outcomes From a Cluster-Randomized Controlled Trial. Psychiatr Serv. 2022;73(3):280-286. doi:10.1176/appi.ps.202000293
Acosta J, Chinman M, Ebener PA, Malone PS, Cannon JS, D’Amico EJ. Sustaining an Evidence-Based Program Over Time: Moderators of Sustainability and the Role of the Getting to Outcomes® Implementation Support Intervention. Prev Sci. 2020;21(6):807-819. doi:10.1007/s11121-020-01118-2
Chinman M, Acosta J, Ebener P, Malone PS, Slaughter ME. Can implementation support help community-based settings better deliver evidence-based sexual health promotion programs? A randomized trial of Getting To Outcomes®. Implement Sci IS. 2016;11(1):78. doi:10.1186/s13012-016-0446-y
Fagan AA, Hanson K, Briney JS, Hawkins JD. Sustaining the Utilization and High Quality Implementation of Tested and Effective Prevention Programs Using the Communities That Care Prevention System. Am J Community Psychol. 2012;49(4-Mar). doi:10.1007/s10464-011-9463-9
Luecking CT, Vaughn AE, Burney R, Hennink-Kaminski H, Hales D, Ward DS. Fidelity and factors influencing implementation of Healthy Me, Healthy: process evaluation of a social marketing campaign for diet and physical activity behaviors of children in childcare. Transl Behav Med. 2021;11(3):733-744. doi:10.1093/tbm/ibab001
Smelson DA, Chinman M, Hannah G, Byrne T, McCarthy S. An evidence-based co-occurring disorder intervention in VA homeless programs: outcomes from a hybrid III trial. BMC Health Serv Res. 2018;18(1):332. doi:10.1186/s12913-018-3123-9
Chinman M, Ebener P, Burkhart Q, et al. Evaluating the impact of getting to outcomes-underage drinking on prevention capacity and alcohol merchant attitudes and selling behaviors. Prev Sci. 2014;15(4):485-496. doi:10.1007/s11121-013-0389-3
Morton I, Hurley B, Castillo EG, et al. Outcomes of two quality improvement implementation interventions for depression services in adults with substance use problems. Am J Drug Alcohol Abuse. 2020;46(2):251-261. doi:10.1080/00952990.2019.1708085
Resnick B, Van Haitsma K, Kolanowski A, et al. Implementation of the Evidence Integration Triangle for behavioral and psychological symptoms of dementia (EIT-4-BPSD) in care communities. Nurs Outlook. 2021;69(6):1058-1071. doi:10.1016/j.outlook.2021.06.004
Promote Adaptability Within the EBP
Strategy Definition
Identifying the ways a clinical innovation can be tailored to meet site needs and clarifying which elements of the innovation must be maintained to preserve fidelity. (Source)
This strategy is a form of modification that is planned or made purposefully to maintain the integrity of the core form, functions, or elements of the EBP. In recent years, a proliferation of theoretical and evaluation frameworks has guided researchers and practitioners regarding the steps to plan, document, and evaluate the impact of EBP adaptations for content, contextual fit, culture, and delivery changes.
Strength of Evidence Supporting Its Effectiveness: Supportive
Based on direct and indirect evidence from included studies and contextual systematic review evidence identified outside of this review, we judged evidence to be either supportive, moderate supportive, limited supportive, indirect supportive only, or inconclusive. We determined the strength of evidence supporting the contribution of this strategy to successful implementations was supportive.
Health Care Areas Where the Promote Adaptability Within the EBP Strategy Was Used
Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.
| Area | Percentage of studies | Number of studies |
|---|---|---|
| Primary care | 20% | 3 |
| Specialty care | 7% | 1 |
| Mental health | 7% | 1 |
| Public Health | 40% | 6 |
| Multiple Settings | 13% | 2 |
| Other | 13% | 2 |
Results From Studies That Included Promote Adaptability Within the EBP Strategy
We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.
How Often Were Studies That Included the "Promote Adaptability Within the EBP" Strategy Effective?
| Outcome | Percentage of studies | Number of studies |
|---|---|---|
| Conducting the EBP at all | 50% | 3/6 |
| Performing the EBP well (i.e., with fidelity) | 78% | 7/9 |
| Sustaining the EBP | 57% | 4/7 |
| Achieving health outcomes | 33% | 3/9 |
The tables below show the recorded outcomes of each study that used this strategy by setting.
Studies That Used Promote Adaptability Within the EBP and Their Outcomes, by Setting
Primary care
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Chinman 2021) | Positive finding | Not tested | Null finding | Null finding |
| (Quanbeck 2018) | Positive finding | Positive finding | Positive finding | Not tested |
| (Parchman 2013) | Not tested | Positive finding | Positive finding | Not tested |
Specialty care
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Hahn 2022) | Not tested | Not tested | Positive finding | Not tested |
Mental health
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Bauer 2019) | Not tested | Positive finding | Not tested | Positive finding |
Public Health
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Acosta 2020) | Not tested | Positive finding | Positive finding | Null finding |
| (Chinman 2016) | Not tested | Positive finding | Not tested | Positive finding |
| (Brown 2014) | Null finding | Positive finding | Null finding | Not tested |
| (Jacob 2022) | Null finding | Not tested | Not tested | Not tested |
| (Smelson 2018) | Not tested | Positive finding | Not tested | Not tested |
| (Chinman 2014) | Not tested | Null finding | Not tested | Null finding |
Multiple
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Kilbourne 2015) | Not tested | Not tested | Not tested | Null finding |
| (Morton 2020) | Not tested | Not tested | Not tested | Null finding |
Other
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Resnick 2021) | Null finding | Null finding | Null finding | Null finding |
| (Westrick 2016) | Positive finding | Not tested | Not tested | Positive finding |
References
Chinman M, Goldberg R, Daniels K, et al. Implementation of peer specialist services in VA primary care: a cluster randomized trial on the impact of external facilitation. Implement Sci. 2021;16(1):60. doi:10.1186/s13012-021-01130-2
Quanbeck A, Brown RT, Zgierska AE, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for Opioid prescribing in primary care. Implement Sci. 2018;13(1):21. doi:10.1186/s13012-018-0713-1
Parchman ML, Noel PH, Culler SD, et al. A randomized trial of practice facilitation to improve the delivery of chronic illness care in primary care: initial and sustained effects. Implement Sci IS. 2013;8:93. doi:10.1186/1748-5908-8-93
Hahn EE, Munoz-Plaza CE, Pounds D, et al. Effect of a Community-Based Medical Oncology Depression Screening Program on Behavioral Health Referrals Among Patients With Breast Cancer: A Randomized Clinical Trial. JAMA. 2022;327(1):41-49. doi:10.1001/jama.2021.22596
Bauer MS, Miller CJ, Kim B, et al. Effectiveness of Implementing a Collaborative Chronic Care Model for Clinician Teams on Patient Outcomes and Health Status in Mental Health: A Randomized Clinical Trial. JAMA Netw Open. 2019;2(3):e190230. doi:10.1001/jamanetworkopen.2019.0230
Acosta J, Chinman M, Ebener PA, Malone PS, Cannon JS, D’Amico EJ. Sustaining an Evidence-Based Program Over Time: Moderators of Sustainability and the Role of the Getting to Outcomes® Implementation Support Intervention. Prev Sci. 2020;21(6):807-819. doi:10.1007/s11121-020-01118-2
Chinman M, Acosta J, Ebener P, Malone PS, Slaughter ME. Can implementation support help community-based settings better deliver evidence-based sexual health promotion programs? A randomized trial of Getting To Outcomes®. Implement Sci IS. 2016;11(1):78. doi:10.1186/s13012-016-0446-y
Brown CH, Chamberlain P, Saldana L, Padgett C, Wang W, Cruden G. Evaluation of two implementation strategies in 51 child county public service systems in two states: results of a cluster randomized head-to-head implementation trial. Implement Sci. 2014;9:134. doi:10.1186/s13012-014-0134-8
Jacob RR, Parks RG, Allen P, et al. How to “Start Small and Just Keep Moving Forward”: Mixed Methods Results From a Stepped-Wedge Trial to Support Evidence-Based Processes in Local Health Departments. Front Public Health. 2022;10. doi:10.3389/fpubh.2022.853791
Smelson DA, Chinman M, Hannah G, Byrne T, McCarthy S. An evidence-based co-occurring disorder intervention in VA homeless programs: outcomes from a hybrid III trial. BMC Health Serv Res. 2018;18(1):332. doi:10.1186/s12913-018-3123-9
Chinman M, Ebener P, Burkhart Q, et al. Evaluating the impact of getting to outcomes-underage drinking on prevention capacity and alcohol merchant attitudes and selling behaviors. Prev Sci. 2014;15(4):485-496. doi:10.1007/s11121-013-0389-3
Kilbourne AM, Goodrich DE, Nord KM, et al. Long-Term Clinical Outcomes from a Randomized Controlled Trial of Two Implementation Strategies to Promote Collaborative Care Attendance in Community Practices. Adm Policy Ment Health. 2015;42(5):642-653. doi:10.1007/s10488-014-0598-5
Morton I, Hurley B, Castillo EG, et al. Outcomes of two quality improvement implementation interventions for depression services in adults with substance use problems. Am J Drug Alcohol Abuse. 2020;46(2):251-261. doi:10.1080/00952990.2019.1708085
Resnick B, Van Haitsma K, Kolanowski A, et al. Implementation of the Evidence Integration Triangle for behavioral and psychological symptoms of dementia (EIT-4-BPSD) in care communities. Nurs Outlook. 2021;69(6):1058-1071. doi:10.1016/j.outlook.2021.06.004
Westrick SC, Owen J, Hagel H, Owensby JK, Lertpichitkul T. Impact of the RxVaccinate program for pharmacy-based pneumococcal immunization: A cluster-randomized controlled trial. J Am Pharm Assoc. 2016;56(1). doi:10.1016/j.japh.2015.11.010
Tailor the Implementation Strategy to the Site
Strategy Definition
Customizing implementation strategies to address contextual barriers and leverage facilitators that were identified through earlier data collection. (Source)
This strategy should identify the core components of the effective practice and implementation strategies that are required to maintain or improve effectiveness. It should also identify components that can be modified to improve fit with context or local population. Modifications should be based on a diagnostic assessment of implementation capacity, complexity of the EBP, cultural factors, and barriers and facilitators to EBP implementation.
Strength of Evidence Supporting Its Effectiveness: Inconclusive
Based on direct and indirect evidence from included studies and contextual systematic review evidence identified outside of this review, we judged evidence to be either supportive, moderate supportive, limited supportive, indirect supportive only, or inconclusive. We determined the strength of evidence supporting the contribution of this strategy to successful implementations was inconclusive.
Health Care Areas Where the Tailor the Implementation Strategy to the Site Strategy Was Used
Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.
| Area | Percentage of studies | Number of studies |
|---|---|---|
| Primary care | 25% | 4 |
| Specialty care | 25% | 4 |
| Mental health | 12% | 2 |
| Public Health | 25% | 4 |
| Multiple Settings | 6% | 1 |
| Other | 6% | 1 |
Results From Studies That Included Tailor the Implementation Strategy to the Site Strategy
We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.
How Often Were Studies That Included the "Tailor the Implementation Strategy to the Site" Strategy Effective?
| Outcome | Percentage of studies | Number of studies |
|---|---|---|
| Conducting the EBP at all | 36% | 4/11 |
| Performing the EBP well (i.e., with fidelity) | 56% | 5/9 |
| Sustaining the EBP | 50% | 4/8 |
| Achieving health outcomes | 0% | 0/5 |
The tables below show the recorded outcomes of each study that used this strategy by setting.
Studies That Used Tailor the Implementation Strategy to the Site and Their Outcomes, by Setting
Primary care
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Gold 2019) | Positive finding | Not tested | Not tested | Not tested |
| (Quanbeck 2018) | Positive finding | Positive finding | Positive finding | Not tested |
| (Parchman 2013) | Not tested | Positive finding | Positive finding | Not tested |
| (Estrada 2011) | Null finding | Positive finding | Not tested | Null finding |
Specialty care
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Hahn 2022) | Not tested | Not tested | Positive finding | Not tested |
| (Friese 2019) | Null finding | Null finding | Null finding | Not tested |
| (Kauth 2010) | Null finding | Null finding | Not tested | Not tested |
| (Naar 2022) | Not tested | Not tested | Null finding | Not tested |
Mental health
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Chokron Garneau 2022) | Positive finding | Positive finding | Positive finding | Null finding |
| (Williams 2017) | Positive finding | Positive finding | Not tested | Not tested |
Public Health
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Knott 2022) | Null finding | Not tested | Null finding | Not tested |
| (Maxwell 2016) | Null finding | Null finding | Not tested | Null finding |
| (Jacob 2022) | Null finding | Not tested | Not tested | Not tested |
| (Chinman 2014) | Not tested | Null finding | Not tested | Null finding |
Multiple
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Kilbourne 2015) | Not tested | Not tested | Not tested | Null finding |
Other
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Spoelstra 2022) | Null finding | Not tested | Null finding | Not tested |
References
Gold R, Bunce A, Cowburn S, et al. Does increased implementation support improve community clinics’ guideline-concordant care? Results of a mixed methods, pragmatic comparative effectiveness trial. Implement Sci. 2019;14(1):100. doi:10.1186/s13012-019-0948-5
Quanbeck A, Brown RT, Zgierska AE, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for Opioid prescribing in primary care. Implement Sci. 2018;13(1):21. doi:10.1186/s13012-018-0713-1
Parchman ML, Noel PH, Culler SD, et al. A randomized trial of practice facilitation to improve the delivery of chronic illness care in primary care: initial and sustained effects. Implement Sci IS. 2013;8:93. doi:10.1186/1748-5908-8-93
Estrada CA, Safford MM, Salanitro AH, et al. A web-based diabetes intervention for physician: a cluster-randomized effectiveness trial. Int J Qual Health Care. 2011;23(6):682-689. doi:10.1093/intqhc/mzr053
Hahn EE, Munoz-Plaza CE, Pounds D, et al. Effect of a Community-Based Medical Oncology Depression Screening Program on Behavioral Health Referrals Among Patients With Breast Cancer: A Randomized Clinical Trial. JAMA. 2022;327(1):41-49. doi:10.1001/jama.2021.22596
Friese CR, Yang J, Mendelsohn-Victor K, McCullagh MC. Randomized Controlled Trial of an Intervention to Improve Nurses’ Hazardous Drug Handling. Oncol Nurs Forum. 2019;46(2). doi:10.1188/19.ONF.248-256
Kauth MR, Sullivan G, Blevins D, et al. Employing external facilitation to implement cognitive behavioral therapy in VA clinics: a pilot study. Implement Sci. 2010;5:75. doi:10.1186/1748-5908-5-75
Naar S, MacDonell K, Chapman J, et al. Tailored Motivational Interviewing in Adolescent HIV Clinics: Primary Outcome Analysis of a Stepped Wedge Implementation Trial. J Acquir Immune Defic Syndr. 2022;90(S1):S74-s83. doi:10.1097/qai.0000000000002967
Chokron Garneau H, Assefa MT, Jo B, Ford JH, Saldana L, McGovern MP. Sustainment of Integrated Care in Addiction Treatment Settings: Primary Outcomes From a Cluster-Randomized Controlled Trial. Psychiatr Serv. 2022;73(3):280-286. doi:10.1176/appi.ps.202000293
Williams, N.J., et al., Mechanisms of Change in the ARC Organizational Strategy: Increasing Mental Health Clinicians' EBP Adoption Through Improved Organizational Culture and Capacity. Adm Policy Ment Health, 2017. 44(2): p. 269-283.
Knott CL, Chen C, Bowie JV, et al. Cluster-randomized trial comparing organizationally tailored versus standard approach for integrating an evidence-based cancer control intervention into African American churches. Transl Behav Med. 2022;12(5):673-682. doi:10.1093/tbm/ibab088
Maxwell AE, Danao LL, Cayetano RT, Crespi CM, Bastani R. Implementation of an evidence-based intervention to promote colorectal cancer screening in community organizations: a cluster randomized trial. Transl Behav Med. 2016;6(2):295-305. doi:10.1007/s13142-015-0349-5
Jacob RR, Parks RG, Allen P, et al. How to “Start Small and Just Keep Moving Forward”: Mixed Methods Results From a Stepped-Wedge Trial to Support Evidence-Based Processes in Local Health Departments. Front Public Health. 2022;10. doi:10.3389/fpubh.2022.853791
Chinman M, Ebener P, Burkhart Q, et al. Evaluating the impact of getting to outcomes-underage drinking on prevention capacity and alcohol merchant attitudes and selling behaviors. Prev Sci. 2014;15(4):485-496. doi:10.1007/s11121-013-0389-3
Kilbourne AM, Goodrich DE, Nord KM, et al. Long-Term Clinical Outcomes from a Randomized Controlled Trial of Two Implementation Strategies to Promote Collaborative Care Attendance in Community Practices. Adm Policy Ment Health. 2015;42(5):642-653. doi:10.1007/s10488-014-0598-5
Spoelstra SL, Schueller M, Basso V, Sikorskii A. Results of a multi-site pragmatic hybrid type 3 cluster randomized trial comparing level of facilitation while implementing an intervention in community-dwelling disabled and older adults in a Medicaid waiver. Implement Sci. 2022;17(1):57. doi:10.1186/s13012-022-01232-5
Conduct Cyclical Small Tests of Change
Strategy Definition
Implementing changes in a cyclical fashion (i.e., test, assess, repeat) using small tests of change before implementing changes systemwide. (Source)
This strategy is recommended to be assessed systematically, and results of the tests should be studied to gain lessons for future iterations. These cyclical tests are also referred to as plan-do-study-act (PDSA) cycles, plan-do-check-act, or a define-measure-analyze-improve-control sequence in the Quality Improvement literature.
Strength of Evidence Supporting Its Effectiveness: Indirect Supportive Only
Based on direct and indirect evidence from included studies and contextual systematic review evidence identified outside of this review, we judged evidence to be either supportive, moderate supportive, limited supportive, indirect supportive only, or inconclusive. We determined the strength of evidence supporting the contribution of this strategy to successful implementations was indirect supportive only.
Health Care Areas Where the Conduct Cyclical Small Tests of Change Strategy Was Used
Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.
| Area | Percentage of studies | Number of studies |
|---|---|---|
| Primary care | 42% | 5 |
| Specialty care | 8% | 1 |
| Mental health | 25% | 3 |
| Public Health | 25% | 3 |
| Multiple Settings | 0 | 0 |
| Other | 0 | 0 |
Results From Studies That Included Conduct Cyclical Small Tests of Change Strategy
We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.
How Often Were Studies That Included the "Conduct Cyclical Small Tests of Change" Strategy Effective?
| Outcome | Percentage of studies | Number of studies |
|---|---|---|
| Conducting the EBP at all | 86% | 6/7 |
| Performing the EBP well (i.e., with fidelity) | 100% | 7/7 |
| Sustaining the EBP | 67% | 4/6 |
| Achieving health outcomes | 57% | 4/7 |
The tables below show the recorded outcomes of each study that used this strategy by setting.
Studies That Used Conduct Cyclical Small Tests of Change and Their Outcomes, by Setting
Primary care
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Bravata 2020) | Not tested | Not tested | Not tested | Positive finding |
| (Lowery 2022) | Null finding | Not tested | Null finding | Not tested |
| (Ornstein 2010) | Positive finding | Not tested | Not tested | Not tested |
| (Quanbeck 2018) | Positive finding | Positive finding | Positive finding | Not tested |
| (Noël 2022) | Positive finding | Not tested | Not tested | Null finding |
Specialty care
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Williams 2016) | Not tested | Not tested | Null finding | Positive finding |
Mental health
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Bauer 2019) | Not tested | Positive finding | Not tested | Positive finding |
| (Chokron Garneau 2022) | Positive finding | Positive finding | Positive finding | Null finding |
| (Williams 2017) | Positive finding | Positive finding | Not tested | Not tested |
Public Health
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Acosta 2020) | Not tested | Positive finding | Positive finding | Null finding |
| (Chinman 2016) | Not tested | Positive finding | Not tested | Positive finding |
| (Fagan 2012) | Positive finding | Positive finding | Positive finding | Not tested |
References
Bravata D. Assessment of the Protocol-Guided Rapid Evaluation of Veterans Experiencing New Transient Neurological Symptoms (PREVENT) Program for Improving Quality of Care for Transient Ischemic Attack: A Nonrandomized Cluster Trial. Published online 2020.
Lowery J, Fagerlin A, Larkin AR, Wiener RS, Skurla SE, Caverly TJ. Implementation of a Web-Based Tool for Shared Decision-making in Lung Cancer Screening: Mixed Methods Quality Improvement Evaluation. JMIR Hum Factors. 2022;9(2):e32399. doi:10.2196/32399
Ornstein S, Nemeth LS, Jenkins RG, Nietert PJ. Colorectal Cancer Screening in Primary Care Translating Research Into Practice. Med Care. 2010;48(10). doi:10.1097/MLR.0b013e3181ec5591
Quanbeck A, Brown RT, Zgierska AE, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for Opioid prescribing in primary care. Implement Sci. 2018;13(1):21. doi:10.1186/s13012-018-0713-1
Noël PH, Barnard JM, Leng M, et al. The Coordination Toolkit and Coaching Project: Cluster-Randomized Quality Improvement Initiative to Improve Patient Experience of Care Coordination. J Gen Intern Med. 2022;37(1):95-103. doi:10.1007/s11606-021-06926-y
Williams L, Daggett V, Slaven JE, et al. A cluster-randomised quality improvement study to improve two inpatient stroke quality indicators. BMJ Qual Saf. 2016;25(4). doi:10.1136/bmjqs-2015-004188
Bauer MS, Miller CJ, Kim B, et al. Effectiveness of Implementing a Collaborative Chronic Care Model for Clinician Teams on Patient Outcomes and Health Status in Mental Health: A Randomized Clinical Trial. JAMA Netw Open. 2019;2(3):e190230. doi:10.1001/jamanetworkopen.2019.0230
Chokron Garneau H, Assefa MT, Jo B, Ford JH, Saldana L, McGovern MP. Sustainment of Integrated Care in Addiction Treatment Settings: Primary Outcomes From a Cluster-Randomized Controlled Trial. Psychiatr Serv. 2022;73(3):280-286. doi:10.1176/appi.ps.202000293
Williams, N.J., et al., Mechanisms of Change in the ARC Organizational Strategy: Increasing Mental Health Clinicians' EBP Adoption Through Improved Organizational Culture and Capacity. Adm Policy Ment Health, 2017. 44(2): p. 269-283.
Acosta J, Chinman M, Ebener PA, Malone PS, Cannon JS, D’Amico EJ. Sustaining an Evidence-Based Program Over Time: Moderators of Sustainability and the Role of the Getting to Outcomes® Implementation Support Intervention. Prev Sci. 2020;21(6):807-819. doi:10.1007/s11121-020-01118-2
Chinman M, Acosta J, Ebener P, Malone PS, Slaughter ME. Can implementation support help community-based settings better deliver evidence-based sexual health promotion programs? A randomized trial of Getting To Outcomes®. Implement Sci IS. 2016;11(1):78. doi:10.1186/s13012-016-0446-y
Fagan AA, Hanson K, Briney JS, Hawkins JD. Sustaining the Utilization and High Quality Implementation of Tested and Effective Prevention Programs Using the Communities That Care Prevention System. Am J Community Psychol. 2012;49(4-Mar). doi:10.1007/s10464-011-9463-9
Assess and Redesign Workflows
Strategy Definition
Observing and mapping current work processes, planning for desired work processes, and identifying necessary changes to accommodate, encourage, or incentivize use of the clinical innovation as designed. (Source)
This strategy may comprise considerations of perceived process (what implementers think is happening), reality process (what the process actually is), and ideal process (what the process could be).
Strength of Evidence Supporting Its Effectiveness: Moderate Supportive
Based on direct and indirect evidence from included studies and contextual systematic review evidence identified outside of this review, we judged evidence to be either supportive, moderate supportive, limited supportive, indirect supportive only, or inconclusive. We determined the strength of evidence supporting the contribution of this strategy to successful implementations was moderate supportive.
Health Care Areas Where the Assess and Redesign Workflows Strategy Was Used
Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.
| Area | Percentage of studies | Number of studies |
|---|---|---|
| Primary care | 70% | 7 |
| Specialty care | 10% | 1 |
| Mental health | 20% | 2 |
| Public Health | 0 | 0 |
| Multiple Settings | 0 | 0 |
| Other | 0 | 0 |
Results From Studies That Included Assess and Redesign Workflows Strategy
We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.
How Often Were Studies That Included the "Assess and Redesign Workflows" Strategy Effective?
| Outcome | Percentage of studies | Number of studies |
|---|---|---|
| Conducting the EBP at all | 83% | 5/6 |
| Performing the EBP well (ie, with fidelity) | 100% | 6/6 |
| Sustaining the EBP | 100% | 4/4 |
| Achieving health outcomes | 50% | 3/6 |
The tables below show the recorded outcomes of each study that used this strategy by setting.
Studies That Used Assess and Redesign Workflows and Their Outcomes, by Setting
Primary care
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Dickinson 2019) | Positive finding | Not tested | Not tested | Null finding |
| (Bravata 2020) | Not tested | Not tested | Not tested | Positive finding |
| (Carter 2018) | Positive finding | Positive finding | Not tested | Positive finding |
| (Ornstein 2010) | Positive finding | Not tested | Not tested | Not tested |
| (Quanbeck 2018) | Positive finding | Positive finding | Positive finding | Not tested |
| (Parchman 2013) | Not tested | Positive finding | Positive finding | Not tested |
| (Estrada 2011) | Null finding | Positive finding | Not tested | Null finding |
Specialty care
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Hahn 2022) | Not tested | Not tested | Positive finding | Not tested |
Mental health
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Bauer 2019) | Not tested | Positive finding | Not tested | Positive finding |
| (Chokron Garneau 2022) | Positive finding | Positive finding | Positive finding | Null finding |
References
Dickinson WP, Dickinson LM, Jortberg BT, et al. A Cluster Randomized Trial Comparing Strategies for Translating Self-Management Support into Primary Care Practices. J Am Board Fam Med JABFM. 2019;32(3):341-352. doi:10.3122/jabfm.2019.03.180254
Bravata D. Assessment of the Protocol-Guided Rapid Evaluation of Veterans Experiencing New Transient Neurological Symptoms (PREVENT) Program for Improving Quality of Care for Transient Ischemic Attack: A Nonrandomized Cluster Trial. Published online 2020.
Carter BL, Levy B, Gryzlak B, et al. Cluster-Randomized Trial to Evaluate a Centralized Clinical Pharmacy Service in Private Family Medicine Offices. Circ-Cardiovasc Qual Outcomes. 2018;11(6). doi:10.1161/CIRCOUTCOMES.117.004188
Ornstein S, Nemeth LS, Jenkins RG, Nietert PJ. Colorectal Cancer Screening in Primary Care Translating Research Into Practice. Med Care. 2010;48(10). doi:10.1097/MLR.0b013e3181ec5591
Quanbeck A, Brown RT, Zgierska AE, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for Opioid prescribing in primary care. Implement Sci. 2018;13(1):21. doi:10.1186/s13012-018-0713-1
Parchman ML, Noel PH, Culler SD, et al. A randomized trial of practice facilitation to improve the delivery of chronic illness care in primary care: initial and sustained effects. Implement Sci IS. 2013;8:93. doi:10.1186/1748-5908-8-93
Estrada CA, Safford MM, Salanitro AH, et al. A web-based diabetes intervention for physician: a cluster-randomized effectiveness trial. Int J Qual Health Care. 2011;23(6):682-689. doi:10.1093/intqhc/mzr053
Hahn EE, Munoz-Plaza CE, Pounds D, et al. Effect of a Community-Based Medical Oncology Depression Screening Program on Behavioral Health Referrals Among Patients With Breast Cancer: A Randomized Clinical Trial. JAMA. 2022;327(1):41-49. doi:10.1001/jama.2021.22596
Bauer MS, Miller CJ, Kim B, et al. Effectiveness of Implementing a Collaborative Chronic Care Model for Clinician Teams on Patient Outcomes and Health Status in Mental Health: A Randomized Clinical Trial. JAMA Netw Open. 2019;2(3):e190230. doi:10.1001/jamanetworkopen.2019.0230
Chokron Garneau H, Assefa MT, Jo B, Ford JH, Saldana L, McGovern MP. Sustainment of Integrated Care in Addiction Treatment Settings: Primary Outcomes From a Cluster-Randomized Controlled Trial. Psychiatr Serv. 2022;73(3):280-286. doi:10.1176/appi.ps.202000293
Obtain Formal Commitments from Leadership to Implement Change
Strategy Definition
Obtaining written commitments from key leaders that outline their roles in implementing the effective practice. (Source)
This strategy should clarify roles, responsibilities, and communicate tangible and non-tangible benefits (e.g., community partnerships). These roles should also be supported within the organization (e.g., workload release credit for providing and receiving supervision in a new clinical practice).
Strength of Evidence Supporting Its Effectiveness: Not Evaluated
For some of the most common strategies we judged the strength of research support based on included studies and contextual systematic review evidence identified outside of this review. We did not evaluate the strength of evidence for this strategy. The information below represents the data we collected from studies including this strategy.
Health Care Areas Where the Obtain Formal Commitments from Leadership to Implement Change Strategy Was Used
Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.
| Area | Percentage of studies | Number of studies |
|---|---|---|
| Primary care | 0 | 0 |
| Specialty care | 0 | 0 |
| Mental health | 0 | 0 |
| Public Health | 100% | 1 |
| Multiple Settings | 0 | 0 |
| Other | 0 | 0 |
Results From Studies That Included Obtain Formal Commitments from Leadership to Implement Change Strategy
We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.
How Often Were Studies That Included the "Obtain Formal Commitments from Leadership to Implement Change" Strategy Effective?
| Outcome | Percentage of studies | Number of studies |
|---|---|---|
| Conducting the EBP at all | 0% | 0/1 |
| Performing the EBP well (i.e., with fidelity) | 0% | 0/0 |
| Sustaining the EBP | 0% | 0/1 |
| Achieving health outcomes | 0% | 0/0 |
The tables below show the recorded outcomes of each study that used this strategy by setting.
Studies That Used Obtain Formal Commitments from Leadership to Implement Change and Their Outcomes, by Setting
Public Health
| Study | Conduct EBP | Perform EBP well | Sustain EBP | Health outcomes |
|---|---|---|---|---|
| (Knott 2022) | Null finding | Not tested | Null finding | Not tested |
References
Knott CL, Chen C, Bowie JV, et al. Cluster-randomized trial comparing organizationally tailored versus standard approach for integrating an evidence-based cancer control intervention into African American churches. Transl Behav Med. 2022;12(5):673-682. doi:10.1093/tbm/ibab088