Several strategies have been developed and used so that evidence-based practice (EBP) efforts better fit the particular and sometimes evolving context and circumstances of the sites where they occur. These strategies are often dynamic and can be focused on the practice to be implemented or the implementation strategy. We include evidence related to six strategies under this category. Select any of the strategies in the box below to learn more.

Download the Full Report

Purposefully Reexamine and Adjust the Implementation


Strategy Definition

This is a strategy that monitors progress and adjusts effective practices and implementation strategies to continuously improve the quality of care. (Source)

This strategy underscores the value of using a concrete schedule for monitoring implementation progress rather than as-needed reevaluation. This should include time-sensitive benchmarks or milestone dates to anticipate when adjustments might be needed.

Strength of Evidence Supporting Its Effectiveness: Inconclusive

Based on direct and indirect evidence from included studies and contextual systematic review evidence identified outside of this review, we judged evidence to be either supportive, moderate supportive, limited supportive, indirect supportive only, or inconclusive. We determined the strength of evidence supporting the contribution of this strategy to successful implementations was inconclusive.

Health Care Areas Where the Purposefully Reexamine and Adjust the Implementation Strategy Was Used

Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.

AreaPercentage of studiesNumber of studies
Mental health12%2
Multiple Settings6%1
Primary care25%4
Public health38%6
Specialty care12%2
Other6%1

Results From Studies That Included Purposefully Reexamine and Adjust the Implementation Strategy

We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.

How Often Were Studies That Included the "Purposefully Reexamine and Adjust the Implementation" Strategy Effective?

OutcomePercentage of studiesNumber of studies
Conducting the EBP at alcohol86%6/7
Performing the EBP well (i.e., with fidelity)60%6/10
Sustaining the EBP50%4/8
Achieving health outcomes33%3/9

The tables below show the recorded outcomes of each study that used this strategy by setting.

Studies That Used Purposefully Reexamine and Adjust the Implementation and Their Outcomes, by Setting

Primary care

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Bravata 2020)Not testedNot testedNot testedPositive
finding
(Chinman 2021)Positive
finding
Not testedNull findingNull finding
(Ornstein 2010)Positive
finding
Not testedNot testedNot tested
(Quanbeck 2018)Positive
finding
Positive
finding
Positive
finding
Not tested

Specialty care

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Williams 2016)Not testedNot testedNull findingPositive
finding
(Naar 2022)Not testedNot testedNull findingNot tested

Mental health

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Chinman 2017)Positive
finding
Null findingNot testedNot tested
(Chokron Garneau 2022)Positive
finding
Positive
finding
Positive
finding
Null finding

Public Health

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Acosta 2020)Not testedPositive
finding
Positive
finding
Null finding
(Chinman 2016)Not testedPositive
finding
Not testedPositive
finding
(Fagan 2012)Positive
finding
Positive
finding
Positive
finding
Not tested
(Luecking 2021)Not testedNull findingNot testedNot tested
(Smelson 2018)Not testedPositive
finding
Not testedNot tested
(Chinman 2014)Not testedNull findingNot testedNull finding

Multiple

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Morton 2020)Not testedNot testedNot testedNull finding

Other

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Resnick 2021)Null findingNull findingNull findingNull finding

References

Bravata D. Assessment of the Protocol-Guided Rapid Evaluation of Veterans Experiencing New Transient Neurological Symptoms (PREVENT) Program for Improving Quality of Care for Transient Ischemic Attack: A Nonrandomized Cluster Trial. Published online 2020.

Chinman M, Goldberg R, Daniels K, et al. Implementation of peer specialist services in VA primary care: a cluster randomized trial on the impact of external facilitation. Implement Sci. 2021;16(1):60. doi:10.1186/s13012-021-01130-2

Ornstein S, Nemeth LS, Jenkins RG, Nietert PJ. Colorectal Cancer Screening in Primary Care Translating Research Into Practice. Med Care. 2010;48(10). doi:10.1097/MLR.0b013e3181ec5591

Quanbeck A, Brown RT, Zgierska AE, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for Opioid prescribing in primary care. Implement Sci. 2018;13(1):21. doi:10.1186/s13012-018-0713-1

Williams L, Daggett V, Slaven JE, et al. A cluster-randomised quality improvement study to improve two inpatient stroke quality indicators. BMJ Qual Saf. 2016;25(4). doi:10.1136/bmjqs-2015-004188

Naar S, MacDonell K, Chapman J, et al. Tailored Motivational Interviewing in Adolescent HIV Clinics: Primary Outcome Analysis of a Stepped Wedge Implementation Trial. J Acquir Immune Defic Syndr. 2022;90(S1):S74-s83. doi:10.1097/qai.0000000000002967

Chinman M, McCarthy S, Hannah G, Byrne TH, Smelson DA. Using Getting To Outcomes to facilitate the use of an evidence-based practice in VA homeless programs: a cluster-randomized trial of an implementation support strategy. Implement Sci. 2017;12(1):34. doi:10.1186/s13012-017-0565-0

Chokron Garneau H, Assefa MT, Jo B, Ford JH, Saldana L, McGovern MP. Sustainment of Integrated Care in Addiction Treatment Settings: Primary Outcomes From a Cluster-Randomized Controlled Trial. Psychiatr Serv. 2022;73(3):280-286. doi:10.1176/appi.ps.202000293

Acosta J, Chinman M, Ebener PA, Malone PS, Cannon JS, D’Amico EJ. Sustaining an Evidence-Based Program Over Time: Moderators of Sustainability and the Role of the Getting to Outcomes® Implementation Support Intervention. Prev Sci. 2020;21(6):807-819. doi:10.1007/s11121-020-01118-2

Chinman M, Acosta J, Ebener P, Malone PS, Slaughter ME. Can implementation support help community-based settings better deliver evidence-based sexual health promotion programs? A randomized trial of Getting To Outcomes®. Implement Sci IS. 2016;11(1):78. doi:10.1186/s13012-016-0446-y

Fagan AA, Hanson K, Briney JS, Hawkins JD. Sustaining the Utilization and High Quality Implementation of Tested and Effective Prevention Programs Using the Communities That Care Prevention System. Am J Community Psychol. 2012;49(4-Mar). doi:10.1007/s10464-011-9463-9

Luecking CT, Vaughn AE, Burney R, Hennink-Kaminski H, Hales D, Ward DS. Fidelity and factors influencing implementation of Healthy Me, Healthy: process evaluation of a social marketing campaign for diet and physical activity behaviors of children in childcare. Transl Behav Med. 2021;11(3):733-744. doi:10.1093/tbm/ibab001

Smelson DA, Chinman M, Hannah G, Byrne T, McCarthy S. An evidence-based co-occurring disorder intervention in VA homeless programs: outcomes from a hybrid III trial. BMC Health Serv Res. 2018;18(1):332. doi:10.1186/s12913-018-3123-9

Chinman M, Ebener P, Burkhart Q, et al. Evaluating the impact of getting to outcomes-underage drinking on prevention capacity and alcohol merchant attitudes and selling behaviors. Prev Sci. 2014;15(4):485-496. doi:10.1007/s11121-013-0389-3

Morton I, Hurley B, Castillo EG, et al. Outcomes of two quality improvement implementation interventions for depression services in adults with substance use problems. Am J Drug Alcohol Abuse. 2020;46(2):251-261. doi:10.1080/00952990.2019.1708085

Resnick B, Van Haitsma K, Kolanowski A, et al. Implementation of the Evidence Integration Triangle for behavioral and psychological symptoms of dementia (EIT-4-BPSD) in care communities. Nurs Outlook. 2021;69(6):1058-1071. doi:10.1016/j.outlook.2021.06.004

Promote Adaptability Within the EBP


Strategy Definition

Identifying the ways a clinical innovation can be tailored to meet site needs and clarifying which elements of the innovation must be maintained to preserve fidelity. (Source)

This strategy is a form of modification that is planned or made purposefully to maintain the integrity of the core form, functions, or elements of the EBP. In recent years, a proliferation of theoretical and evaluation frameworks has guided researchers and practitioners regarding the steps to plan, document, and evaluate the impact of EBP adaptations for content, contextual fit, culture, and delivery changes.

Strength of Evidence Supporting Its Effectiveness: Supportive

Based on direct and indirect evidence from included studies and contextual systematic review evidence identified outside of this review, we judged evidence to be either supportive, moderate supportive, limited supportive, indirect supportive only, or inconclusive. We determined the strength of evidence supporting the contribution of this strategy to successful implementations was supportive.

Health Care Areas Where the Promote Adaptability Within the EBP Strategy Was Used

Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.

AreaPercentage of studiesNumber of studies
Primary care20%3
Specialty care7%1
Mental health7%1
Public Health40%6
Multiple Settings13%2
Other13%2

Results From Studies That Included Promote Adaptability Within the EBP Strategy

We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.

How Often Were Studies That Included the "Promote Adaptability Within the EBP" Strategy Effective?

OutcomePercentage of studiesNumber of studies
Conducting the EBP at all50%3/6
Performing the EBP well (i.e., with fidelity)78%7/9
Sustaining the EBP57%4/7
Achieving health outcomes33%3/9

The tables below show the recorded outcomes of each study that used this strategy by setting.

Studies That Used Promote Adaptability Within the EBP and Their Outcomes, by Setting

Primary care

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Chinman 2021)Positive
finding
Not testedNull findingNull finding
(Quanbeck 2018)Positive
finding
Positive
finding
Positive
finding
Not tested
(Parchman 2013)Not testedPositive
finding
Positive
finding
Not tested

Specialty care

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Hahn 2022)Not testedNot testedPositive
finding
Not tested

Mental health

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Bauer 2019)Not testedPositive
finding
Not testedPositive
finding

Public Health

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Acosta 2020)Not testedPositive
finding
Positive
finding
Null finding
(Chinman 2016)Not testedPositive
finding
Not testedPositive
finding
(Brown 2014)Null findingPositive
finding
Null findingNot tested
(Jacob 2022)Null findingNot testedNot testedNot tested
(Smelson 2018)Not testedPositive
finding
Not testedNot tested
(Chinman 2014)Not testedNull findingNot testedNull finding

Multiple

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Kilbourne 2015)Not testedNot testedNot testedNull finding
(Morton 2020)Not testedNot testedNot testedNull finding

Other

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Resnick 2021)Null findingNull findingNull findingNull finding
(Westrick 2016)Positive
finding
Not testedNot testedPositive
finding

References

Chinman M, Goldberg R, Daniels K, et al. Implementation of peer specialist services in VA primary care: a cluster randomized trial on the impact of external facilitation. Implement Sci. 2021;16(1):60. doi:10.1186/s13012-021-01130-2

Quanbeck A, Brown RT, Zgierska AE, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for Opioid prescribing in primary care. Implement Sci. 2018;13(1):21. doi:10.1186/s13012-018-0713-1

Parchman ML, Noel PH, Culler SD, et al. A randomized trial of practice facilitation to improve the delivery of chronic illness care in primary care: initial and sustained effects. Implement Sci IS. 2013;8:93. doi:10.1186/1748-5908-8-93

Hahn EE, Munoz-Plaza CE, Pounds D, et al. Effect of a Community-Based Medical Oncology Depression Screening Program on Behavioral Health Referrals Among Patients With Breast Cancer: A Randomized Clinical Trial. JAMA. 2022;327(1):41-49. doi:10.1001/jama.2021.22596

Bauer MS, Miller CJ, Kim B, et al. Effectiveness of Implementing a Collaborative Chronic Care Model for Clinician Teams on Patient Outcomes and Health Status in Mental Health: A Randomized Clinical Trial. JAMA Netw Open. 2019;2(3):e190230. doi:10.1001/jamanetworkopen.2019.0230

Acosta J, Chinman M, Ebener PA, Malone PS, Cannon JS, D’Amico EJ. Sustaining an Evidence-Based Program Over Time: Moderators of Sustainability and the Role of the Getting to Outcomes® Implementation Support Intervention. Prev Sci. 2020;21(6):807-819. doi:10.1007/s11121-020-01118-2

Chinman M, Acosta J, Ebener P, Malone PS, Slaughter ME. Can implementation support help community-based settings better deliver evidence-based sexual health promotion programs? A randomized trial of Getting To Outcomes®. Implement Sci IS. 2016;11(1):78. doi:10.1186/s13012-016-0446-y

Brown CH, Chamberlain P, Saldana L, Padgett C, Wang W, Cruden G. Evaluation of two implementation strategies in 51 child county public service systems in two states: results of a cluster randomized head-to-head implementation trial. Implement Sci. 2014;9:134. doi:10.1186/s13012-014-0134-8

Jacob RR, Parks RG, Allen P, et al. How to “Start Small and Just Keep Moving Forward”: Mixed Methods Results From a Stepped-Wedge Trial to Support Evidence-Based Processes in Local Health Departments. Front Public Health. 2022;10. doi:10.3389/fpubh.2022.853791

Smelson DA, Chinman M, Hannah G, Byrne T, McCarthy S. An evidence-based co-occurring disorder intervention in VA homeless programs: outcomes from a hybrid III trial. BMC Health Serv Res. 2018;18(1):332. doi:10.1186/s12913-018-3123-9

Chinman M, Ebener P, Burkhart Q, et al. Evaluating the impact of getting to outcomes-underage drinking on prevention capacity and alcohol merchant attitudes and selling behaviors. Prev Sci. 2014;15(4):485-496. doi:10.1007/s11121-013-0389-3

Kilbourne AM, Goodrich DE, Nord KM, et al. Long-Term Clinical Outcomes from a Randomized Controlled Trial of Two Implementation Strategies to Promote Collaborative Care Attendance in Community Practices. Adm Policy Ment Health. 2015;42(5):642-653. doi:10.1007/s10488-014-0598-5

Morton I, Hurley B, Castillo EG, et al. Outcomes of two quality improvement implementation interventions for depression services in adults with substance use problems. Am J Drug Alcohol Abuse. 2020;46(2):251-261. doi:10.1080/00952990.2019.1708085

Resnick B, Van Haitsma K, Kolanowski A, et al. Implementation of the Evidence Integration Triangle for behavioral and psychological symptoms of dementia (EIT-4-BPSD) in care communities. Nurs Outlook. 2021;69(6):1058-1071. doi:10.1016/j.outlook.2021.06.004

Westrick SC, Owen J, Hagel H, Owensby JK, Lertpichitkul T. Impact of the RxVaccinate program for pharmacy-based pneumococcal immunization: A cluster-randomized controlled trial. J Am Pharm Assoc. 2016;56(1). doi:10.1016/j.japh.2015.11.010

Tailor the Implementation Strategy to the Site


Strategy Definition

Customizing implementation strategies to address contextual barriers and leverage facilitators that were identified through earlier data collection. (Source)

This strategy should identify the core components of the effective practice and implementation strategies that are required to maintain or improve effectiveness. It should also identify components that can be modified to improve fit with context or local population. Modifications should be based on a diagnostic assessment of implementation capacity, complexity of the EBP, cultural factors, and barriers and facilitators to EBP implementation.

Strength of Evidence Supporting Its Effectiveness: Inconclusive

Based on direct and indirect evidence from included studies and contextual systematic review evidence identified outside of this review, we judged evidence to be either supportive, moderate supportive, limited supportive, indirect supportive only, or inconclusive. We determined the strength of evidence supporting the contribution of this strategy to successful implementations was inconclusive.

Health Care Areas Where the Tailor the Implementation Strategy to the Site Strategy Was Used

Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.

AreaPercentage of studiesNumber of studies
Primary care25%4
Specialty care25%4
Mental health12%2
Public Health25%4
Multiple Settings6%1
Other6%1

Results From Studies That Included Tailor the Implementation Strategy to the Site Strategy

We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.

How Often Were Studies That Included the "Tailor the Implementation Strategy to the Site" Strategy Effective?

OutcomePercentage of studiesNumber of studies
Conducting the EBP at all36%4/11
Performing the EBP well (i.e., with fidelity)56%5/9
Sustaining the EBP50%4/8
Achieving health outcomes0%0/5

The tables below show the recorded outcomes of each study that used this strategy by setting.

Studies That Used Tailor the Implementation Strategy to the Site and Their Outcomes, by Setting

Primary care

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Gold 2019)Positive
finding
Not testedNot testedNot tested
(Quanbeck 2018)Positive
finding
Positive
finding
Positive
finding
Not tested
(Parchman 2013)Not testedPositive
finding
Positive
finding
Not tested
(Estrada 2011)Null findingPositive
finding
Not testedNull finding

Specialty care

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Hahn 2022)Not testedNot testedPositive
finding
Not tested
(Friese 2019)Null findingNull findingNull findingNot tested
(Kauth 2010)Null findingNull findingNot testedNot tested
(Naar 2022)Not testedNot testedNull findingNot tested

Mental health

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Chokron Garneau 2022)Positive
finding
Positive
finding
Positive
finding
Null finding
(Williams 2017)Positive
finding
Positive
finding
Not testedNot tested

Public Health

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Knott 2022)Null findingNot testedNull findingNot tested
(Maxwell 2016)Null findingNull findingNot testedNull finding
(Jacob 2022)Null findingNot testedNot testedNot tested
(Chinman 2014)Not testedNull findingNot testedNull finding

Multiple

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Kilbourne 2015)Not testedNot testedNot testedNull finding

Other

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Spoelstra 2022)Null findingNot testedNull findingNot tested

References

Gold R, Bunce A, Cowburn S, et al. Does increased implementation support improve community clinics’ guideline-concordant care? Results of a mixed methods, pragmatic comparative effectiveness trial. Implement Sci. 2019;14(1):100. doi:10.1186/s13012-019-0948-5

Quanbeck A, Brown RT, Zgierska AE, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for Opioid prescribing in primary care. Implement Sci. 2018;13(1):21. doi:10.1186/s13012-018-0713-1

Parchman ML, Noel PH, Culler SD, et al. A randomized trial of practice facilitation to improve the delivery of chronic illness care in primary care: initial and sustained effects. Implement Sci IS. 2013;8:93. doi:10.1186/1748-5908-8-93

Estrada CA, Safford MM, Salanitro AH, et al. A web-based diabetes intervention for physician: a cluster-randomized effectiveness trial. Int J Qual Health Care. 2011;23(6):682-689. doi:10.1093/intqhc/mzr053

Hahn EE, Munoz-Plaza CE, Pounds D, et al. Effect of a Community-Based Medical Oncology Depression Screening Program on Behavioral Health Referrals Among Patients With Breast Cancer: A Randomized Clinical Trial. JAMA. 2022;327(1):41-49. doi:10.1001/jama.2021.22596

Friese CR, Yang J, Mendelsohn-Victor K, McCullagh MC. Randomized Controlled Trial of an Intervention to Improve Nurses’ Hazardous Drug Handling. Oncol Nurs Forum. 2019;46(2). doi:10.1188/19.ONF.248-256

Kauth MR, Sullivan G, Blevins D, et al. Employing external facilitation to implement cognitive behavioral therapy in VA clinics: a pilot study. Implement Sci. 2010;5:75. doi:10.1186/1748-5908-5-75

Naar S, MacDonell K, Chapman J, et al. Tailored Motivational Interviewing in Adolescent HIV Clinics: Primary Outcome Analysis of a Stepped Wedge Implementation Trial. J Acquir Immune Defic Syndr. 2022;90(S1):S74-s83. doi:10.1097/qai.0000000000002967

Chokron Garneau H, Assefa MT, Jo B, Ford JH, Saldana L, McGovern MP. Sustainment of Integrated Care in Addiction Treatment Settings: Primary Outcomes From a Cluster-Randomized Controlled Trial. Psychiatr Serv. 2022;73(3):280-286. doi:10.1176/appi.ps.202000293

Williams, N.J., et al., Mechanisms of Change in the ARC Organizational Strategy: Increasing Mental Health Clinicians' EBP Adoption Through Improved Organizational Culture and Capacity. Adm Policy Ment Health, 2017. 44(2): p. 269-283.

Knott CL, Chen C, Bowie JV, et al. Cluster-randomized trial comparing organizationally tailored versus standard approach for integrating an evidence-based cancer control intervention into African American churches. Transl Behav Med. 2022;12(5):673-682. doi:10.1093/tbm/ibab088

Maxwell AE, Danao LL, Cayetano RT, Crespi CM, Bastani R. Implementation of an evidence-based intervention to promote colorectal cancer screening in community organizations: a cluster randomized trial. Transl Behav Med. 2016;6(2):295-305. doi:10.1007/s13142-015-0349-5

Jacob RR, Parks RG, Allen P, et al. How to “Start Small and Just Keep Moving Forward”: Mixed Methods Results From a Stepped-Wedge Trial to Support Evidence-Based Processes in Local Health Departments. Front Public Health. 2022;10. doi:10.3389/fpubh.2022.853791

Chinman M, Ebener P, Burkhart Q, et al. Evaluating the impact of getting to outcomes-underage drinking on prevention capacity and alcohol merchant attitudes and selling behaviors. Prev Sci. 2014;15(4):485-496. doi:10.1007/s11121-013-0389-3

Kilbourne AM, Goodrich DE, Nord KM, et al. Long-Term Clinical Outcomes from a Randomized Controlled Trial of Two Implementation Strategies to Promote Collaborative Care Attendance in Community Practices. Adm Policy Ment Health. 2015;42(5):642-653. doi:10.1007/s10488-014-0598-5

Spoelstra SL, Schueller M, Basso V, Sikorskii A. Results of a multi-site pragmatic hybrid type 3 cluster randomized trial comparing level of facilitation while implementing an intervention in community-dwelling disabled and older adults in a Medicaid waiver. Implement Sci. 2022;17(1):57. doi:10.1186/s13012-022-01232-5

Conduct Cyclical Small Tests of Change


Strategy Definition

Implementing changes in a cyclical fashion (i.e., test, assess, repeat) using small tests of change before implementing changes systemwide. (Source)

This strategy is recommended to be assessed systematically, and results of the tests should be studied to gain lessons for future iterations. These cyclical tests are also referred to as plan-do-study-act (PDSA) cycles, plan-do-check-act, or a define-measure-analyze-improve-control sequence in the Quality Improvement literature.

Strength of Evidence Supporting Its Effectiveness: Indirect Supportive Only

Based on direct and indirect evidence from included studies and contextual systematic review evidence identified outside of this review, we judged evidence to be either supportive, moderate supportive, limited supportive, indirect supportive only, or inconclusive. We determined the strength of evidence supporting the contribution of this strategy to successful implementations was indirect supportive only.

Health Care Areas Where the Conduct Cyclical Small Tests of Change Strategy Was Used

Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.

AreaPercentage of studiesNumber of studies
Primary care42%5
Specialty care8%1
Mental health25%3
Public Health25%3
Multiple Settings00
Other00

Results From Studies That Included Conduct Cyclical Small Tests of Change Strategy

We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.

How Often Were Studies That Included the "Conduct Cyclical Small Tests of Change" Strategy Effective?

OutcomePercentage of studiesNumber of studies
Conducting the EBP at all86%6/7
Performing the EBP well (i.e., with fidelity)100%7/7
Sustaining the EBP67%4/6
Achieving health outcomes57%4/7

The tables below show the recorded outcomes of each study that used this strategy by setting.

Studies That Used Conduct Cyclical Small Tests of Change and Their Outcomes, by Setting

Primary care

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Bravata 2020)Not testedNot testedNot testedPositive
finding
(Lowery 2022)Null findingNot testedNull findingNot tested
(Ornstein 2010)Positive
finding
Not testedNot testedNot tested
(Quanbeck 2018)Positive
finding
Positive
finding
Positive
finding
Not tested
(Noël 2022)Positive
finding
Not testedNot testedNull finding

Specialty care

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Williams 2016)Not testedNot testedNull findingPositive
finding

Mental health

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Bauer 2019)Not testedPositive
finding
Not testedPositive
finding
(Chokron Garneau 2022)Positive
finding
Positive
finding
Positive
finding
Null finding
(Williams 2017)Positive
finding
Positive
finding
Not testedNot tested

Public Health

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Acosta 2020)Not testedPositive
finding
Positive
finding
Null finding
(Chinman 2016)Not testedPositive
finding
Not testedPositive
finding
(Fagan 2012)Positive
finding
Positive
finding
Positive
finding
Not tested

References

Bravata D. Assessment of the Protocol-Guided Rapid Evaluation of Veterans Experiencing New Transient Neurological Symptoms (PREVENT) Program for Improving Quality of Care for Transient Ischemic Attack: A Nonrandomized Cluster Trial. Published online 2020.

Lowery J, Fagerlin A, Larkin AR, Wiener RS, Skurla SE, Caverly TJ. Implementation of a Web-Based Tool for Shared Decision-making in Lung Cancer Screening: Mixed Methods Quality Improvement Evaluation. JMIR Hum Factors. 2022;9(2):e32399. doi:10.2196/32399

Ornstein S, Nemeth LS, Jenkins RG, Nietert PJ. Colorectal Cancer Screening in Primary Care Translating Research Into Practice. Med Care. 2010;48(10). doi:10.1097/MLR.0b013e3181ec5591

Quanbeck A, Brown RT, Zgierska AE, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for Opioid prescribing in primary care. Implement Sci. 2018;13(1):21. doi:10.1186/s13012-018-0713-1

Noël PH, Barnard JM, Leng M, et al. The Coordination Toolkit and Coaching Project: Cluster-Randomized Quality Improvement Initiative to Improve Patient Experience of Care Coordination. J Gen Intern Med. 2022;37(1):95-103. doi:10.1007/s11606-021-06926-y

Williams L, Daggett V, Slaven JE, et al. A cluster-randomised quality improvement study to improve two inpatient stroke quality indicators. BMJ Qual Saf. 2016;25(4). doi:10.1136/bmjqs-2015-004188

Bauer MS, Miller CJ, Kim B, et al. Effectiveness of Implementing a Collaborative Chronic Care Model for Clinician Teams on Patient Outcomes and Health Status in Mental Health: A Randomized Clinical Trial. JAMA Netw Open. 2019;2(3):e190230. doi:10.1001/jamanetworkopen.2019.0230

Chokron Garneau H, Assefa MT, Jo B, Ford JH, Saldana L, McGovern MP. Sustainment of Integrated Care in Addiction Treatment Settings: Primary Outcomes From a Cluster-Randomized Controlled Trial. Psychiatr Serv. 2022;73(3):280-286. doi:10.1176/appi.ps.202000293

Williams, N.J., et al., Mechanisms of Change in the ARC Organizational Strategy: Increasing Mental Health Clinicians' EBP Adoption Through Improved Organizational Culture and Capacity. Adm Policy Ment Health, 2017. 44(2): p. 269-283.

Acosta J, Chinman M, Ebener PA, Malone PS, Cannon JS, D’Amico EJ. Sustaining an Evidence-Based Program Over Time: Moderators of Sustainability and the Role of the Getting to Outcomes® Implementation Support Intervention. Prev Sci. 2020;21(6):807-819. doi:10.1007/s11121-020-01118-2

Chinman M, Acosta J, Ebener P, Malone PS, Slaughter ME. Can implementation support help community-based settings better deliver evidence-based sexual health promotion programs? A randomized trial of Getting To Outcomes®. Implement Sci IS. 2016;11(1):78. doi:10.1186/s13012-016-0446-y

Fagan AA, Hanson K, Briney JS, Hawkins JD. Sustaining the Utilization and High Quality Implementation of Tested and Effective Prevention Programs Using the Communities That Care Prevention System. Am J Community Psychol. 2012;49(4-Mar). doi:10.1007/s10464-011-9463-9

Assess and Redesign Workflows


Strategy Definition

Observing and mapping current work processes, planning for desired work processes, and identifying necessary changes to accommodate, encourage, or incentivize use of the clinical innovation as designed. (Source)

This strategy may comprise considerations of perceived process (what implementers think is happening), reality process (what the process actually is), and ideal process (what the process could be).

Strength of Evidence Supporting Its Effectiveness: Moderate Supportive

Based on direct and indirect evidence from included studies and contextual systematic review evidence identified outside of this review, we judged evidence to be either supportive, moderate supportive, limited supportive, indirect supportive only, or inconclusive. We determined the strength of evidence supporting the contribution of this strategy to successful implementations was moderate supportive.

Health Care Areas Where the Assess and Redesign Workflows Strategy Was Used

Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.

AreaPercentage of studiesNumber of studies
Primary care70%7
Specialty care10%1
Mental health20%2
Public Health00
Multiple Settings00
Other00

Results From Studies That Included Assess and Redesign Workflows Strategy

We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.

How Often Were Studies That Included the "Assess and Redesign Workflows" Strategy Effective?

OutcomePercentage of studiesNumber of studies
Conducting the EBP at all83%5/6
Performing the EBP well (ie, with fidelity)100%6/6
Sustaining the EBP100%4/4
Achieving health outcomes50%3/6

The tables below show the recorded outcomes of each study that used this strategy by setting.

Studies That Used Assess and Redesign Workflows and Their Outcomes, by Setting

Primary care

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Dickinson 2019)Positive
finding
Not testedNot testedNull finding
(Bravata 2020)Not testedNot testedNot testedPositive
finding
(Carter 2018)Positive
finding
Positive
finding
Not testedPositive
finding
(Ornstein 2010)Positive
finding
Not testedNot testedNot tested
(Quanbeck 2018)Positive
finding
Positive
finding
Positive
finding
Not tested
(Parchman 2013)Not testedPositive
finding
Positive
finding
Not tested
(Estrada 2011)Null findingPositive
finding
Not testedNull finding

Specialty care

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Hahn 2022)Not testedNot testedPositive
finding
Not tested

Mental health

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Bauer 2019)Not testedPositive
finding
Not testedPositive
finding
(Chokron Garneau 2022)Positive
finding
Positive
finding
Positive
finding
Null finding

References

Dickinson WP, Dickinson LM, Jortberg BT, et al. A Cluster Randomized Trial Comparing Strategies for Translating Self-Management Support into Primary Care Practices. J Am Board Fam Med JABFM. 2019;32(3):341-352. doi:10.3122/jabfm.2019.03.180254

Bravata D. Assessment of the Protocol-Guided Rapid Evaluation of Veterans Experiencing New Transient Neurological Symptoms (PREVENT) Program for Improving Quality of Care for Transient Ischemic Attack: A Nonrandomized Cluster Trial. Published online 2020.

Carter BL, Levy B, Gryzlak B, et al. Cluster-Randomized Trial to Evaluate a Centralized Clinical Pharmacy Service in Private Family Medicine Offices. Circ-Cardiovasc Qual Outcomes. 2018;11(6). doi:10.1161/CIRCOUTCOMES.117.004188

Ornstein S, Nemeth LS, Jenkins RG, Nietert PJ. Colorectal Cancer Screening in Primary Care Translating Research Into Practice. Med Care. 2010;48(10). doi:10.1097/MLR.0b013e3181ec5591

Quanbeck A, Brown RT, Zgierska AE, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for Opioid prescribing in primary care. Implement Sci. 2018;13(1):21. doi:10.1186/s13012-018-0713-1

Parchman ML, Noel PH, Culler SD, et al. A randomized trial of practice facilitation to improve the delivery of chronic illness care in primary care: initial and sustained effects. Implement Sci IS. 2013;8:93. doi:10.1186/1748-5908-8-93

Estrada CA, Safford MM, Salanitro AH, et al. A web-based diabetes intervention for physician: a cluster-randomized effectiveness trial. Int J Qual Health Care. 2011;23(6):682-689. doi:10.1093/intqhc/mzr053

Hahn EE, Munoz-Plaza CE, Pounds D, et al. Effect of a Community-Based Medical Oncology Depression Screening Program on Behavioral Health Referrals Among Patients With Breast Cancer: A Randomized Clinical Trial. JAMA. 2022;327(1):41-49. doi:10.1001/jama.2021.22596

Bauer MS, Miller CJ, Kim B, et al. Effectiveness of Implementing a Collaborative Chronic Care Model for Clinician Teams on Patient Outcomes and Health Status in Mental Health: A Randomized Clinical Trial. JAMA Netw Open. 2019;2(3):e190230. doi:10.1001/jamanetworkopen.2019.0230

Chokron Garneau H, Assefa MT, Jo B, Ford JH, Saldana L, McGovern MP. Sustainment of Integrated Care in Addiction Treatment Settings: Primary Outcomes From a Cluster-Randomized Controlled Trial. Psychiatr Serv. 2022;73(3):280-286. doi:10.1176/appi.ps.202000293

Obtain Formal Commitments from Leadership to Implement Change


Strategy Definition

Obtaining written commitments from key leaders that outline their roles in implementing the effective practice. (Source)

This strategy should clarify roles, responsibilities, and communicate tangible and non-tangible benefits (e.g., community partnerships). These roles should also be supported within the organization (e.g., workload release credit for providing and receiving supervision in a new clinical practice).

Strength of Evidence Supporting Its Effectiveness: Not Evaluated

For some of the most common strategies we judged the strength of research support based on included studies and contextual systematic review evidence identified outside of this review. We did not evaluate the strength of evidence for this strategy. The information below represents the data we collected from studies including this strategy.

Health Care Areas Where the Obtain Formal Commitments from Leadership to Implement Change Strategy Was Used

Strategies were used in a variety of settings and areas of health care. The “Other” category includes non-traditional settings in which health care might be provided, like schools.

AreaPercentage of studiesNumber of studies
Primary care00
Specialty care00
Mental health00
Public Health100%1
Multiple Settings00
Other00

Results From Studies That Included Obtain Formal Commitments from Leadership to Implement Change Strategy

We recorded results from four different types of outcomes across research studies that included this strategy. The table below shows the proportion of these studies that showed improvement compared to a control.

How Often Were Studies That Included the "Obtain Formal Commitments from Leadership to Implement Change" Strategy Effective?

OutcomePercentage of studiesNumber of studies
Conducting the EBP at all0%0/1
Performing the EBP well (i.e., with fidelity)0%0/0
Sustaining the EBP0%0/1
Achieving health outcomes0%0/0

The tables below show the recorded outcomes of each study that used this strategy by setting.

Studies That Used Obtain Formal Commitments from Leadership to Implement Change and Their Outcomes, by Setting

Public Health

StudyConduct EBPPerform EBP wellSustain EBPHealth outcomes
(Knott 2022)Null findingNot testedNull findingNot tested

References

Knott CL, Chen C, Bowie JV, et al. Cluster-randomized trial comparing organizationally tailored versus standard approach for integrating an evidence-based cancer control intervention into African American churches. Transl Behav Med. 2022;12(5):673-682. doi:10.1093/tbm/ibab088

Stay Connected

Get weekly updates about opportunities to apply for funding, newly funded research and engagement projects, results of PCORI-funded research, webinars and other news.