PCORI expects all of its funded patient-centered comparative clinical effectiveness research (CER) projects to produce new evidence that fills evidence gaps related to real-world healthcare choices or questions from patients, caregivers, and other stakeholders. This focus on real-world, stakeholder-driven questions necessitates that PCORI-funded researchers consider how the research can be conducted in typical clinical care settings. This planning often entails the incorporation of pragmatic elements of clinical practice into the study design, regardless of whether the study involves a Broad, Targeted, or Pragmatic Clinical Studies PCORI Funding Announcement or award. The purpose of this guidance document is to advise current and prospective PCORI investigators and awardees on approaches for study design and operations while conducting research under real-world clinical care conditions, with the goal to produce high quality and representative study findings that can be readily applied to health care.

PCORI’s patient-centered outcomes research (PCOR) is stakeholder-driven comparative CER that compares two or more alternatives for specific clinical services, the organization of healthcare delivery systems, or communication about healthcare decisions. Pragmatic or real-world research designs are encouraged by PCORI to allow the findings to be generalized and readily adopted into the current clinical care landscape. Pragmatic trials stand in contrast to traditional explanatory clinical trials done in highly-selective patients and tightly- controlled research environments which often do not reflect the complexities of real world patient populations and clinical practices.

PCORI expects PCORI-funded clinical trials using pragmatic designs to adhere to the PCORI Methodology Standards, which lay out minimum requirements for patient-centered outcomes research. Many investigators also frequently refer to a series of publications about the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) in designing their studies [¹, ²]. The PRECIS and PCORI approaches to pragmatic studies have much in common, including their encouragement of diverse populations, typical care settings, and minimally intrusive study conduct. Both recommend designing research studies to be fit for purpose in answering stakeholder questions, but they emphasize different comparisons and decisions which result in different recommendations for handling usual care comparators, fidelity of study interventions, and participant adherence.

To date, PRECIS and PRECIS-2 have been oriented primarily to stakeholder questions of whether to introduce or provide coverage of new, efficacious interventions within existing clinical practice(s) or usual care. The principal focus is upon a single intervention compared to a range of established interventions already in use or covered by policy or payment. PRECIS and PRECIS-2 refer to this range of interventions as usual care. The guidance in these documents gives limited attention to the direct comparison of two or more distinct interventions, which is the focus of PCOR. With PCOR, there are stringent considerations when employing “usual care” as a distinct comparator because usual care is often ill-defined, variable, and poorly quantified. These qualities adversely affect meaningful and robust comparisons. PCORI explicitly discourages usual care comparators in PCORI-funded research projects unless they are well justified, detailed, measurable, and serve as a coherent and distinct clinical alternative, such as care that is tightly aligned with clinical practice guidelines.

To allow distinction of small but meaningful differences between active interventions, PCORI recommends that treatment fidelity and participant adherence be monitored judiciously. Careful judgement is required because differential fidelity in conduct of the intervention by providers or adherence by patients may create or obscure outcome differences. Since the goal of CER is to improve the evidence base for guiding real world clinical practice, the approach to controlling or monitoring the intervention’s use by providers and patients should be as unobtrusive as possible and reflect the conditions of anticipated future use. The degree of monitoring will ideally match the conditions of where and how the research will ultimately be applied. While PCORI agrees with the intent of PRECIS and PRECIS-2 to minimize distortions of outcomes or interventions due to study processes themselves, the PCORI Methodology Standards specify a more definitive process for managing variability in the clinical interventions. To be replicated elsewhere, real-world research and its conduct must try to emulate real- life variability in intervention practices and behaviors in a scientifically rigorous way. PCORI’s Methodology Standards for Studies of Complex Interventions establish the necessary framework and approach to identify the core functions, forms, and causal pathways of interventions and comparators and specify how adaptations to their form will be allowed and recorded.

To achieve generalizable and externally valid findings, PCORI-funded clinical trials employing pragmatic design features should loosen the typical, tight study controls that have been used in past clinical research. To preserve internal validity in CER, the fidelity, adherence, and monitoring of the interventions being compared should be explicitly addressed and their allowed implementation, variation and adaptation defined. In contrast, fewer constraints may be needed in the study population, settings, and manner of data collection. By expanding eligibility criteria to encompass more diverse patients, carrying out the trial in non-research settings, and using routinely collected data to minimize study burdens on clinical care delivery, pragmatic approaches can increase generalizability, minimize participant burden, and improve study efficiency. However, pragmatic clinical trials in

PCOR should never be construed to be laissez-faire or anything goes. On the contrary: pragmatic trials are not simple or easy. They require extensive and purposeful planning to optimize internal and external validity.