Download the Principles

In March 2021, PCORI's Board of Governors approved a set of four Principles for the Consideration of the Full Range of Outcomes Data in PCORI-Funded Research, which will guide how we implement our expanded authority to consider the potential burdens and economic impacts of healthcare services in the research we fund.

This authority, part of PCORI's 2019 reauthorizing law, clarifies that PCORI-funded studies should capture "the full range of clinical and patient-centered outcomes," specifically adding "the potential burdens and economic impacts" of healthcare services to the patient-centered outcomes and clinical effectiveness measures that have been the hallmarks of the research we have funded to date. Learn more about the Principles and our implementation plans in this blog post.

These are the themes of the four Principles that will guide us in our research funding application and review process, and the work we do more broadly. Click them to learn more.

• Identifying Outcomes Important to Patients and Caregivers
• Identifying Outcomes Important to Stakeholders
• Collection of Data as Appropriate and Relevant
• Conduct of Certain Types of Economic Analysis

How We Developed the Principles

Public Comment

In September 2020, PCORI’s Board approved the release of a set of proposed Principles for the Consideration of the Full Range of Outcomes Data for a 60-day public comment period between September and November 2020. We heard from dozens of individuals and organizations representing the voices of patients, caregivers, clinicians, insurers, researchers, purchasers, representatives of the life sciences industry, and others. A report on the public comments will be available here soon.

View a report on the public comments and click here to read a summary of the comments we received and PCORI's responses.

Webinars

PCORI also hosted two public webinars and individual stakeholder discussions, which informed the development of the Principles and highlighted opportunities where we can further advance the implementation of this new legislative authority and support the needs of patients, caregivers, and other stakeholders.

• Considering the Full Range of Outcomes in PCORI Research – Patients, Caregivers and Consumers
  October 5, 2020
• Considering the Full Range of Outcomes in PCORI Research – Payers, Purchasers, Providers, Health Systems and Pharmaceutical Industry
  October 6, 2020

Principles

These are the key Principles PCORI will follow in developing guidance to applicants and to merit reviewers for use when assessing applications for funding.

Principle #1: Considering the full range of outcomes important to patients and caregivers, including potential burdens and economic impacts, is central to PCORI-funded research.

Patient and caregiver priorities have always guided PCORI’s research agenda. Because burdens and economic impacts of care options significantly affect patients and their caregivers, PCORI seeks to incorporate those factors within PCORI-funded research. The goal of doing so is to provide information on the costs and impacts of treatments and services to patients and caregivers. A better understanding of the potential burdens and economic impacts of treatment options for patients and caregivers will inform not only their decisions but also those made by payers, health systems, and providers.

PCORI requires that patients be engaged in the research it funds—not as participants but as partners who help determine what to study and how. This tenet of PCORI-funded research is especially important when considering potential burdens and economic impacts important to patients and their caregivers. Patient engagement is critical not only in identifying important outcomes but also in understanding how to capture the relevant costs and burdens, as they may vary depending on many factors, such as insurance coverage. PCORI will continue to seek and support the active engagement of patients and caregivers in PCORI-funded research, including when identifying the burden and economic impact outcomes relevant to them that may not be traditionally or systematically measured.

Back to top

Principle #2: PCORI-funded research may consider the full range of outcomes relevant to other stakeholders, when these outcomes have a near-term or longer-term impact on patients.

Stakeholders such as payers, employers, health systems, clinicians, and other decision makers responsible for designing health plans, formulary decisions, or health system improvements make decisions that have near- and long-term impacts on patients. They also benefit from a robust evidence base on the clinical and economic impacts of healthcare treatments and services to inform their decisions. Therefore, under the new authority, PCORI is charged with, as appropriate, considering the potential burdens and economic impacts of healthcare utilization on these “different stakeholders and decision-makers.”

Burdens and impacts that may inform stakeholders’ decisions include, for example, costs associated with treatments or other interventions, or with healthcare utilization resulting from these treatments or interventions. Components of such costs may include a range of relevant resource use, such as staff time (wages and benefits), costs of diagnostic tests, and medical supplies. In many cases, the costs of an intervention will involve “program costs,” such as training and supervising staff or purchasing needed equipment.

To ensure that the data being collected are appropriate and relevant, applicants should address why the economic outcomes—such as impacts on productivity or costs of implementing a specific program—are important for informing stakeholder choices that will affect patient outcomes.

One clear example of when collection of economic data may be important is in studies where two options are equivalent with respect to clinical and patient-centered outcomes. In these circumstances, burdens or economic impacts may provide important additional information for decision making. For example, if a study establishes that health outcomes for patients receiving in-person specialty care are not better than those receiving these services through telemedicine, decisions about establishing these telemedicine programs may well turn on other factors. If the telemedicine program costs less to deliver, or if it provides substantial benefits to patients in terms of time saved in travel to in-person visits, then this information may prove valuable to decision makers at all levels—from payers to health systems administrators to patients.

The authorizing law, as amended, directs PCORI-funded research to capture, as appropriate, burden and economic impact data. Following the example above, PCORI-funded research can provide that data as a contribution to the evidence stakeholders have available to conduct analyses to inform their decisions. Different stakeholders can use different types of analyses, such as return on investment analyses, to inform their decisions, and they can draw upon information regarding patient burden and economic impact collected by PCORI-funded research in a variety of ways.

Back to top

Principle #3: The collection of data on potential burdens and economic impacts of intervention options must be appropriate and relevant to the clinical aims of the study.

PCORI appreciates the need for information on the full range of outcomes, including on the burdens and economic impacts of various intervention options and services, when making healthcare decisions, and PCORI encourages the collection of these data in PCORI-funded research. PCORI’s foremost mandate, as dictated in our authorizing law, is to support the conduct of comparative CER. In recognition of this priority, PCORI does not require the collection of data on burdens and economic impacts of intervention options across all PCORI-funded research.

PCORI encourages investigators to capture appropriate and relevant cost burdens and economic impacts associated with the impact of an intervention (Example 1) for two or more alternative approaches that are studied within the context of CER. Additionally, when capturing the costs of the intervention contributes valuable and relevant information, investigators should capture this information. Investigators may also capture the costs associated with implementing an intervention when these reflect real-world costs of relevance to potential adopters (Example 2).

PCORI does not fund studies for which cost and economic impacts are the primary outcome. Further, PCORI does not support studies that conduct cost effectiveness analysis.

However, as indicated in Principle #1, PCORI has a special interest in patient and/or family burden. For this reason, PCORI welcomes and will support studies that directly compare patient and/or family cost burden (e.g., out-of-pocket costs, time costs associated with an approach, financial toxicity) as part of secondary outcomes.

Addressing health disparities and health equity continues to be a priority in PCORI-funded research. Cost burdens, both direct and indirect, associated with other social risk factors and social determinants of health have a direct impact on health disparities and inequities within our healthcare system. PCORI will continue to support patient-centered outcomes research that addresses and eliminates these challenges. Therefore, when considering relevant burdens and economic impacts, PCORI-funded research should consider the inclusion of outcomes that capture social risk factors and social determinants of health.

Before developing guidance and potential methodology standards, PCORI will seek opportunities to engage with patients, caregivers, and stakeholders across the healthcare community to further define this priority.

In all research that it funds, PCORI expects that study outcomes should be relevant and important to patients and other stakeholders. To meet this expectation, PCORI requires applicants to engage relevant stakeholders in the formulation of the research question and the development of the study design, as well as the identification of outcomes to measure. This approach is intended to ensure that PCORI-funded research will provide evidence that is ultimately relevant and applicable for the end user; it also seeks to avoid the unnecessary capture of data that are not relevant to the aims of the study and may not be beneficial to the goals of the research. This same expectation should apply when considering whether and which potential burden and other economic impact data a research study should capture.

When considering data collection on burdens and economic impact, applicants should consider the feasibility of capturing these data types when submitting funding applications. This includes considering the availability and accessibility of relevant data, the added burdens that capturing these data may impose on study participants, ensuring that data are captured from reliable sources, data generalizability to real-world practice, and the potential limitations of the data.

When reviewing an application’s proposed collection of economic impact and cost data, PCORI will consider whether the applicant is leveraging the unique opportunities afforded by their comparative CER to collect this information; that is, the collection of economic impact and cost data should directly follow from and relate to the specific research study.

Back to top

Principle #4: PCORI may support the conduct of certain types of economic analyses, as part of a funded research study or independently, to enhance the relevance and value to patients and other stakeholders of information PCORI-funded investigators collect on potential burdens and economic impacts burdens.

The language of the authorizing law, as amended, charges PCORI with consideration of the full range of “outcomes data”—that is, the consideration of data relevant to potential burdens and economic impacts of healthcare interventions, as appropriate. As such, and described in the Principles above, PCORI-funded CER can include capturing data on these impacts.

PCORI understands that to facilitate the consideration of the full range of data, investigators may propose to extend their activities beyond the simple collection of existing data to the measurement of cost elements and/or to the conduct of certain types of economic analyses. For example, a study might collect data on a variety of different cost components that factor into the delivery of an emergency department (ED) visit for a patient with a suspected cardiac event: costs of staff time for clinicians and support staff, costs for use of the ED bed (e.g., linens, cleaning), oxygen, cardiac monitoring, and so on. However, the collection of these data may be more relevant—and rendered more useful—if they are summarized in a total estimated cost per visit. Developing this estimate might require analysis, for example, to reflect median use of supplies and staff time. This type of analysis is consistent with the intention of PCORI’s authorizing law. As relevant, PCORI applicants should ensure that their proposed capture of economic impact includes such analyses and that their report of study findings includes a presentation of the results.

There are, however, statutory limits on the types of analyses that PCORI can fund. In accordance with its authorizing law, as amended, PCORI is prohibited from funding research that includes practice guidelines or coverage, payment, or policy recommendations and developing or employing a dollars-per-quality adjusted life year threshold. Thus, investigators seeking PCORI funding should not propose aggregating findings on health outcomes with findings on economic impacts—as in a cost-effectiveness ratio.

PCORI understands the utility of certain types of analyses that may fall beyond the scope of its mission and the parameters of its authorizing law. PCORI’s limitations on this type of analysis extend only to studies conducted with PCORI funding. Investigators may choose to seek additional funding from other funders to support further analysis. Similarly, the prohibitions against the development of practice guidelines, coverage, payment, or policy recommendations in PCORI-funded research findings apply only to work funded by PCORI. PCORI-funded research may be used by others to inform such products.

Back to top

Download the Principles

Posted: March 18, 2021; Updated: March 25, 2021