What are the PCORI Methodology Standards?
The PCORI Methodology Standards provide baseline requirements for the development and conduct of patient-centered outcomes research (PCOR), which is comparative clinical effectiveness research that seeks the perspectives and input of patients, clinicians, and other healthcare stakeholders throughout the research process. The Methodology Standards specify the minimal requirements for sound science. Applicants for PCORI research funding and awardees are required to demonstrate adherence to the Methodology Standards in the design, conduct, and reporting of their research.
PCORI’s Board of Governors adopted the original PCORI Methodology Standards in December 2012. Updated Methodology Standards were formally adopted by the Board in May 2017, April 2018, and February 2019.
What topics do the PCORI Methodology Standards address?
The 65 standards fall broadly into two groups:
Cross-Cutting Standards for PCOR
- Formulating research questions—six standards that specify requirements for ensuring that studies are designed to generate evidence needed to support informed health decisions
- Patient-centeredness—four standards that promote effective engagement of patients and other stakeholders to improve the quality and relevance of PCOR findings
- Data integrity and rigorous analyses—seven standards that describe necessary documentation of key decisions regarding collection, management, and analysis of data
- Preventing and handling missing data—four standards that specify how researchers should prevent, document, and statistically account for missing data
- Heterogeneity of treatment effects—three standards that describe how to account for differences in the way different people may respond to the same treatment
Standards for Specific Study Designs and Methods
- Data registries—four standards to help ensure that registries contain relevant, high-quality data that are used appropriately
- Data networks as research-facilitating structures—two standards to help ensure that key components are included in network design and considered when network data are used in studies
- Causal inference methods—six standards that specify requirements for identifying and addressing possible sources of bias to produce valid conclusions about the benefits and risks of an intervention (the first of these is considered a cross-cutting standard)
- Adaptive and Bayesian trial designs—four standards providing guidance on the design, conduct, and analysis of such PCOR studies
- Studies of medical tests—three standards that address studying the impact of medical tests on subsequent care and patient outcomes
- Systematic reviews—one standard that outlines the application of standards for systematic reviews
- Research designs using clusters—five standards that specify requirements for the design and analysis of studies in which groups (rather than individuals) are randomized to the interventions (This set was added in the 2017 update.)
- Studies of complex interventions – five standards that specify requirements for studies examining healthcare interventions that involve multiple components (This set was added in the 2018 update.)
- Qualitative methods – four standards to ensure the robust capture and understanding of information from patients, caregivers, clinicians, and other stakeholders in research
- Mixed methods research – three standards that specify requirements for the integration of methods, data, findings, and interpretations from studies using this approach
- Individual participant-level data meta-analysis (IPD-MA) – four standards that address the appropriate design, conduct, analysis, and reporting of IPD-MA
What is the PCORI Methodology Report?
The PCORI Methodology Report details the context and rationale for the PCORI Methodology Standards. The report addresses two legislative mandates: the development of methodology standards and a translation framework to guide the choice of study designs for specific research questions. The Methodology Report was most recently updated in 2019, following a 2018 update to the Methodology Standards. An updated version of the report is expected later in 2019 to include the standards for qualitative methods, mixed methods research, and IPD-MA adopted by the Board in February 2019.
To demonstrate the importance of appropriate approaches to research, the Methodology Report presents vignettes that illustrate different ways that good methodology makes a difference to patients and their care. These include examples of published studies and stories of patients’ experiences navigating choices and weighing options.
The PCORI Methodology Standards appear as Appendix A of the Methodology Report.
Why are the Methodology Standards and Methodology Report needed?
Patient-centered outcomes research (PCOR) provides evidence to help people make informed healthcare decisions and allows their voices to be heard in the research that assesses the effectiveness of the healthcare options available to them. This patient-centered approach aims to fill the gaps in knowledge that matter most to those making clinical decisions and to achieve faster adoption of new findings in clinical practice.
PCOR is a relatively new approach to conducting comparative clinical effectiveness research. For the findings to be considered reliable and useful for informing decisions, all research programs need to adhere to best practices in the planning, design, and conduct of research projects. In short, methods matter.
The PCORI Methodology Standards and Methodology Report provide baseline requirements and a framework for those best practices.
How were the Methodology Standards and Methodology Report developed?
To develop the original PCORI Methodology Standards, the PCORI Methodology Committee assessed the range of potential standards and narrowed its scope to those it deemed most important. The committee then authored a draft of the Methodology Report, including an initial set of standards, that was issued in July 2012. PCORI solicited feedback through a public comment period that lasted through September 2012. Comments received addressed both the standards specifically and the report more broadly.
The committee analyzed the public comments to assess specific and thematic suggestions and concerns, and then undertook substantial revisions. An appendix of the report details the committee’s response to the feedback received. The committee confirmed the revised Methodology Standards through consensus of its members, and PCORI’s Board adopted them in December 2012. The Board accepted the Methodology Report, which took longer to revise, the next year.
In more recently updating the standards and report, the Methodology Committee followed a systematic process of review and revision of the original standards. The Committee solicited public comment on the proposed changes and revised accordingly. The Board adopted updated Methodology Standards in May 2017, April 2018, and February 2019. Updated versions of the Methodology Report were released in July 2017 and January 2019. An additional update to the Methodology Report is expected later in 2019.
What’s next?
PCORI will continue to pursue a comprehensive, coordinated approach to promote awareness and use of the Methodology Standards and Methodology Report as a resource and guide for those who develop, fund, and conduct research. This effort will include broad outreach to healthcare organizations and collaborative initiatives focused on strengthening research practices and developing training resources and other tools to support integration of the standards in practice.
PCORI has a commitment to evaluate and update the guidance that it provides to the research community. Future editions of the Methodology Report and Methodology Standards will provide updated methodological guidance for PCOR, supporting the generation of high-quality and relevant evidence that patients, clinicians, and other stakeholders need to make informed health decisions.
Posted: November 19, 2013; Updated: February 26, 2019