What is the PCORI Methodology Report?
The PCORI Methodology Report provides baseline requirements for the development and conduct of patient-centered comparative effectiveness research. The report addresses two legislative mandates: the development of methodology standards and development of a model for creating translation tables. The report details the context and rationale for the PCORI Methodology Standards.
What are the PCORI Methodology Standards?
The Methodology Standards apply to patient-centered outcomes research (PCOR), which is comparative clinical effectiveness research that seeks the perspectives and input of patients, clinicians, and other healthcare stakeholders throughout the research process. The Methodology Standards specify the minimal requirements for best practices in the design and conduct of PCOR. They were formally adopted by PCORI’s Board of Governors in December 2012 following revisions based on a required public comment period and have since been incorporated into all PCORI Funding Announcements. As of PCORI’s August 2013 funding cycle, all applicants must show that they follow the standards.
The 47 standards fall broadly into two groups:
Cross-Cutting Standards for PCOR
- Formulating research questions – six standards that specify what to include in research protocols as a means of increasing study quality as well as transparency in research
- Patient-centeredness – four standards that promote effective patient engagement and the explicit incorporation of patient needs, values, and preferences into research
- Data integrity and rigorous analyses – six standards that describe necessary documentation of key decisions and tests of the assumptions made in analyses
- Preventing and handling missing data – five standards outlining proper statistical methods for handling missing data
- Heterogeneity of treatment effects (HTE) – four standards on how to account for the fact that different people do not always respond the same way to the same treatment
Standards for Specific Study Designs and Methods
- Data registries – three standards to help ensure that registries contain relevant, high-quality data that are used appropriately
- Data networks as research-facilitating infrastructures – two standards to help ensure that key components are included in network design and considered when network data are used in studies
- Causal inference methods – six standards on accounting for possible sources of bias and addressing them to produce valid conclusions about the causal effect of an intervention
- Adaptive and Bayesian trial designs – five standards providing guidance on the design and conduct of studies that use such designs
- Studies of diagnostic tests – five standards that address studying the impact of diagnostic tests on subsequent care and patient outcomes
- Systematic reviews – one standard that outlines the application of standards for systematic reviews
View the full text of the Methodology Standards (Appendix A of the PCORI Methodology Report)
Why is the report needed?
PCOR provides evidence to help people make informed healthcare decisions and allows their voices to be heard in the research that assesses the effectiveness of the healthcare options available to them. This patient-centered approach aims to fill gaps in knowledge that matter most to those making clinical decisions and to achieve faster adoption of new findings in clinical practice.
PCOR is a relatively new approach to conducting comparative effectiveness research. All research programs need to adhere to best practices in the planning, design, and conduct of every research project for the findings to be considered reliable and worth adoption. In short, methods matter. The PCORI Methodology Report provides baseline requirements and a framework for those best practices. The report includes vignettes that illustrate different ways that good methodology makes a difference to patients and their care. These include stories of patients’ experiences navigating choices and weighing options, and examples of published studies that capture the impact of good methodology.
How were the report and standards developed?
The PCORI Methodology Report is a substantially revised version of a draft report authored by the PCORI Methodology Committee. That draft, including an initial set of standards, was issued in July 2012; a public comment period followed from July through September 2012. Comments received during that period addressed both the standards specifically and the report more broadly. The standards were revised in response to those comments and adopted by PCORI’s Board in December 2012. Because there were so many detailed comments about the draft report, those revisions took longer.
To develop the PCORI Methodology Standards, the committee assessed the range of potential standards, narrowed its scope to those it deemed most important, solicited feedback through a public comment period, revised the draft standards, and confirmed the final set of standards through consensus of its members. In updating the draft report, the committee returned to the public comments, analyzed them to assess specific and thematic suggestions and concerns, and undertook substantial revisions as a result. Appendix B of the report details the committee’s response to the feedback received, related to both the Methodology Standards and the report.
What’s next for the report and standards?
PCORI will pursue a comprehensive, coordinated approach to promote awareness and use of the Methodology Standards as a resource and guide for those who develop, fund, and conduct research. This effort will include broad outreach to healthcare organizations and collaborative initiatives focused on strengthening research practices and developing training resources and other tools to support integration of the standards in practice. The release of the report is part of an ongoing process of refinement and improvement.
The Methodology Committee expects the scope of the standards to be broadened and revised further over time to ensure that they continue to address evolving PCOR approaches. PCORI will continue to identify relevant research methods for which standards are needed to improve the field of clinical comparative effectiveness research.
Posted November 19, 2013