In February 2015, following a public comment period, our Board of Governors adopted PCORI's Process for Peer Review of Primary Research and Public Release of Research Findings. By law, all PCORI-funded primary research (i.e., research asking which of two or more approaches is better) must undergo peer review, and all results must be made available to the public.
The goal of peer review is to ensure that the primary research studies funded by PCORI are held to the highest standards of scientific integrity, methodological rigor, and relevance and usefulness to patients, caregivers, clinicians, and other healthcare stakeholders. Our process includes review of study findings by content experts, methodologists, patients, and other healthcare stakeholders with experience related to the study.
How do we assess scientific integrity? Scientific integrity refers to the transparency and completeness of presentations of the scientific work. In writing manuscripts, some authors communicate only the findings that present their research in the most positive light. However, consumers of research need to know about all findings and also any tests that turned out to not give significant results. We ask research teams to submit to PCORI final research reports that provide a full report on the research.
How do we assess methodological rigor? PCORI has a comprehensive list of methodological standards that were developed by our Methodology Committee in 2013 and revised in 2016. These include both broad standards—those that should be followed by all PCORI-funded research—and specific standards, which apply only to specific types of studies. Within our peer review process, we will evaluate how well the investigators met the Methodology Standards.
How do we assess relevance and usefulness? Scientific integrity and methodological rigor are specified as goals in our authorizing legislation, but our Board added a third goal when it laid out the peer review process in 2015. The Board wanted to ensure that the reports from the studies that we fund address real issues and situations faced by patients, caregivers, clinicians, and others, and provide information that individuals and families can use to make better healthcare decisions. In our peer review process, we ask patients and other stakeholders serving as reviewers whether the research findings are relevant to the experiences of patients, clinicians, and others, and potentially useful in helping them make better decisions.
Who Are Our Peer Reviewers?
PCORI peer reviewers come from a variety of communities under the healthcare umbrella. Reviewers include
- Scientists and other subject-matter specialists
- Biostatisticians and other methodologists who can assess the quality of the study methods
- Clinicians and other healthcare stakeholders, such as members of industry and policy makers
- Patients, caregivers, or others with a personal or professional stake in the research topic
Peer review has an important role in ensuring the quality and usefulness of our research findings. If the research PCORI funds is to be valuable to the diverse communities within the healthcare system, we need a wide variety of individuals to provide our peer review process with the knowledge and perspectives they have acquired through their own experiences.
How Does Peer Review Work?
PCORI’s Process for Peer Review provides a roadmap for PCORI that reflects the requirements of our authorizing law. This process is part of PCORI's broad dissemination and implementation plan to put knowledge gained from high-quality patient-centered outcomes research (PCOR) into practice. Input from patients and the rest of the healthcare community was vital to developing this process.
Our roadmap begins with the principal investigator (PI) submitting a draft final research report (Summary of Changes to the Draft Final Research Report) that includes a comprehensive description of the project’s methods and the results from analyses of the main study aims. At PCORI, the project’s program officer makes sure that the draft report includes all of the necessary components.
The PI submits the draft report to our electronic peer review system, with which we follow a process similar to that of a peer-reviewed journal. An associate editor assigns reviewers—from the appropriate scientific specialties and healthcare communities—and later assembles their key points into a letter to the PI, with input from program staff. The PI can respond to the points raised and make changes in the draft report.
The associate editor and the project’s program officer then either recommend that PCORI accept the report as final or ask the PI for more information. On rare occasions, if major issues were not addressed in the revision, there may be a second round of peer review.
Once the final research report has been accepted, PCORI’s Patient-Centered Outcomes Research Translation Center finalizes lay-language and professional summaries for the website.
Draft Final Research Report - Tips for Awardees
Please remember that the draft final research report should be a detailed report of all work completed as part of the PCORI funded contract. The report should:
- Include a detailed discussion of the study’s adherence to PCORI methodology standards
- Describe in detail how the researchers engaged patients and stakeholders in the development, execution, and interpretation of the study
- Incorporate all study aims, methods, and results that were planned and executed
- Include the statistical analyses and results, even if inconclusive
- Step-by-Step Instructions for Awardees: Peer Review of Draft Final Research Report
- How to Submit the Draft Final Research Report
- Ancillary Information Conflicts of Interest Disclosure Form Relating to PCORI-Funded Research Project
Posted: October 10, 2016; Updated: December 8, 2016