In February 2015, following a public comment period, our Board of Governors adopted PCORI's Process for Peer Review of Primary Research and Public Release of Research Findings. By law, all PCORI-funded primary research (i.e., research asking which of two or more approaches is better) must undergo peer review, and all results must be made available to the public.
The goal of peer review is to ensure that the primary research studies funded by PCORI are held to the highest standards of scientific integrity, methodological rigor, and relevance and usefulness to patients, caregivers, clinicians, and other healthcare stakeholders. Our process includes review of study findings by content experts, methodologists, patients, and other healthcare stakeholders with experience related to the study.
How do we assess scientific integrity? Scientific integrity refers to transparency, completeness, and candor as applied to presentations of the scientific work. From experience with journal peer review, we know that some authors communicate only the findings that present their research in the most positive light and underplay the importance of shortcomings in the conduct of the research. Selective reporting of study results and study limitations might affect confidence in the conclusions. Consumers of research need to know the whole story. We ask research teams to provide a full report on the research, including any cautions about interpreting the study results.
How do we assess methodological rigor? PCORI has a comprehensive list of methodological standards that our Methodology Committee developed in 2013. These include both broad standards—those that should be followed by all patient-centered outcomes research—and specific standards, which apply only to certain types of studies. Our peer review process will evaluate how well the investigators met our methodology standards. Our findings will help consumers of research to judge whether the study results will apply to their clinical setting.
How do we assess relevance and usefulness? Scientific integrity and methodological rigor are specified as goals in our authorizing legislation. Our Board added a third goal when it approved our peer review process in 2015. The Board wanted to ensure that the final research reports address real problems faced by patients, caregivers, clinicians, and others, and provide information that they can use to make better healthcare decisions. In our peer review process, we ask patients and other stakeholders serving as reviewers whether the research findings are relevant to them as representatives of patients, clinicians, and others.
What are the Investigator’s Responsibilities in Peer Review?
The principal investigator, who heads the research team, is responsible for meeting the peer review milestones laid out in the research contract. These may include the submission of results tables to ClinicalTrials.gov or another required registry, submission of the draft final research report for peer review, and revisions of that report based on peer review comments. The draft final research report will be due as specified by the contract, four to six months after either data collection is complete for the primary outcome (the Primary Completion Date) or the end of the project research period. If the project is required to submit results for posting on ClinicalTrials.gov or another registry, it must do so at least 30 days before submitting the draft final research report. The instructions for completing the draft final research report provide a detailed guide to submitting results to a registry and writing the report.
Who Are Our Peer Reviewers?
PCORI peer reviewers come from a variety of communities under the healthcare umbrella. Reviewers include
- Scientists and other subject-matter specialists
- Biostatisticians and other methodologists who can assess the quality of the study methods
- Clinicians and other healthcare stakeholders, such as managers of clinics, manufacturers of healthcare devices, and policy makers
- Patients, caregivers, or others with a personal stake in the research topic
Peer review has an important role in ensuring the quality and usefulness of our research findings. If the research funded by PCORI is to have value for the diverse communities within the healthcare system, we need input from a wide variety of individuals to provide our peer review process with the knowledge and perspectives they have acquired through their own experiences.
How Does Peer Review Work?
PCORI’s Process for Peer Review provides a roadmap that reflects the requirements of our authorizing law. Peer review is part of PCORI's program to put knowledge gained from high-quality patient-centered outcomes research (PCOR) into practice. Patients and the rest of the healthcare community had a vital role in developing the dissemination and implementation program.
Our roadmap begins with the principal investigator submitting a draft final research report that includes background about the project and the project’s aims and methods, as well as results and discussion of all study aims. At PCORI, the project’s program officer reviews the draft report to make sure that it includes all of the necessary components and is ready to enter peer review.
The principal investigator submits the draft report to our electronic peer review system, which supports a process similar to that of a peer-reviewed journal. An associate editor assigns reviewers—from the appropriate scientific specialties and healthcare communities—and later, with input from PCORI staff, assembles their key concerns and suggestions into a letter to the principal investigator. This process takes about 60 days. The PI then has 45 working days (about nine weeks) to respond to the points raised and make changes in the draft report.
The associate editor and the project’s program officer then either recommend that PCORI accept the report as final or ask the principal investigator for additional revisions. On rare occasions, if the revision fails to address major concerns, a second round of peer review may be necessary.
Once PCORI accepts the final research report, PCORI’s Patient-Centered Outcomes Research Translation Center completes two summaries of the project—one for the public and one for health professionals. In accordance with the law, PCORI posts the peer review summary and later the final research report and anonymized comments from the peer reviewers on the project’s web page on the PCORI website.
Draft Final Research Report - Tips for Awardees
Please remember that the draft final research report should be a detailed report of all work completed as part of the PCORI-funded contract. The report should
- Closely follow the draft final research report instructions
- Describe in detail how the researchers engaged patients and other healthcare stakeholders in the development, execution, and interpretation of the study
- Include the study protocol as an appendix
- Incorporate all study aims, methods, and results that were planned and executed
- Include the statistical analyses and results, even if inconclusive, for all analyses planned in the study protocol
- Pay particular attention to describing the study limitations and how they could influence the study results and conclusions
- Draft Final Research Report: Instructions for Awardees
- Step-by-Step Instructions for Awardees: Peer Review of Draft Final Research Report
- Instructions for PCORI Online: How to Submit the Draft Final Research Report
- Ancillary Information Conflicts of Interest Disclosure Form Relating to PCORI-Funded Research Project
Posted: October 10, 2016; Updated: March 9, 2017