The Peer Review Process
What is being reviewed during the peer review process?
Each project’s draft final research report will be reviewed. This report includes the background, methods, results, and conclusions for the complete PCORI-funded study protocol. It should include all aims, analyses, and outcomes. The instructions for preparing the report and a description of the report’s content is provided here.
Where should the draft final research report be submitted?
Awardees are instructed to submit the draft final research report and all related materials to PCORI Online. Instructions for submission can be found here.
What happens after the awardee submits the report?
The project’s program officer will check the report to make sure it is a complete account of all methods and analyses planned in the study protocol and is ready for peer review. The program officer may request revisions if some required sections of the report are incomplete or missing. Awardees will be given two weeks to submit a revised report. The program officer will then review the revised report.
How does the program officer judge the draft final research report?
The program officer determines whether the draft final research report is adequately developed and includes the necessary components to go through peer review. The program officer does not judge the quality of the science. That is the role of the peer reviewers.
In evaluating the report’s readiness, the program officer may ask the following questions:
- Does the abstract sufficiently describe the background, objectives, methods, results, and conclusions of the approved research project?
- Does the background section sufficiently describe critical evidence gaps, the research question(s), and the potential significance of the research and its results?
- Does the report sufficiently describe the involvement of patients and other stakeholders in the design, conduct, and dissemination of research?
- Does the methods section sufficiently describe the study population, formation of the study cohort, and reasons for choices in ineligibility criteria, design, setting, interventions, data sources, and statistical approaches?
- Does the results section sufficiently present key findings related to the specific research questions, study aims, and hypotheses of the approved research project? Does the draft adhere to the appropriate reporting guidelines?
- Does the discussion section sufficiently address applicability and generalizability of results beyond the study settings? Does it place results in the context of the existing body of evidence, as well as present implementation considerations and study limitations?
- Is the conclusions section appropriately tailored to the study results? Does it address threats to internal and external validity?
- Are the tables and figures in the correct order? Do they have relevant titles, descriptions, and/or footnotes?
- Does the report include a sufficient description of the PCORI Methodology Standards applicable to the research study?
- Has the investigator adhered to PCORI’s instructions to awardees and produced a draft report that is acceptable to proceed to peer review?
What happens once the program officer approves the draft final research report?
Once the program officer deems the draft final research report acceptable, the awardee will receive an invitation from PCORI’s peer review system (Editorial Manager) to submit the draft report for peer review. In peer review, the report will be assigned to an associate editor as well as four peer reviewers. The reviewers will have two to three weeks to write their critiques. The associate editor will review these and then prepare a letter synthesizing the reviewers’ main comments.
How long does the peer review process take?
The initial round of peer review can take about 60 days from submission of the draft final research report to Editorial Manager to completion of the synthesis letter. We estimate that the full process of peer review, from initial submission of the draft final research report to PCORI, through PCORI’s acceptance of the final research report, will average six months.
How will I get the results of the peer review?
The PCORI Editorial Office will notify the awardee once the synthesis letter is ready. The letter will provide the unedited peer reviews and instructions for submitting any revisions.
How long do awardees have to respond to reviewers’ comments?
As noted in PCORI’s Process for Peer Review of Primary Research and Public Release of Research Findings, awardees have 45 working days (about nine weeks) to respond to reviewers’ comments. However, we are willing to work with awardees to adjust these deadlines.
What happens after we revise our report?
You will submit to Editorial Manager both the revised draft final research report and a disposition table of reviewers’ comments. The associate editor will notify you if there are additional revisions to be made, or if (in rare cases) a second round of peer review is required. If the revisions are complete, the associate editor will recommend that the draft final research report be accepted as final.
What is the difference between the draft final research report and the final research report?
The submitted report is considered “draft” until it has been peer reviewed, revised if necessary, and accepted by PCORI. After acceptance, the final research report will be posted on our website.
Preparing Draft Final Research Reports
What is the difference between the final progress report and the draft final research report, both of which are due at the end of the study?
The final progress report documents the final phase of the study. It includes how the study was completed; whether the team met the milestones set forth in the contract; any challenges encountered; remedial actions taken in mitigating and resolving concerns, including risks to study completion; and notable accomplishments. Study findings are not necessarily included in the final progress report. The final progress report is reviewed by PCORI staff but is not publicly disclosed.
In contrast, the draft final research report documents all of the work completed in the PCORI-funded study. This report must include a detailed description of the study’s background, methods, results, and conclusions, consistent with PCORI’s Process for Peer Review of Primary Research and Public Release of Research Findings. The draft final research report goes through external peer review and may be revised by the awardee before it is accepted by PCORI as the final research report. This final report will be posted on the PCORI website.
When should the draft final research report be submitted?
The draft final research report should be submitted by the date listed in the study’s negotiated milestones. Typically, this will be four to six months after the primary completion date (i.e., the date the last participant has been evaluated, the date of final collection of data for the primary outcome, or another date to be determined for studies not registered at ClinicalTrials.gov). Although PCORI’s Process for Peer Review of Primary Research and Public Release of Research Findings states that the report must be submitted within 13 months of the primary completion date, our goal is to peer review and post the reports much sooner in order to meet our mandate to get research results to the public as quickly as possible.
What information should I include with the draft final research report?
The Instructions to Awardees provides a detailed description of the information to be included. Along with the draft final research report, your submission should include a copy of your study protocol, the results tables that you prepared and submitted to ClinicalTrials.gov or another database, the PCORI Methodology Standards checklist, and the Ancillary Information form identifying conflicts of interest for you and the institution.
How polished must the draft final research report be?
Awardees should submit their best version of the report. It should include all sections outlined in the Instructions to Awardees, as well as any necessary ancillary information. The report should include all analyses and results planned in the study protocol. If the report is not ready for peer review, the program officer will ask for additions or other changes.
A manuscript with our study’s main findings has been peer reviewed and published in a journal. Can I submit that as my report?
No. The draft final research report must follow a specific format to be considered adequate for peer review. Peer reviewed journal articles also tend to focus on only part of the research conducted in a funded study, but the draft final research report should include a full accounting of the methods and analyses in the study protocol.
Can I use parts of a published manuscript in the draft final research report, such as tables and figures already published?
Yes, but please note that if you are planning to include any figures and tables or extensive text in the draft final research report that you have used in manuscripts you have published (or intend to submit for publication), you may need the journal publisher’s permission. Use of these materials may be affected by the length of any text to be reused and whether you transferred copyright to a journal as part of your author agreement or you published under an open access license where you retain rights. It is your responsibility to check with the journal publisher and get permission for using any part of the manuscript in your draft final research report. For our records, please provide us a copy of any approvals you receive from the publisher.
What will be posted on PCORI’s website after the peer review process is completed, and when?
As noted in PCORI’s Process for Peer Review of Primary Research and Public Release of Research Findings, within 90 days of accepting the final research report, PCORI will post
- A 500-word abstract written in language accessible to the public (with audio and Spanish translations)
- A 500-word abstract for medical professionals
- A link to the study results tables on ClinicalTrials.gov (if applicable)
- Ancillary Information required by law
- Identity of the institution and investigators conducting the research
- Any conflicts of interest reported by the entity and investigators conducting the research
- Any direct or indirect links of the entity to industry
- Anonymized comments from peer reviewers and your responses
- A summary of the peer review process
The final research report will also be posted on the website no later than 12 months after PCORI approval. This gives the research team extra time to publish manuscripts in peer refereed journals.
After findings are posted to the PCORI website, is there an opportunity to submit additional results?
Yes, awardees may update or otherwise refine the information posted at pcori.org, for example, to align with information in other publications or updates submitted to ClinicalTrials.gov, or to provide new or updated material.
Is posting results tables to ClinicalTrials.gov considered prior publication? What about the summaries that are posted on PCORI’s website?
In laying the groundwork for PCORI’s Peer Review Process, our leadership spoke with the editors of several top-tier journals and representatives from the National Library of Medicine as to what would constitute prior publication and jeopardize an investigator’s ability to publish a paper resulting from the project. The guidance we received matches the policy of the International Committee of Medical Journal Editors (ICMJE):
The ICMJE will not consider as prior publication the posting of trial results … if results are limited to a brief (500 word) structured abstract or tables (to include patients enrolled, key outcomes, and adverse events).
Given this, we are confident that there shouldn’t be major issues with following PCORI’s Peer Review policy, which calls for project results to be posted on ClinicalTrials.gov or other appropriate registries and for 500-word lay and clinician summaries of research results to be posted on PCORI’s website. If a journal editor raises concerns, please let us know, so we can directly contact the editor to explain PCORI’s mandates.
Similarly, the ICMJE does not consider as prior publication the posting of results tables on ClinicalTrials.gov, as required by the Food and Drug Administration Amendments Act of 2007 (U.S. Public Law 110-85, Title VIII). For more information, see
- ICMJE Statement: http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/clinical-trial-registration.html
- ICMJE FAQ: http://www.icmje.org/about-icmje/faqs/clinical-trials-registration/
Are there any restrictions on where awardees can publish articles based on PCORI-funded research? For example, are awardees expected to publish only in open-access journals?
Awardees may publish articles resulting from PCORI-funded projects in any journal at any time. We ask, however, that awardees notify us when manuscripts are submitted to journals and when they are accepted for publication. We also ask that awardees inquire about open-access options because we are interested in linking to the published articles from our website. For more information about PCORI’s policy, refer to Public Access to Journal Articles Presenting Findings from PCORI-Funded Research.
Does PCORI peer-review manuscripts submitted before the completion of the study?
PCORI does not peer-review manuscripts submitted to journals at any time. As noted above, awardees are asked to keep PCORI informed of their plans for publication.
If the final report is posted no later than 12 months after acceptance, what will happen if a manuscript describing that work has not yet published in a peer reviewed journal? Most journals require that the information in submitted manuscripts has not been published previously.
PCORI will work closely with awardees, investigators, and journals to coordinate posting of the final report with publication of papers in press. However, we must balance journal publication with our obligation under our authorizing law, affirmed through public comments on our draft Peer Review Process document, to make the results of PCORI-funded research widely available to the public as expeditiously as possible. The document states, “PCORI will keep all final research reports on file, coordinating their public release on PCORI’s website with Awardee Institutions and principal investigators based on planned publication of journal articles resulting from the study, to avoid disqualifying manuscripts from being considered for publication by a journal.”
Posted: March 27, 2017