The Peer Review Process
What is being reviewed during the peer review process?
Each project’s draft final research report will be reviewed. This report includes the background, methods, results, and conclusions for the complete PCORI-funded study protocol. It should include all aims, analyses, and outcomes. The instructions for preparing the report and a description of the report’s content is provided here.
Where should the draft final research report be submitted?
Awardees are instructed to submit the draft final research report and all related materials to PCORI Online. Instructions for submission can be found here.
What happens after the awardee submits the report?
The project’s program officer will check the report to make sure it is a complete account of all methods and analyses planned in the study protocol and is ready for peer review. The program officer may request revisions if some required sections of the report are incomplete or missing. Awardees will be given two weeks to submit a revised report. The program officer will then review the revised report.
How does the program officer judge the draft final research report?
The program officer determines whether the draft final research report is adequately developed and includes the necessary components to go through Peer Review. The program officer does not judge the quality of the science. That is the role of the peer reviewers. Below is a list of questions the program officer may use in evaluating the report’s readiness:
- Does the draft final research report include an abstract that sufficiently describes the background, objectives, methods, results, and conclusions of the approved research project?
- Does the draft final research report include a sufficiently described BACKGROUND section which introduces critical evidence gaps, research question(s), and the potential significance of research and results being presented? Does the draft final research report include sufficient description of the involvement of patients and other stakeholders in the design, conduct, and dissemination of research?
- Does the draft final research report include a sufficiently described METHODS section which describes the study population, how the study cohort was formed and reasons for ineligibility, design, setting, interventions, data sources, and statistical approaches?
- Does the draft final research report include a sufficiently described RESULTS section which presents key finds related to the specific research questions, study aims, and hypotheses of the approved research project? Does the draft final research report adhere to the appropriate reporting guidelines?
- Does the draft final research report include a sufficiently described DISCUSSION section that addresses applicability and generalizability of results beyond the study settings and places results in the context of the existing body of evidence, as well as implementation considerations, and study limitations?
- Does the draft final research report include a sufficiently described CONCLUSIONS section that is appropriately tailored to the study results and addresses threats to internal and external validity?
- Does the draft final research report include TABLES and FIGURES that are in the correct order, and contain a title, description, and/or footnote of their content?
- Does the draft final research report include a sufficient description of the METHODOLOGY STANDARDS that are applicable to the research study?
- In your judgment has the investigator adhered to PCORI’s draft final research report instructions to awardees and produced a draft final research report that is acceptable to proceed to peer review?
What happens once the program officer approves the draft final research report?
Once the program officer deems the report acceptable, the awardee will receive an invitation from PCORI’s peer review system (Editorial Manager) to submit the draft report for peer review. In peer review, the draft final research report will be assigned to an Associate Editor as well as four peer reviewers. The reviewers will have 2–3 weeks to write their critiques. The Associate Editor will review these and then prepare a letter synthesizing the reviewer’s main comments.
How long does the peer review process take?
The initial round of peer review can about 60 days from submission of the draft final research report to Editorial Manager to completion of the synthesis letter. We estimate that the full process of peer review, from initial submission of the draft final research report to PCORI, to PCORI’s acceptance of the final research report, will average six months.
How will I get the results of the peer review?
The PCORI Editorial Office will notify the awardee once the synthesis letter, with instructions for submitting any revisions and the unedited peer reviews appended.
How long do awardees have to respond to reviewers’ comments?
As noted in PCORI’s Process for Peer Review of Primary Research and Public Release of Research Findings, awardees have 45 working days (about 9 weeks) to respond to reviewers’ comments. However, we are willing to work with awardees to adjust these deadlines.
What happens after we revise our report?
The revised draft final research report and a disposition table of reviewers’ comments will be submitted to Editorial Manager. The Associate Editor will notify you if there are additional revisions to be made, or if (in rare cases) a second round of peer review is required. If the revisions are complete, the Associate Editor will recommend that the draft final research report be accepted as final.
What is the difference between the draft final research report and the final research report?
The submitted report is considered “draft” until it has been peer reviewed, revised if necessary, and accepted by PCORI. After acceptance, the final research report will be posted on our website.
Preparing Draft Final Research Reports
What is the difference between the final progress report and the draft final research report, both of which are due at the end of the study?
The final progress report documents the final phase of the study. It includes how the study was completed; whether the team met the milestones set forth in the contract; any challenges encountered; remedial actions taken in mitigating and resolving concerns, including risks to study completion; and notable accomplishments. Study findings are not necessarily included in the final progress report. The final progress report is reviewed by PCORI staff, but is not publicly disclosed.
In contrast, the draft final research report documents all the work completed in the PCORI-funded study. This report must include a detailed description of the study’s background, methods, results, and conclusions, consistent with PCORI’s Process for Peer Review of Primary Research and Public Release of Research Findings. The draft final research report goes through external peer review and may be revised by the awardee before it is accepted by PCORI as the final research report. This final report will be posted on the PCORI website (see Question 4).
When should the draft final research report be submitted?
The draft final research report should be submitted by the date listed in the study’s negotiated milestones. Typically, this will be four to six months after the primary completion date (i.e., the date the last participant has been evaluated, the date of final collection of data for the primary outcome, or another date to be determined for studies not registered at ClinicalTrials.gov). Although PCORI’s Process for Peer Review of Primary Research and Public Release of Research Findings states that the report must be submitted within 13 months of the primary completion date, our goal is to peer review and post the reports much sooner in order to meet our mandate to get research results to the public as quickly as possible.
What information should I include with the draft final research report?
The Instructions to Awardees provides a detailed description of the information to be included. Along with the draft final research report, your submission should include a copy of your study protocol, the results tables that you prepared and submitted to ClinicalTrials.gov or other database, the checklist of Methodology Standards, and the Ancillary Information Form identifying conflicts of interest for you and the institution.
How polished must the draft final research report be?
Awardees should submit their best version of a final research report. It should include all sections outlined in the Instructions to Awardees, as well as any necessary ancillary information. The report should include all analyses and results planned in the study protocol. If the report is not ready for peer review, the program officer will ask for additions or other changes.
A manuscript with our study’s main findings has been peer reviewed and published in a journal. Can I submit that as my report?
No. The draft final research report must follow a specific format to be considered adequate for peer review. Peer-reviewed journal articles also tend to focus on only part of the research conducted in a funded study, but the draft final research report should include all a full accounting methods and analyses planned in the study protocol.
Can I use parts of a published manuscript in the draft final research report, such as tables and figures already published?
The draft final research report can include information, such as tables, figures, and sections of text, from previously published journal articles. Cite the source of material, including any long sections of text, taken directly from such publications. Awardees are responsible for checking with the journal publisher and, if necessary, getting permission for reprinting or using any part of the published article in the final research report, which will be made publicly available on PCORI’s website following peer review and finalization. Please note that the permissible use of previously published content may depend on the length of text and whether the authors transferred copyright to the journal as part of the author agreement. Please include a copy of any relevant copyright permissions or licenses for PCORI’s records as a separate file with your submission of the draft final research report.
Research Publication
What will be posted on PCORI’s website after the peer review process is completed, and when?
As noted in PCORI’s Process for Peer Review of Primary Research and Public Release of Research Findings, within 90 days of accepting the final research report, PCORI will post:
- A 500-word abstract written in language accessible to the public (with audio and Spanish translations)
- A 500-word abstract for medical professionals
- A link to the study results tables on ClinicalTrials.gov (if applicable)
- Ancillary Information required by law:
- Identity of the institution and investigators conducting the research
- Any conflicts of interest reported by the entity and investigators conducting the research
- Any direct or indirect links the entity has to industry
- Anonymized comments from peer reviewers and the Awardee Institutions’ responses
- A summary of the peer-review process
After findings are posted to the PCORI website, is there an opportunity to submit additional results?
Yes, awardees may update or otherwise refine the information posted at pcori.org, for example, to align with information in other publications or updates submitted to ClinicalTrials.gov, or to provide new or updated material.
Is posting results tables to ClinicalTrials.gov considered prior publication? What about the summaries that are posted on PCORI’s website?
In laying the groundwork for PCORI’s peer review process, PCORI leadership staff had phone conversations with the editors of several top-tier journals and representatives from the National Library of Medicine as to whether what PCORI plans to fulfill its legal mandate would constitute prior publication and jeopardize a PI’s ability to publish a paper resulting from his/her project. The guidance we received matches the policy of the International Committee of Medical Journal Editors (ICMJE):
The ICMJE will not consider as prior publication the posting of trial results … if results are limited to a brief (500 word) structured abstract or tables (to include patients enrolled, key outcomes, and adverse events).
Given this, we are confident that there shouldn’t be any major issues with following PCORI’s peer review process, which calls for project results to be posted on ClinicalTrials.gov or other appropriate registries and for lay and clinician summaries of research results to be posted on PCORI’s website. If a journal editor raises concerns, please let us know so we can contact the editor directly to discuss.
Similarly, the ICMJE does not consider as prior publication the author’s requirements under the Food and Drug Administration Amendments Act of 2007 (U.S. Public Law 110-85, Title VIII) to post results tables on ClinicalTrials.gov. For more information, see:
- ICMJE Statement: http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/clinical-trial-registration.html
- ICMJE FAQ: http://www.icmje.org/about-icmje/faqs/clinical-trials-registration/
Are there any restrictions on where awardees can publish articles based on PCORI-funded research? For example, are awardees expected to publish only in open-access journals?
Awardees may publish articles resulting from PCORI-funded projects in any journal at any time. We ask, however, that awardees notify us when manuscripts are submitted to journals and when they are accepted for publication. We also ask that awardees inquire about open-access options because we are interested in linking to the published articles from our website. For more information about PCORI’s policy, refer to Public Access to Journal Articles Presenting Findings from PCORI-Funded Research.
Does PCORI peer-review manuscripts submitted before the completion of the study?
PCORI does not peer review manuscripts submitted to journals at any time. As noted above, awardees are asked to keep PCORI informed of their plans for publication.
If the final report is posted no later than 12 months after acceptance, what will happen if a manuscript describing that work has not yet published in a peer reviewed journal? Most journals require that the information in submitted manuscripts has not been published previously.
PCORI will work closely with awardees, investigators, and journals to coordinate posting of the final report with publication of papers in press. However, we must balance journal publication with our obligation under our authorizing law, affirmed through public comments on our draft Peer Review Process document, to make the results of PCORI-funded research widely available to the public as expeditiously as possible. The document states, “PCORI will keep all final research reports on file, coordinating their public release on PCORI’s website with Awardee Institutions and principal investigators based on planned publication of journal articles resulting from the study, to avoid disqualifying manuscripts from being considered for publication by a journal.”
Posted: March 27, 2017; Updated: July 18, 2017