The Peer-Review Process
What is under review during the peer-review process?
Reviewers consider each project’s draft final research report (DFRR). This report includes the background, methods, results, and conclusions for the complete PCORI-funded study protocol. The report should include all aims, analyses, and outcomes. Instructions for preparing the report and a description of the report’s content appear here.
Where should awardees submit DFRRs?
About two weeks before DFRR due dates, awardees will receive an email from our editorial office inviting them to submit DFRRs through an individual link to our peer-review system. Instructions for submission appear here.
What happens after the awardee submits the report?
Once submitted, the DFRR is reviewed by a scientist-editor (a program officer in the PCORI peer-review office or an associate editor) to make sure that it includes the required information. The editor will send awardees feedback within two weeks of report submission, and awardees will then have two weeks to make any revisions.
How does the scientist-editor judge the DFRR?
The scientist-editor will determine whether the DFRR is adequately developed and includes the necessary components to go through peer review. The scientist-editor will be checking whether the DFRR is written so that a general audience of scientists can understand it well enough to evaluate the underlying science content. Some specific aspects scientist-editors look at include:
- Clarity of language
- Comprehensive discussion of each area indicated in the DFRR instructions, along with the appropriate sub-headings.
- Absence of field-specific jargon
- Description of methods and results that are clear enough for a peer reviewer to assess scientific integrity
What happens once the scientist-editor approves the DFRR?
Once the scientist-editor deems a report acceptable, awardees will receive a notification from PCORI’s peer-review system, called Editorial Manager, that the DFRR is in peer review. In peer review, the DFRR will be assigned to an associate editor in addition to as many as four peer reviewers. The reviewers read the reports and write critiques. The associate editor then reviews these critiques and prepares a letter synthesizing the reviewers’ main comments.
How long does the peer-review process take?
Once the DFRR is approved to go to peer review, it’s automatically moved into that status and sent out to reviewers. Peer review can take about 60 days from the scientist-editor’s approval of the DFRR to peer review, to completion of the synthesis letter. We estimate that the full process of peer review, from initial submission of the DFRR to PCORI’s acceptance of the final research report, will average six months.
How will awardees get the results of the peer review?
All information about peer-review results are provided electronically. The PCORI editorial office sends the synthesis letter, with instructions for submitting any revisions. Unedited peer reviews are appended to the email.
How long do awardees have to respond to reviewers’ comments?
As noted in PCORI’s Process for Peer Review of Primary Research and Public Release of Research Findings, awardees have 45 working days (about 9 weeks) to respond to reviewers’ comments. For a second revision, awardees have two to four weeks to make revisions. However, in some cases we are willing to work with awardees to adjust these deadlines.
What happens after awardees revise their reports?
Awardees then submit the revised DFRR and a disposition table of reviewers’ comments to editorial manager. Associate editors will notify awardees if additional revisions are necessary, or if a second round of peer review is required. If the revisions are complete, the associate editor will recommend that the DFRR be accepted and move on for PCORI final approval.
What is PCORI Final approval?
Once our peer-review contractor determines that the report has met the requirements of peer review, PCORI’s peer-review program director will provide a final review of the report. The program director notifies the PI regarding next steps in the final review process by email. The program director will review the associate editor-accepted version of the DFRR and if there are additional changes needed, they will be outlined to the PI in an email with a due date for resubmission; if no changes are desired, then a final decision letter will be sent out, completing peer review. PCORI must accept the report as final before awardees have fulfilled their contractual obligations.
What is the difference between the DFRR and the final research report?
The submitted report is considered a draft until it has been peer reviewed, revised if necessary, and accepted by PCORI. After acceptance, the final research report will be posted on our website.
Preparing Draft Final Research Reports
What is the difference between the final progress report and the draft final research report (DFRR), both of which are due at the end of the study?
The final progress report documents the final phase of the study. It includes how the study was completed; whether the team met the milestones set forth in the contract; any challenges encountered; remedial actions taken in mitigating and resolving concerns, including risks to study completion; and notable accomplishments. Study findings are not necessarily included in the final progress report. The final progress report is reviewed by PCORI staff, but is not publicly disclosed.
In contrast, the DFRR documents all of the work completed in the PCORI-funded study. This report must include a detailed description of the study’s background, methods, results, and conclusions, consistent with PCORI’s Process for Peer Review of Primary Research and Public Release of Research Findings. The DFRR goes through external peer review and may be revised by the awardee before it is accepted by PCORI as the final research report. This final report will be posted on the PCORI website.
What information should awardees include with the DFRR?
Along with the DFRR, a submission should include a copy of the study protocol, the checklist of Methodology Standards, and the Ancillary Information form identifying conflicts of interest for PIs and the institution. More specific instructions on what to include with DFRRs appear in the Instructions to Awardees.
How polished must the DFRR be?
Awardees should submit the best version of a final research report, of the same quality that would be submitted to a journal. It should include all sections outlined in the Instructions to Awardees, as well as any necessary ancillary information. The report should include all analyses and results planned in the study protocol. If the report is not ready for peer review, the program officer will ask for additions or other changes.
A manuscript with our study’s main findings has been peer reviewed and published in a journal. Can awardees submit that as their report?
No. The DFRR must follow a specific format to be considered adequate for peer review. Peer-reviewed journal articles also tend to focus on only part of the research conducted in a funded study, but the DFRR should include a full accounting of methods and analyses planned in the study protocol.
Can awardees use parts of the published manuscript in the DFRR, such as tables and figures already published?
The DFRR can include information, such as tables, figures, and sections of text, from previously published journal articles. Cite the source of material, including any long sections of text, taken directly from such publications. Awardees are responsible for checking with the journal publisher and, if necessary, getting permission for reprinting or using any part of the published article in the final research report, which will be made publicly available on PCORI’s website following peer review and finalization. Please note that the permissible use of previously published content may depend on the length of text and whether the authors transferred copyright to the journal as part of the author agreement. Please include a copy of any relevant copyright permissions or licenses for PCORI’s records as a separate file when submitting the DFRR. For more information on copyright see the ICMJE Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly work in Medical Journals.
Research Publication
What will be posted on PCORI’s website after the peer-review process is completed, and when?
As noted in PCORI’s Process for Peer Review of Primary Research and Public Release of Research Findings, within 90 days of accepting the final research report, PCORI will post:
- A 500-word abstract written in language accessible to the public (See projects that have moved to the status of Completed PCORI Peer Review with Posted Results)
- A 500-word abstract for medical professionals
- A link to the study results tables on ClinicalTrials.gov (if applicable)
- Ancillary Information required by law:
- Identity of the institution and investigators conducting the research
- Any conflicts of interest reported by the entity and investigators conducting the research
- Any direct or indirect links the entity has to industry
- Anonymized comments from peer reviewers and the awardee’s responses as applicable
- A summary of the peer-review process
In addition, within 12 months of peer review approval, the final research report, accompanying appendices, and all anonymized comments from peer reviewers and awardee’s responses will be posted.
What is posted online about the peer review of my DFRR?
As with any peer review process, the editors and content experts who review the DFRRs that awardees submit when they complete their studies often ask the researcher to address concerns about the way the project was conducted, and/or its results and conclusions. Awardees usually revise the report to address these concerns to the reviewers’ satisfaction, and PCORI posts a summary of that process along with public and professional abstracts of the study results.
On rare occasions, researchers disagree with a reviewer comment key to interpreting the study results. When that disagreement cannot be resolved, PCORI describes it in the peer review summary and invites awardees to submit a brief explanation of their reasoning. Such statements are limited to 200 words and subject to review by PCORI program staff. Any such awardee response will be posted with the peer-review summary.
We do not expect awardees to ask to write a response to the peer-review summary when there are no unresolved issues. But if an awardee does so, or asks for changes to the summary for clarification, we will consider such requests on a case-by-case basis.
After findings are posted to the PCORI website, is there an opportunity to submit additional results?
Yes, awardees may update or otherwise refine information posted at pcori.org to align with information in other publications or updates submitted to ClinicalTrials.gov, or to provide new or updated material.
Is posting results tables to ClinicalTrials.gov considered prior publication? What about the summaries that are posted on PCORI’s website?
In laying the groundwork for PCORI’s Peer Review Process, PCORI leadership met with editors of top-tier journals and representatives from the National Library of Medicine to discuss what PCORI planned to fulfill its legal mandate, and whether it would constitute prior publication and jeopardize a PI’s ability to publish a paper resulting from his/her project.
The guidance we received matches the policy of the International Committee of Medical Journal Editors (ICMJE):
The ICMJE will not consider as prior publication the posting of trial results … if results are limited to a brief (500 word) structured abstract or tables (to include patients enrolled, key outcomes, and adverse events).
Given this, we are confident there shouldn’t be any major issues with following PCORI’s peer-review policy, which calls for project results to be posted on ClinicalTrials.gov or other appropriate registries, and for lay and clinician summaries of research results to be posted on PCORI’s website. If a journal editor raises concerns, let us know so we can contact the editor directly to discuss.
Similarly, the ICMJE does not consider as prior publication the author’s requirements under the Food and Drug Administration Amendments Act of 2007 (U.S. Public Law 110-85, Title VIII) to post results tables on ClinicalTrials.gov. For more information, see
- ICMJE Statement: http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/clinical-trial-registration.html
- ICMJE FAQ: http://www.icmje.org/about-icmje/faqs/clinical-trials-registration/
Are there any restrictions on where awardees can publish articles based on PCORI-funded research? For example, are awardees expected to publish only in open-access journals?
Awardees may publish articles resulting from PCORI-funded projects in any journal at any time. We ask, however, that awardees notify us when manuscripts are submitted to journals and when they are accepted for publication. Additionally, we ask that awardees inquire about open-access options because we are interested in linking to the published articles from our website. For more information about PCORI’s policy, refer to Public Access to Journal Articles Presenting Findings from PCORI-Funded Research.
Does PCORI peer review manuscripts submitted before the completion of the study?
PCORI does not peer review manuscripts submitted to journals at any time. As noted above, awardees are asked to keep PCORI informed of their plans for publication.
If the final report is posted no later than 12 months after acceptance, what will happen if a manuscript describing that work has not yet published in a peer-reviewed journal? Most journals require that the information in submitted manuscripts has not been published previously.
PCORI will work closely with awardees, investigators, and journals to coordinate posting of the final report with publication of papers in press. However, we must balance journal publication with our obligation under our authorizing law, affirmed through public comments on our draft Peer Review Process document, to make the results of PCORI-funded research widely available to the public as expeditiously as possible. The document states, “PCORI will keep all final research reports on file, coordinating their public release on PCORI’s website with Awardee Institutions and principal investigators based on planned publication of journal articles resulting from the study, to avoid disqualifying manuscripts from being considered for publication by a journal.”
Posted: March 27, 2017; Updated: August 21, 2018