Our peer-review process starts with study investigators writing a Draft Final Research Report (DFRR) once their study analyses are complete. This report should include a complete account of all the aims, methods, outcomes, and results from the PCORI-funded study. The study investigators submit the DFRR into our secure online peer-review system, ReView, where it will be checked to make sure the submission is complete. The online system, associate editors, and reviewers are provided by our peer-review contractor.

The DFRR will then enter pre-review where an experienced editor will check it for clarity and the project’s Science Program team will check it for completeness. The DFRR is then sent to peer reviewers for their comments and an associate editor writes a synthesis of reviewers’ comments for the investigators. The investigators will be asked to revise the report at least once but often twice to address the peer-reviewer comments.

Finally, PCORI’s Program Director for Peer Review and Scientific Publications reads through the report to make sure it is clear and understandable for a clinical researcher who is not an expert in the topic. Once the program director’s review is complete, the report will be accepted as final and the peer-review process is complete!

The peer-review process, from submission to acceptance, takes an average of seven to eight months. This timeline shows what happens in each phase of peer review in more detail:

Preparing and Submitting the DFRR

About four to six months before the DFRR is due, the principal investigator and their research team will meet with the peer-review contractor’s Author Services team to learn what to expect from the peer-review process and receive advice about the organization of the DFRR. Principal investigators can contact the Author Services team after this meeting for advice or feedback on their DFFRs.

The Author Services team will again reach out to the principal investigator about a week before the submission due date to provide them with an invitation and link to submit their DFRR and accompanying materials into the ReView system. The Author Services team is also responsible for the first readthrough of the DFRR, known as pre-review, to help make sure that the DFRR is clear, complete, and ready to send to reviewers. The principal investigator will receive comments from the team in about two weeks and will be asked to revise and resubmit their report in two weeks. Once the principal investigator revises the report adequately, the DFRR will move on to external peer review.

The External Peer-Review Process

When the DFRR enters external peer review, the assigned associate editor will recruit subject matter experts, methodologists, and patients or stakeholder reviewers, three to four in total, to carefully review the report. A statistician will also review the DFRR.  The goal of external peer review is to assess the report’s scientific integrity; adherence to PCORI Methodology Standards; and relevance and usefulness to patients, clinicians, and other stakeholders as described in the 2015 guidance on the peer-review process.

Scientific integrity. Scientific integrity refers to transparency, completeness, and candor as applied to presentations of the scientific work. From experience with journal peer review, we know that some authors communicate only the findings that present their research in the most positive light and underplay the importance of shortcomings in the conduct of the research. Selective reporting of study results and study limitations might lead a reader to overestimate the benefits of certain interventions, or to have more confidence in the conclusions than the results warrant. Consumers of research need to know the whole story. We ask research teams to provide a full report on the research, including a full account of any cautions about interpreting the study results.

Adherence to PCORI’s Methodology Standards. The PCORI Methodology Standards is a comprehensive list developed in 2013 by the PCORI Methodology Committee and subsequently expanded to cover additional topics. These include both broad “cross-cutting” standards—those that should be followed by all patient-centered outcomes research studies—and specific standards that apply only to certain types of studies. Our peer-review process will evaluate how well the investigators met our methodology standards, and it will encourage investigators to report information needed to assess their adherence to PCORI’s standards. 

Relevance and usefulness.  Scientific integrity and methodological rigor are specific goals in our authorizing legislation. Our Board of Governors added a third goal when it approved our peer-review process in 2015. The Board wanted to ensure that the Final Research Reports address real problems faced by patients, caregivers, clinicians, and others. In our peer-review process, we ask patients and other stakeholders serving as reviewers whether the research findings will help people like them make informed healthcare decisions.

Posted: March 8, 2019; Updated: October 18, 2022