PCORI Merit Review is a multi-phase process that includes the external review panel’s preliminary online review of full applications and an in-person panel discussion of a subset of applications (identified by PCORI’s Program staff and based on the preliminary review and program priorities). Merit review is informed by the PFA Development and LOI Review processes. After merit review, key steps include: post-panel review of application by PCORI staff; Selection Committee recommendation of applications for funding; and finally, Board of Governors award approval.
The Broad PCORI Funding Announcements (PFAs) are developed based on PCORIs five National Priority Areas which include: Addressing Disparities; Assessment of Prevention, Diagnosis, and Treatment Options; Communication and Dissemination Research; Improving Healthcare Systems; and Improving Methods for Conducting Patient-Centered Outcomes Research. The Targeted PFAs are developed by PCORI staff with input from patients and other stakeholders on research topics which are later prioritized. Additional evidence is collected and shared with Advisory Panels, Science Oversight Committee, and relevant workgroups. The Board of Governors-approved topics are incorporated into the funding announcement.
PCORI staff screen LOIs based on importance and relevance of the topic to PCORI priorities; response to LOI questions; and other criteria as outlined in the PFA and invite full applications only from applicants whose LOIs are deemed most responsive to the PFA. Only invited applicants may submit a full application.
After receipt of invited full applications, PCORI staff:
- Review the application for administrative compliance and responsiveness to the funding announcement.
- May administratively withdraw an application if it is incomplete; submitted past the stated due date and time; or it does not meet the formatting criteria outlined in the Application Guidelines, in the PCORI templates, and in PCORI Online.
- May scientifically withdraw an application if it is non-responsive to the PFA.
PCORI Merit Review Officers (MROs) recruit external reviewers to serve on panels based on their level of expertise or experience with the topic areas in the invited LOIs. MROs recruit the panel chair and all reviewers, including scientists, patient representatives, and other stakeholder representatives. All panel reviewers are trained to help ensure that they have the tools needed to evaluate each application based on the programmatic and organizational goals of merit review.
On the basis of their expertise and absence of conflict of interest, members of each panel are assigned to complete a preliminary online review of assigned applications based on:
- Merit Review Criteria
- Adherence to PCORI’s Methodology Standards
- Appropriate human subjects protections
Merit Review Officers (MROs) monitor the critiques and scores, and work with reviewers to ensure that critiques provide clear feedback to inform PCORI staff as they consider which applications to recommend for funding and negotiate contracts with successful applicants. Reviewer critiques are also included in the final summary statements that are issued to applicants at the end of the process.
- On the basis of application critiques and scores, PCORI staff select a subset of applications to move forward for discussion at the in-person review panel. Not all submitted applications move forward to in-person review, but reviewers evaluate and score all applications according to PCORI’s Merit Review criteria, including evaluation of adherence to PCORI’s Methodology Standards.
- All members of the panel discuss to further clarify the merits of the proposed research and identify areas for improvement, and re-score the applications included in discussion.
- The in-person panel meeting is led by a Chair and a PCORI MRO, who ensure that all applications receive a fair and thorough review that is informed by the standards outlined in the PFA. The MROs also prepare notes from the discussion to provide applicants and PCORI staff with additional feedback.
- PCORI staff consider the critiques from preliminary review, discussion notes, final application scores following discussion, and portfolio balance to propose a slate of applications for funding.
- Factoring in the total available funds allotted for their PFA, PCORI staff present to the Selection Committee high-scoring applications that fit programmatic needs, satisfactorily address reviewers’ concerns, and adhere to PCORI’s Methodology Standards. If PCORI staff have a strong interest in an application but have programmatic and administrative concerns, staff send the applicant their questions and concerns for response by a given deadline (also known as PCORI Information Request, PIR).
- The Selection Committee is made up of members of PCORI’s Methodology Committee and a subset of the Board of Governors. The Selection Committee reviews and recommends the final slates of proposed projects to the Board for approval during a public meeting.
- Applicants receive their summary statements approximately two weeks before funding decisions are announced. If an application progressed to in-person discussion, it will include:
- Discussion notes from the in-person meeting
- Final average overall score and quartile (so applicants may understand how they did relative to other discussed applications)
- Preliminary critiques
If an application did not progress to in-person discussion, the summary statement will include only the preliminary critiques.
For further information about a funded application's journey, see our blog post.