PCORI Merit Review
PCORI Merit Review is a multi-phase process that includes: evaluation of Letters of Intent by committees of PCORI staff; preliminary review of full applications by external reviewers; in-person panel discussion of full applications (selected by PCORI staff, based on the preliminary review scores and program priorities); Selection Committee recommendation of applications for funding; and finally, Board of Governors award approval.
We conduct scientifically rigorous evaluations of all responsive research applications. Our review process is distinguished by the participation of scientists, patients, and other healthcare stakeholders, enabling us to identify high-quality, methodologically rigorous, and patient-centered studies.
Letter of Intent
Committees of PCORI staff screen Letters of Intent based on the criteria outlined in the PFA and invite full applications only from those applicants whose proposed research best fits with programmatic requirements and priorities. Only invited applicants may submit a full application.
Full Application Receipt
After receipt of invited full applications, PCORI staff:
- Review the application for administrative compliance and responsiveness to the funding announcement.
- May administratively withdraw an application if it is incomplete or submitted past the stated due date and time, or it does not meet the administrative or formatting criteria outlined in the Application Guidelines, in the PCORI templates, and in the PCORI Online System.
- May withdraw an application if it is non-responsive to the PFA.
PCORI Merit Review Officers (MROs) recruit and train reviewers to serve on panels based on their level of experience or expertise with the topics in the applications.
On the basis of their expertise and absence of conflict of interest, members of each panel are assigned to complete a preliminary online review of assigned applications based on:
- Merit Review Criteria
- Adherence to PCORI’s Methodology Standards
- Appropriate human subjects protections
Merit Review Officers (MROs) monitor the critiques and scores, and work with reviewers to ensure that critiques provide clear feedback to inform PCORI staff as they consider which applications to recommend for funding and negotiate contracts with successful applicants. Reviewer critiques are also included in the final summary statements that are issued to applicants at the end of the process.
Selecting Applications for Discussion
- On the basis of application critiques and scores, PCORI staff select a subset of applications to move forward for discussion at the in-person review panel. Not all submitted applications move forward to in-person review, but reviewers evaluate and score all applications according to PCORI’s Merit Review criteria, including evaluation of adherence to PCORI’s Methodology Standards.
In-Person Meeting of the Review Panel
- All members of the panel discuss to further clarify the merits of the proposed research and identify areas for improvement, and re-score the applications included in discussion.
- The in-person panel meeting is led by a Chair and a PCORI MRO, who ensure that all applications receive a fair and thorough review that is informed by the standards outlined in the PFA. The MROs also prepare notes from the discussion to provide applicants and PCORI staff with additional feedback.
Application Selection for Funding and Summary Statement Release
- PCORI staff consider the critiques from preliminary review, discussion notes, final application scores following discussion, and portfolio balance to propose a slate of applications for funding.
- Factoring in the total available funds allotted for their PFA, PCORI staff present to the Selection Committee high-scoring applications that fit programmatic needs, satisfactorily address reviewers’ concerns, and adhere to PCORI’s Methodology Standards. If PCORI staff have a strong interest in an application but have programmatic and administrative concerns, staff send the applicant their questions and concerns for response by a given deadline (also known as PCORI Information Request, PIR).
- The Selection Committee is made up of members of PCORI’s Methodology Committee and a subset of the Board of Governors. The Selection Committee reviews and recommends the final slates of proposed projects to the Board for approval during a public meeting.
- Applicants receive their summary statements approximately two weeks before funding decisions are announced. If an application progressed to in-person discussion, it will include:
- Discussion notes from the in-person meeting
- Final average overall score and quartile (so applicants may understand how they did relative to other discussed applications)
- Preliminary critiques
If an application did not progress to in-person discussion, the summary statement will include only the preliminary critiques.
For further information about a funded application's journey, see our blog post.
Board of Governors Approval
- PCORI staff present the funding slates at a public Board meeting.
- The Board considers the recommendation from the Selection Committee and approves or denies the awards.
- PCORI announces the awards.
Posted: September 10, 2014; Updated: March 10, 2016