Evaluating Our Application Merit Review Process
PCORI's Merit Review Process is unlike that of any other U.S. research funder, bringing together patients and others with a stake in the research, along with research experts. To ensure that the benefits of the very different perspectives were realized and to ensure ongoing merit review program quality PCORI has actively evaluated the process and outcomes of merit review. Funding applicants are asked about their experiences with the PCORI application process while reviewers are asked about their view of the process and how we can make it better.
We continue to use information from our ongoing surveys of PCORI applicants and reviewers to streamline the application process overall and guide us in fine-tuning our merit review process. We have used focus groups with reviewers to solicit in-depth feedback about ways to improve the training and resources PCORI provides. We have analyzed reviewer critiques in depth to understand how different types of reviewers view the applications. We analyze review scores to understand the role each of the PCORI merit review criteria plays in evaluation of funding applications and the impact of including patients and other stakeholders in review of health research funding applications. All this work has led to changes intended to improve the process and ensure that the most promising research is funded.
Publications
- Researchers, Patients, and Stakeholders Evaluating Comparative Effectiveness Research: A Mixed-Methods Study of the PCORI Reviewer Experience
LP Forsythe, LB Frank, R Hemphill, et al. Value in Health. (June 2018) - Unique Review Criteria and Patient and Stakeholder Reviewers: Analysis of PCORI’s Approach to Research Funding
LP Forsythe, LB Frank, TA Tafari et al. Value in Health. (June 2018) - Stakeholder-Driven Comparative Effectiveness Research: An Update From PCORI
JV Selby, LP Forsythe, HC Sox. Journal of the American Medical Association. (November 2015) - Engaging Patients and Stakeholders in Research Proposal Review: The Patient-Centered Outcomes Research Institute
RL Fleurance, LP Forsythe, M Lauer, et al. Annals of Internal Medicine. (2014)
Presentations
- Scientist, Patient, and Stakeholder Roles in Research Application Review
Presentation at Eighth International Congress on Peer Review and Biomedical Publication (September 2017) - Follow-up Analysis of Letters of Intent (LOIs) on Rare Diseases Presented to the May 2015 Rare Disease Advisory Panel
- Analysis of Rare Diseases Applications Presented to the Board of Governors - May 2015
- Methods Consultation Panel for Pragmatic Clinical Studies: Evaluation and Recommendations Presented to the May 2015 Clinical Trials Advisory Panel
- Presentation on the Engagement Resources from the Applicant and Reviewer Survey at the January 2015 Advisory Panel on Patient Engagement Meeting
- Analysis of PCORI Review of Application on Rare Diseases Presented to the January 2015 Advisory Panel on Rare Disease
- Results presented to Board from a trend-over-time analysis of the Merit Reviewer Survey - September 2014
- Engaging Patients and Stakeholders in Scientific Peer Review of Research Applications: Lessons From PCORI’s First Cycle of Funding
Presentation at Seventh International Congress on Peer Review and Biomedical Publication (September 2013)
What's Happening at PCORI?
The Patient-Centered Outcomes Research Institute sends weekly emails about opportunities to apply for funding, newly funded research studies and engagement projects, results of our funded research, webinars, and other new information posted on our site.
Image
