At PCORI, we’re serious about engaging patients and other healthcare stakeholders in the research process from start to finish. That includes the merit review process we use to identify the high-quality, patient-centered comparative clinical effectiveness research projects we support.

Unlike most funders, we include patients, clinicians, and other members of the healthcare community, along with scientists, in selecting the scientifically rigorous research projects that we expect to generate trustworthy information to inform practice and improve outcomes. The perspectives of a wide range of reviewers help us choose projects that are important, practical, and relevant to the audiences who will use the results of our work.

As you can imagine, setting up this system and ensuring that it keeps running smoothly is no small task. It means ensuring that we have a comprehensive, well-organized, and steadily updated database of potential participants to allow us to quickly identify individuals to be part of our merit review panels.

We greatly appreciate the commitment of all of those across the healthcare community who have agreed to serve as reviewers since we began asking for volunteers in 2012. And we’re very pleased to have about 4,000 potential reviewers in our database at the moment. But we’re looking for your help in building up our ranks of reviewers in critical stakeholder categories.

Maintaining our commitment to broad stakeholder balance

PCORI merit review officers assign reviewers to assess research proposals on the basis of their expertise, including health experiences, and stakeholder sector. In doing so, we always look to maintain the balance on each review panel to which we’ve committed from the start—scientists hold half the seats, patients fill a quarter of the positions, and other healthcare stakeholders make up another quarter.

A recent analysis of our database identified the challenge we face in maintaining that commitment—even though we have a substantial pool of volunteers whom we’ve not yet assigned to specific panels, we need to add potential reviewers from specific stakeholder groups to be sure our panels remain balanced and diverse.

For example, we have ample representation from scientists and clinicians. But we need more patients (including caregivers and patient advocates), who now represent only 12 percent of our reviewer pool. And while we have substantial representation from “other stakeholders” as a general category, we need more volunteers from some important audiences within that broad community—including insurers, employers, industry, policy makers, hospitals and health system workers, and training institutions. So we plan to focus our recruitment efforts on these groups and work to communicate the value they add and gain through their participation in the merit review process.

What we’re doing

We’ve started a wide-ranging effort to reach out to relevant groups and organizations representing the categories of stakeholders we want to see more involved in our review process, asking their help in spreading the word and encouraging applications. In addition, we will continue to recruit reviewers with expertise in the specific areas we expect to fund.

We’re also taking steps to make it easier for those who are interested to work with us. For example, we have updated our reviewer application and shortened it by half. It now takes only 10 minutes to complete. We have also accelerated our reviewer evaluation process. Each month, we’ll consider submitted applications and review personal statements, healthcare experience or role, and professional background, where relevant. We will then let applicants know whether they have been moved into our reviewer database, and we will invite them, when appropriate, to staff our merit review panels.

To further encourage patients to apply to serve as reviewers, we have streamlined the training, revised our Methodology 101 guidebook, and reduced the number of research proposals that each reviewer must evaluate. Our mentor program provides our patient reviewers with additional support from veteran reviewers and helps patient reviewers clearly communicate their perspective on their assigned proposals and feel comfortable voicing their thoughts during the in-person discussion.

Appreciating Reviewers’ Input

While we seek more volunteers from particular stakeholder groups, we want to be sure to thank the many people who have already joined our reviewer pool, including those who have already served as reviewers. Each provides an individual, in-depth evaluation of four to six proposals relevant to the reviewer’s experience and expertise. Then, the reviewers come together, at our expense, for an in-person discussion of 20 to 25 proposals. It’s a substantial commitment, and we greatly value the effort.

We are especially grateful to the 22 percent of those in our database who have participated in at least one review panel and are willing to serve again; some have come back to us and served repeatedly. All reviewers receive training on our goals and review criteria, as well as information relevant to the current funding announcements. In addition, they work closely with our merit review staff and, for the nonscientist reviewers, also with mentors. Returning reviewers enable us to especially benefit from both our investment in training and the skills that reviewers obtain through their experience.

Most reviewers tell us that they find the experience to be rewarding, and 93 percent of reviewers say they’d review for us again. They enjoy the opportunity to sit at the table to discuss the importance, feasibility, and potential impact of proposed projects and know they have made a valuable contribution.

Broad participation in our proposal review program helps ensure that PCORI-funded studies will answer the real-world health questions of patients and the healthcare community. We invite you to apply to become a PCORI reviewer.