"The Beginning of the Beginning" — Building a Community of Reviewers to Advance Patient-Centered Outcomes Research

February 11, 2013 by PCORI

It looked like a traditional scientific gathering – about 150 people deciding which of about 100 research proposals to recommend for funding by PCORI in our first round of patient-centered comparative effectiveness awards. But it was hardly business as usual.

A room of meeting participants pay attentions while one woman raises her hand

Sitting around the tables were dozens of scientific reviewers – and a similar number of patients and other stakeholders from across the healthcare community. That latter group is not one you’d expect to see advising a major research funder on how to spend tens of millions of dollars. But for PCORI, having them there was just another example of our commitment to “research done differently.”

Our focus on patient-centeredness and stakeholder engagement means this: we not only expect those we fund to do research differently than they might have before, but that we also have a different approach to funding. We believe that only by including diverse healthcare stakeholders in our process in a meaningful way can we fund research that will produce information that better meets the needs of patients and those who care for them. We also think this type of research ultimately will improve the quality of care.

Clearly, vetting proposals for scientific rigor is essential. But we also know that patients are experts in their conditions or diseases, while other healthcare stakeholders – clinicians, caregivers, payers, industry – provide additional perspectives key to understanding how research findings will be applied on a real-world basis.So we established a program to seek and train patients and other stakeholders to participate in our reviews. This program lets us gauge both technical merit and factors that assess the impactof the work we support. That training, which began last October and is ongoing, covers the criteria we use to evaluate proposals, as well as scoring and review panel procedures. Potential reviewers receive comprehensive explanations of each review criterion and how it should be applied.

With a clear understanding of the principles and logistics behind the review process, participants can focus on their expertise.After they were trained, our first class of patient and other stakeholder reviewers returned to Washington. They joined their scientist colleagues for a full day of in-person panels to assess the funding proposals; each panel consisted of one patient and one other stakeholder reviewer for every two scientists.

Screenshot of the You Tube video of Debra Madden

“The difference between an ‘other stakeholder’ and a patient stakeholder doesn’t even matter,” said Debra Madden, who works for a health professional association and a patient advocacy group. “We are all there for the same reason. The scientists are there for the same reason. There is such a level of respect in that room toward the expertise that every single person brings to this.”

The applications the group considered all had passed an initial round of technical review. Then they were evaluated mainly through three merit criteria: patient-centeredness; innovation and potential for improvement through research; and the research team and environment. In assigning these scores, we asked reviewers to consider whether a project had the potential to change clinical practice or patient behavior to improve outcomes and health, and how quickly study results could be disseminated and applied.Scientists, patients and other stakeholders all had the same amount of time to explain how they rated each proposal — an equality that had been emphasized in training.

“Initially, as a patient stakeholder, I was a little intimidated by being side-by-side with scientific stakeholders, but I also felt like my input was valuable to the panel,” said Crystal Brown-Tatum, another patient advocacy group representative. “Everyone wanted to hear my thoughts and they appreciated what the patients were bringing to the panel, because our experiences are obviously so different than scientists’ … I learned things from the other stakeholders and I know they learned things from me.”

In the long-term, we believe our approach to the review process will yield a strong portfolio of patient-centered research that leads to better-informed healthcare decision making. But its immediate impact is to provide a window into our vision for research.We continue to be impressed by the passion and insight of those who participate in our work; you can get a sense of this enthusiasm in this short video. And we’re excited to work with new groups of patients and other stakeholders who are eager to serve as reviewers and will help us pursue our mission of improving health and health care through better informed decision making. If you are interested in applying, you can find out more here.

“I think the patients bring a really unique experience because of their own experience,” said patient advocate Vernal Branch. “And you can also change the minds of the scientists, because they’re thinking totally clinically or scientifically where it really may not make an impact. We can see where the evidence is and if it’s going to really make a difference in people’s lives.”

Reviwer Button button

Perhaps most gratifying is how many of the patients and other stakeholders who went through our initial training and reviews are serving as mentors as we train others. They, like us and the many scientists we work with, see the promise this approach offers to help build a new patient-centered research community that will benefit everyone.

“I was very thrilled to see how empowered the stakeholder groups … and the patient advocacy groups are,” said Chris Dezii, a health professional who works for the pharmaceutical industry. “We are still in the beginning of the beginning here. But just [having] the interest to include the patient’s voice is very important.”

Susan Hildebrandt, MA, is PCORI’s Director of Stakeholder Engagement.

Gregory Martin is PCORI’s Deputy Director of Stakeholder Engagement.

Sue Sheridan, MBA, MIM, is PCORI’s Director of Patient Engagement.

Martin Duenas, MPA, is PCORI’s Director of Contracts Management.

2 Responses to“"The Beginning of the Beginning" — Building a Community of Reviewers to Advance Patient-Centered Outcomes Research”

  1. Lena J. Gamble

    I am a neuroscience researcher at the NIH who recently graduated with my MD/.PhD (University of Alabama at Birmingham 2012). I am interested in learning more about becoming a Picori reviewer. Please send more information.


Leave a Reply