Results Summary

What was the project about?

In stakeholder-engaged research, patients, families, and other community members are part of a research team. As part of the team, community members can influence the design of a study and other parts of the research process so that the results are useful for their communities.

How research teams involve community members varies across studies. Some involve people in just one part or in a limited way, while others involve people much more. Having a consistent way to assess how involved community members are in a research project may help improve the process and the impact of stakeholder-engaged research.

In this study, the research team created a survey that researchers can use to assess how much community members are involved in research teams.

What did the research team do?

The research team used the Delphi method to create a survey with input from a panel of community members. The Delphi method helps panels share feedback and reach agreement through rounds of discussion and voting.

The panel included 19 patients, advocacy organization members, and researchers. Of the panelists, 63 percent were African American, 32 percent were White, and 5 percent were multiracial. The median age was 55, and 90 percent were women.

The research team started with a list of 11 best practices on how to engage community members in research. The team created survey questions about each practice. The team also developed definitions and scenarios about how research teams involved community members.

In each round of discussion, panelists gave feedback on the materials and decided to keep, change, or remove each survey question based on its importance. The panel reached consensus on the survey questions to include in the final survey.

Patients, caregivers, healthcare providers, and researchers helped design and conduct the study.

What were the results?

The final survey had 32 questions organized by eight best practices. These practices included:

  • Focus on community views of health.
  • Include community input.
  • Make sure partnerships between researchers and the community last.
  • Help community members learn and benefit from the research.
  • Make sure research teams build on communities’ strengths.
  • Make sure partnerships involve working together and sharing power.
  • Have community members share results of studies.
  • Build and keep trust.

Panelists and the research team also created a way to score studies on community engagement. The scoring can show a range of engagement from simply getting input from community members, to having community members be partners in making decisions about the research.

The research team also created a shorter version of the survey that takes less time to complete.

What were the limits of the project?

The research team tested the survey in English. It may not reflect the experiences of people who speak other languages.

Future research could translate the survey and test it with people who speak other languages.

How can people use the results?

Researchers can use the survey to assess how involved community members are in research studies.

Final Research Report

View this project's final research report.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:

  • The reviewers asked about the report’s references to item response theory, stating that it was not clear in the text why this work was done and how it contributed to instrument development. The researchers explained that they did not use item response theory in the traditional sense, instead applying the technique to help determine which items might be potential candidates for inclusion or exclusion in the condensed version of their final Research Engagement Survey Tool (REST).
  • The reviewers questioned the development of tailored versions of REST for different demographic groups, pointing out that individual scale items showing differential responding by demographic group should probably be excluded instead, to ensure the consistency of the instrument. The researchers agreed that having different groups of people respond to different items on the same scale would not be helpful for scale development. However, they explained that they tested the tailored versions of REST to determine the feasibility of using such versions in the future. The researchers noted that future investigators might want to use a tailored subset of REST items for specific demographic groups.
  • The reviewers requested more statistical information regarding the factor analysis described in the report. The researchers explained that they did not make any inferences from the factor analysis or create factors or subscales from the results, instead using the factor analysis descriptively as one component of a triangulation approach to identify which REST items i would be best to use in a condensed version of the measure.

Conflict of Interest Disclosures

Project Information

Vetta Sanders Thompson, PhD, MA^
Washington University School of Medicine
Developing and Validating Quantitative Measures to Assess Community Engagement in Research: Addressing the Measurement Challenge

Key Dates

July 2016
October 2022

Study Registration Information

^Melody S. Goodman, PhD, MPH, was the principal investigator when this project was initially awarded.


Has Results
Award Type
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: March 14, 2024