Results Summary
What was the research about?
Organizations that fund research often seek input about what to study. But patients, health professionals, and researchers may have different ideas about what’s important.
In this project, the study team created a method of setting research agendas called Stakeholder Engagement in quEstion Development, or the SEED Method. These agendas help identify research topics and questions that are important to study. The team tested the SEED Method at two sites in Virginia. The team wanted to learn if the method developed research agendas that reflected differing views and if people were satisfied with the process.
Using the SEED Method, the study team gathered input from three sets of people at each site:
- Research Team. Community members and staff from local colleges who worked with the study team to lead the project
- Topic Groups. Three groups of people who developed research questions. Each group had people with a type of viewpoint, such as patients, health professionals, or research funders. The groups worked independently with the Research Team.
- Stakeholder Consultants. Patients and health professionals with knowledge of the research topic who took part in interviews and group discussions
What were the results?
Developing research agendas. At both sites, the SEED Method resulted in research agendas that reflected the views of patients, health professionals, and research funders.
- At the first site, the research topic was managing diabetes and high blood pressure. The Topic Groups discussed 91 research questions and chose 18 questions as most important.
- At the second site, the research topic was lung cancer. The Topic Groups discussed 77 research questions and chose 12.
Satisfaction with the process. People at both sites reported feeling prepared to help set research agendas. They were also satisfied with the process.
What did the study team do?
At each site, the SEED Method included six steps:
- The Research Team chose the research topic and selected people for the Topic Groups.
- The Research Team interviewed and had group discussions with the Stakeholder Consultants. Stakeholder Consultants talked about the factors that affect the research topic and ways to improve health. The Topic Groups had three meetings with the Research Team: two to plan interviews and group discussions and one to review results.
- Each Topic Group built visual displays to show the relationships between the factors that affect the research topic.
- Each Topic Group reviewed all the visual displays and developed research questions.
- Each Topic Group reviewed, discussed, and finalized the research questions. The Topic Groups then voted on which questions to include in the research agendas.
- The Research Team searched for studies about each research question, identified gaps in research, and made suggestions for future studies. Research Team members also made plans for sharing the research agendas.
Research Team and Topic Group members completed surveys about how well the method worked and how satisfied they were with the process.
What were the limits of the study?
The research agendas reflected the views of the people who took part in the study. The agendas may differ for other people. The SEED Method required a lot of time, expertise, and resources and may not be possible for all research funders.
Future research could test the method in different settings or adapt it to use fewer resources.
How can people use the results?
Research funders can use these results when considering how to get input from patients and health professionals for research agendas.
Professional Abstract
Objective
To conduct a demonstration of a patient and stakeholder engagement method that develops research agendas for patient-centered outcomes research studies
Study Design
Design Elements | Description |
---|---|
Design | Engagement methods demonstration |
Data Sources and Data Sets | Focus groups, interviews, meeting discussions, conceptual models, literature reviews |
Analytic Approach | Descriptive statistics, bivariate tests and correlations, qualitative analyses |
Group readiness; satisfaction with the engagement process, facilitated group exercises, group dynamics, and decision-making processes |
In this demonstration study, researchers created a method for engaging patients and other stakeholders in conceptualizing and developing research agendas. The demonstration took place at two sites in Virginia. The Stakeholder Engagement in quEstion Development (SEED) Method engaged three sets of stakeholders at each site:
- Research Team. A group of community members and staff from local universities who worked with the study researchers to lead the demonstration and finalize the research agendas.
- Topic Groups. Three groups of stakeholders who worked to conceptualize and develop research questions. Each group represented one stakeholder perspective—such as patient, health professional, or research funder—and worked independently with the Research Team.
- Stakeholder Consultants. People with knowledge of the research topic, such as patients and healthcare providers, who participated in focus groups and interviews.
The SEED Method has six steps:
- Identify and Engage. The Research Team identified the research topic and Topic Group members for each site.
- Consult. The Research Team conducted focus groups and interviews with Stakeholder Consultants to ask about beliefs, perceptions, and behaviors about the research topic and barriers and facilitators to better health outcomes. Each Topic Group had three meetings with the Research Team: two to plan interviews and group discussions with the Stakeholder Consultants and one to review results.
- Conceptualize. Each of the three Topic Groups created conceptual models to illustrate their perceptions of the relationships between factors affecting health outcomes.
- Generate Questions. Each Topic Group reviewed the three sets of conceptual models for their site and participated in a facilitated exercise to develop research questions.
- Prioritize Questions. Each Topic Group reviewed, discussed, and finalized the research questions and then voted to select the top questions for the research agenda.
- Finalize and Disseminate Questions. For prioritized questions, the Research Team reviewed the literature, identified research gaps, and made recommendations for future studies. Then the Research Team developed dissemination plans for each research agenda.
Research Team and Topic Group members completed assessments about engagement and the effectiveness of the SEED Method throughout the study.
Results
Implementing the SEED Method. The SEED Method facilitated the generation of conceptual models and research agendas that reflected the priorities of participating stakeholders.
- At one site, the research topic was managing diabetes and high blood pressure. The Topic Groups developed and discussed 91 research questions, prioritizing 18.
- At the other site, the research topic was lung cancer outcomes. The Topic Groups developed and discussed 77 research questions, prioritizing 12.
Assessments. At both sites, the Research Team and Topic Group members had high rates of group readiness for engagement and satisfaction with their overall experiences, facilitated group exercises, group dynamics, and decision-making processes.
Limitations
The research agendas reflected priorities of the Topic Group members and may not generalize to other populations. The projects required extensive time, expertise, and resources and may not be feasible for all research funders.
Conclusions and Relevance
Research funders may consider using the SEED Method to get input from stakeholders for research agendas.
Future Research Needs
Future research could test the SEED Method in different settings or adapt the process to use fewer resources.
Final Research Report
View this project's final research report.
Engagement Resources
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented, and the researchers made changes or provided responses. The comments and responses included the following:
- Reviewers asked for greater clarity on the purpose of and relationships among the different types of patient and other stakeholder groups that were part of the engagement process for this study. The researchers added details in the text and tables about the different types of groups, how they functioned, and how to prioritize their contributions.
- Reviewers asked how applicable the study’s approach could be expected to be in different environments. The researchers said they designed the method and the toolkit that they have made available to be scalable and adaptable. However, the method design is currently face-to-face which may limit its application, for example, in remote settings. Also, the tools are only available in English at this point.
- Reviewers expressed concern regarding limited diversity in the research team, which could limit the way the researchers approached studying health outcomes. The researchers responded that the matrices they developed are customizable so that any research team could choose which subgroups of stakeholders to include based on their own communities.
- Reviewers considered it a limitation of the study that a single person conducted coding and interpretation of qualitative evaluation data. Reviewers also asked for a clearer description of the qualitative methodology used. The researchers added a detailed description of the qualitative coding process, which they noted was only used for evaluation data. Qualitative data from other parts of the project were not coded formally.