Results Summary
What was the project about?
Research that focuses on what’s most important to patients can inform health decisions. Researchers use different methods to identify what’s most important to patients.
In this study, the research team compared two methods for identifying what’s most important to patients: one-on-one interviews and group concept mapping, or GCM. GCM is a three-round process that helps researchers get input from a group. In the first round, people brainstorm topics that are important to them. Next, people sort the topics into clusters based on similar ideas. Finally, researchers create a map to display and discuss the topics. Researchers can use the complete GCM process or the brainstorming round only.
The research team looked at one-on-one interviews versus GCM and compared the number of topics patients named and the amount of time and money required.
What did the research team do?
The research team did one-on-one interviews with 89 patients and three complete GCM processes with a total of 52 patients. All patients had type 1 or type 2 diabetes. The team asked patients what was most important to them when making decisions about managing diabetes. The team compared the number of topics mentioned in GCM versus in interviews. To look at the amount of time and money each method required, the team kept an activity log for each method. The log recorded costs and time required of patients and research staff for each task.
Of the patients, 72 percent were black, 20 percent were white, and 6 percent were another race. In addition, 8 percent were Hispanic. All sought care from one health system in Philadelphia.
Patients, patient advocates, and healthcare professionals gave input throughout the study.
What were the results?
Patients named 38 topics in GCM compared with 26 topics in interviews. They named 41 unique topics overall. Be healthy was the most common topic. Across 41 unique topics, the research team identified seven themes:
- Improve daily self-care.
- Improve long-term health.
- Learn about diabetes.
- Measure and achieve goals.
- Manage medicines.
- Manage diet.
- Best use medical or professional services.
Both GCM brainstorming alone and the complete GCM process required fewer research team hours (78 hours for GCM brainstorming, 104 hours for complete GCM) than interviews (295 hours in total). But GCM required more hours per patient (three hours for GCM brainstorming, eight hours for complete GCM) than interviews (one hour per interview). GCM brainstorming cost less and complete GCM cost more than interviews ($3,130 for interviews, $1,200 for GCM brainstorming, $5,000 for complete GCM).
What were the limits of the project?
All patients in the study had diabetes and received care at the same health system. Future research could include people with other health needs or people who receive care at other health systems.
How can people use the results?
Researchers can use the results when selecting methods to identify what’s most important to patients.
Professional Abstract
Background
Patient-important outcomes (PIOs) describe patient values and preferences in health care. Including PIOs in research may yield findings that are relevant to patients. Methods to identify PIOs may vary in efficiency and result in PIOs that differ in comprehensiveness.
Researchers can use individual interviews or group concept mapping (GCM) to elicit PIOs. Interviews identify information individuals may not want to discuss in a group setting. GCM is a three-phase process that generates group input. In GCM, participants brainstorm PIOs in response to a prompt, then sort and rate ideas along predefined dimensions, such as importance or feasibility. Finally, participants interpret and discuss a concept map of the categories and ideas. Researchers can use the complete GCM process or the brainstorming phase only.
Objective
To compare the comprehensiveness and efficiency of individual interviews and GCM for generating PIOs
Study Design
Design Element | Description |
---|---|
Design | Empirical analysis |
Data Sources and Data Sets | Transcripts from 89 interviews and PIOs generated in 3 iterations of GCM with 52 patients |
Analytic Approach | Qualitative analysis comparing individual interviews with the GCM brainstorming session alone and the complete GCM process |
Outcomes | Data comprehensiveness as defined by number of PIOs generated; efficiency of data collection and analysis as determined by patient time, research staff time, and total cost |
Methods
This study compared the comprehensiveness and efficiency of data collection and analysis using individual interviews versus GCM for eliciting PIOs related to chronic disease management.
Researchers identified PIOs using interviews with 89 patients and three GCM iterations with 52 patients. All patients had moderately to poorly controlled type 1 or type 2 diabetes and sought care from one health system in Philadelphia. Of these patients, 72% were black, 20% were white, and 6% were of another race. In addition, 8% were Hispanic.
To assess the comprehensiveness of PIOs, researchers consolidated PIOs across GCM iterations and across interviews and then compared the number of PIOs generated using each method. To assess efficiency, researchers kept an activity log that included cost and time required for each method.
Patients, patient advocates, and healthcare professionals provided feedback throughout the study.
Results
Researchers identified 26 PIOs from interviews and 38 from the three GCM iterations; the two methods together generated 41 unique PIOs. The most frequently identified PIO overall was being healthy. Among unique PIOs, researchers identified seven themes:
- Optimize daily self-care.
- Optimize long-term health.
- Learn about diabetes.
- Achieve measurable goals.
- Manage medications.
- Manage diet.
- Best utilize medical or professional services.
Both GCM brainstorming alone and the complete GCM process required fewer research team hours (78 per GCM brainstorming, 104 per complete GCM) compared with interviews (295 hours). However, the GCM processes required more hours per participant (three hours for GCM brainstorming, eight for complete GCM) compared with interviews (one hour per interview). Compared with the total cost of interviews, GCM brainstorming sessions cost less and the complete GCM process cost more ($3,130 for interviews, $1,200 for GCM brainstorming, $5,000 for complete GCM).
Limitations
All patients in the study had diabetes and sought care within the same health system. Findings may differ for patients with other health needs or in other health systems.
Conclusions and Relevance
GCM generated more PIOs than interviews. GCM required less time for research staff but more time for patients. Results may help researchers considering different ways to elicit PIOs.
Future Research Needs
Future research could further examine the PIOs developed using these processes.
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers suggested that PCORI Methodology Standards require that the group concept mapping (GCM) approach used in this study be treated as a complex intervention, which would necessitate the inclusion of a specific table describing the approach. The researchers did not complete the suggested table. They reviewed the standards for a complex intervention and concluded that the GCM approach did not fit the construct of a complex intervention or evaluation. They noted that GCM is not an intervention and does not have comparator groups.
- The reviewers criticized the researchers’ conclusions that GCM brainstorming was more comprehensive and more efficient than patient interviews, pointing out that interviews provided rich contextual information that could not be captured in concept mapping. The researchers agreed with this aspect of individual interviews. However, as their focus in this study was to understand which method garnered more patient-important outcomes, the depth of information from individual interviews was not a factor in their results. The researchers revised the report to clarify that GCM brainstorming surpassed individual interviews when the broader context was not relevant to the research question.
- The reviewers noted that participants in the GCM brainstorming sessions likely varied in their ability to move individual ideas into higher domains. The reviewers ask how the researchers dealt with the potential for bias in the GCM brainstorming sessions if some participants were much more capable of performing the higher-level analyses. They also worried that the cognitive limitations of some patients made GCM a less appropriate approach and the results obtained from the study less generalizable. The researchers agreed that individuals varied in their ability to engage in the GCM sessions and agreed that the challenges that participants had with GCM limited the generalizability of the study. The researchers noted in their report that they anticipated this issue by having members of their patient and stakeholder advisory board facilitate the sessions.
- The reviewers asked why the researchers excluded people with complications, like amputations and blindness, from participating. The researchers explained that their goal was to compare two different methods for identifying outcomes important to patients, GCM versus individual interviews. They believed that people who had already experienced serious irreversible complications from diabetes might have a perspective that differed too markedly for comparison from those who had not. The researchers added an explanation of this decision in the sampling strategy section of the methods.