Results Summary
What was the project about?
Data collected from interviews and group discussions, called qualitative data, can help researchers understand people’s experiences, values, and cultures. But large amounts of qualitative data can be hard to show in a way that’s easy for people to understand.
In this study, the research team created charts called ethnoarrays. These charts use color coding to show individual stories and overall patterns in qualitative data. The team wanted to learn whether ethnoarrays were useful and easy to understand.
What did the research team do?
The research team studied ethnoarrays in two ways. First the research team asked an advisory board for feedback on mock-up ethnoarrays. The board included social scientists, patients, caregivers, doctors, and nurses.
Then the research team used ethnoarrays to analyze data from two sets of patient interviews. One set included 96 patients with late-stage cancers; the other set included 36 patients newly diagnosed with breast cancer. The team noted concepts and actions related to decision making, called themes, for each patient. The ethnoarrays listed patients in the rows and themes in the columns. The color coding showed whether a patient mentioned a given theme. The ethnoarrays grouped patients by similarities in themes, showing patterns in the data.
What were the results?
Advisory Board Feedback. The board thought the strengths of ethnoarrays were
- Displaying many concepts
- Showing individual data and overall patterns in the same chart
- Not requiring special knowledge to understand
The board also said that ethnoarrays were too complex to use during clinic visits. Instead, ethnoarrays would be most useful to help guide researchers in data analysis.
Decision-Making Ethnoarrays. The ethnoarrays showed patterns in how the two groups of patients made cancer treatment decisions. The ethnoarrays grouped patients based on similarities in how patients looked for and discussed information about their illnesses and treatment options.
The late-stage cancer ethnoarray showed two groups of patients. In the first group, patients had less healthcare knowledge and relied on doctors to make treatment decisions. In the second group, patients had healthcare connections and were active in decision making.
The breast cancer ethnoarray had the following groups:
- Group A patients focused on whether the surgery would remove the entire tumor or whether they would need more surgeries.
- Group B patients had the fewest decision-making themes. They also didn’t often talk about their worries.
- Groups C1 and C2 patients had the most decision-making themes, including the role of their partner, doctor recommendations, and desire to change breast appearance. These two groups differed in how much care they wanted to receive.
What were the limits of the project?
Although ethnoarrays can help people understand information in a study, patterns in ethnoarrays only apply to the people in that study. These patterns may not be true for other groups of people.
Future research could develop software for making ethnoarrays.
How can people use the results?
Researchers can consider using ethnoarrays to look for patterns in large qualitative data sets.
Professional Abstract
Background
Qualitative data, such as information collected from interviews and group discussions, are useful for understanding people’s experiences, values, and cultures. Patient-centered outcomes research could benefit from additional methods of analyzing and reporting findings from large qualitative data sets.
Objective
To develop a new qualitative data display, called an ethnoarray
Study Design
| Design Elements | Description |
|---|---|
| Design | Empirical analysis |
| Data Sources and Data Sets | Interview transcripts and observational data related to cancer treatment decision making among 96 patients with late-stage cancer at 2 Comprehensive Cancer Centers and 36 newly diagnosed patients with breast cancer at a private oncology practice |
| Analytic Approach | Thematic coding of secondary qualitative data and data-display development |
| Outcomes | Usefulness and interpretability of the ethnoarrays, factors that influence patient approaches to cancer treatment decision making |
Methods [or Methods and Results]
In this study, the research team developed a new type of qualitative data display called an ethnoarray. Ethnoarrays are charts that use color coding to reveal patterns within large qualitative data sets while preserving subject-centered narratives.
The research team used two methods to evaluate the usefulness and interpretability of ethnoarrays. First the team asked an advisory board of seven social scientists, six patients, two caregivers, and five clinicians to provide feedback on prototype ethnoarrays.
Then the research team used ethnoarrays to conduct a secondary data analysis of two sets of patient interviews: one set from 96 patients with late-stage cancer and another set from 36 patients who were newly diagnosed with breast cancer. The team identified concepts and behaviors, or themes, related to cancer treatment decision making within each of the two data sets. The ethnoarrays listed patients in the rows and themes in the columns. The color coding indicated the presence or absence of themes within an observation. The team developed statistical models that clustered patients together based on similarities in themes, which revealed patterns in the data.
Results
Advisory Board Feedback. The board thought the strengths of ethnoarrays were
- Displaying many concepts
- Showing individual-level data and group-level patterns in the same visual display
- Presenting complex information without requiring specialized knowledge to understand
However, the board noted that ethnoarrays were too complex to use during patient–clinician encounters and would be most appropriately used by researchers to analyze qualitative data.
Decision-Making Ethnoarrays. The ethnoarrays showed distinct groups based on similarities in patients’ approaches to learning about and discussing their illnesses and treatment options.
The late-stage cancer ethnoarray clustered the patients into two groups. In one group, members had less medical knowledge and relied on their clinicians to make treatment decisions. In the second group, members had deeper medical knowledge and greater access to medical expertise and took an active decision-making role.
The breast cancer ethnoarray clustered patients into the following groups:
- Group A patients had the greatest focus on whether the surgery would remove the entire tumor or whether they would need more surgeries.
- Group B patients had the lowest density of decision-making factors—notably, a lack of discussion about worry.
- Groups C1 and C2 patients had the highest density of decision-making factors, including the role of significant others, clinician recommendations, and desire to change breast appearance. Groups C1 and C2 differed in the amount of treatment they wanted to receive.
Limitations
Ethnoarrays, like most products of qualitative research, describe only the experiences of a given study sample and may not be suitable for drawing generalizable conclusions.
Conclusions and Relevance
Ethnoarrays may help researchers identify and interpret patterns within large qualitative data sets.
Future Research Needs
Future research could develop software for producing ethnoarrays.
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Stories and Videos
Videos
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers said it would have been helpful to see more information on how different stakeholders responded to the ethnoarray tool. In particular, it was not clear from the report that there was sufficient feedback from journal readers and other nonscientists to justify the superiority and usefulness of the ethnoarray. The researchers agreed with the reviewers’ points and revised the report to clarify the evaluative tasks and views of different stakeholder groups while also describing the limitations of the stakeholder findings.
- Some reviewers questioned how novel or useful the ethnoarray technology would be for describing qualitative research results, given an earlier history of anthropologists using visual displays, and what the reviewers saw as considerable overlap between ethnoarrays and existing qualitative software programs. The researchers maintained that in their review of available software and research methods, the ethnoarray would still be considered a novel tool that is more customizable, scalable, and capable of higher-level computational analysis than existing tools. The researchers indicated that feedback they received from their stakeholder advisory board and their colleagues supported these assertions.
- The reviewers noted that the ethnoarray comes after a 50-year history of visual displays developed for qualitative data, but that the researchers cited only some of the previous work in their report. The researchers revised the report to credit the earlier works, incorporating the suggested references in the background section and describing how improvements in computational power allowed them to build on earlier attempts to visualize qualitative data.