Results Summary

What was the project about?

Data collected from interviews and group discussions, called qualitative data, can help researchers understand people’s experiences, values, and cultures. But large amounts of qualitative data can be hard to show in a way that’s easy for people to understand.

In this study, the research team created charts called ethnoarrays. These charts use color coding to show individual stories and overall patterns in qualitative data. The team wanted to learn whether ethnoarrays were useful and easy to understand.

What did the research team do?

The research team studied ethnoarrays in two ways. First the research team asked an advisory board for feedback on mock-up ethnoarrays. The board included social scientists, patients, caregivers, doctors, and nurses.

Then the research team used ethnoarrays to analyze data from two sets of patient interviews. One set included 96 patients with late-stage cancers; the other set included 36 patients newly diagnosed with breast cancer. The team noted concepts and actions related to decision making, called themes, for each patient. The ethnoarrays listed patients in the rows and themes in the columns. The color coding showed whether a patient mentioned a given theme. The ethnoarrays grouped patients by similarities in themes, showing patterns in the data.

What were the results?

Advisory Board Feedback. The board thought the strengths of ethnoarrays were

  • Displaying many concepts
  • Showing individual data and overall patterns in the same chart
  • Not requiring special knowledge to understand

The board also said that ethnoarrays were too complex to use during clinic visits. Instead, ethnoarrays would be most useful to help guide researchers in data analysis.

Decision-Making Ethnoarrays. The ethnoarrays showed patterns in how the two groups of patients made cancer treatment decisions. The ethnoarrays grouped patients based on similarities in how patients looked for and discussed information about their illnesses and treatment options.

The late-stage cancer ethnoarray showed two groups of patients. In the first group, patients had less healthcare knowledge and relied on doctors to make treatment decisions. In the second group, patients had healthcare connections and were active in decision making.

The breast cancer ethnoarray had the following groups:

  • Group A patients focused on whether the surgery would remove the entire tumor or whether they would need more surgeries.
  • Group B patients had the fewest decision-making themes. They also didn’t often talk about their worries.
  • Groups C1 and C2 patients had the most decision-making themes, including the role of their partner, doctor recommendations, and desire to change breast appearance. These two groups differed in how much care they wanted to receive.

What were the limits of the project?

Although ethnoarrays can help people understand information in a study, patterns in ethnoarrays only apply to the people in that study. These patterns may not be true for other groups of people.

Future research could develop software for making ethnoarrays.

How can people use the results?

Researchers can consider using ethnoarrays to look for patterns in large qualitative data sets.

Final Research Report

View this project's final research report.

Stories and Videos

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:

  • The reviewers said it would have been helpful to see more information on how different stakeholders responded to the ethnoarray tool. In particular, it was not clear from the report that there was sufficient feedback from journal readers and other nonscientists to justify the superiority and usefulness of the ethnoarray. The researchers agreed with the reviewers’ points and revised the report to clarify the evaluative tasks and views of different stakeholder groups while also describing the limitations of the stakeholder findings.
  • Some reviewers questioned how novel or useful the ethnoarray technology would be for describing qualitative research results, given an earlier history of anthropologists using visual displays, and what the reviewers saw as considerable overlap between ethnoarrays and existing qualitative software programs. The researchers maintained that in their review of available software and research methods, the ethnoarray would still be considered a novel tool that is more customizable, scalable, and capable of higher-level computational analysis than existing tools. The researchers indicated that feedback they received from their stakeholder advisory board and their colleagues supported these assertions.
  • The reviewers noted that the ethnoarray comes after a 50-year history of visual displays developed for qualitative data, but that the researchers cited only some of the previous work in their report. The researchers revised the report to credit the earlier works, incorporating the suggested references in the background section and describing how improvements in computational power allowed them to build on earlier attempts to visualize qualitative data.

Conflict of Interest Disclosures

Project Information

Daniel Dohan, PhD
University of California, San Francisco
$1,461,719
10.25302/06.2020.ME.140922996
Visual Displays of Qualitative Data to Advance PCOR

Key Dates

April 2015
April 2020
2015
2019

Study Registration Information

Tags

Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: April 29, 2024