Results Summary
What was the research about?
To learn about people’s experiences, researchers often use one-on-one interviews and group interviews, called focus groups. Researchers can do interviews and focus groups in person or online.
In this study, the research team wanted to learn if people shared more or different information in person versus online. The team did interviews and focus groups with women about safety during pregnancy and compared the information collected in person or online. To collect information online, the team used either online video, chat, a message board, or email. The team then compared differences in the average number of words and what women discussed across the methods.
What were the results?
The average number of words was similar for interviews and focus groups in person and using online video. In-person interviews had a higher average number of words than interviews using online chat or email. In-person focus groups had a higher average number of words than focus groups using online chat or message boards. Focus groups using online video had a higher average number of words than those using online chat.
In-person and online interviews and focus groups resulted in a similar number of topics discussed. The amount of sensitive information women shared was about the same in person and online. Women were more likely to disagree in focus groups using online chat than in in-person groups or groups using online video.
Who was in the study?
The study included 171 women in North Carolina. Of these, 82 percent were white, 4 percent were black, and 7 percent were of another race. In addition, 6 percent were Hispanic. The median age was 32, and 77 percent had a bachelor’s degree or higher. Women in the study had been pregnant in the past three years and hoped to become pregnant in the next three years.
What did the research team do?
The research team assigned women, by chance, to either an interview or a focus group. Next, the team assigned women to one of four groups:
- In person. The team held interviews and focus groups in an office conference room.
- Online video. The team created a private online video chat room. The team used webcams for video and a telephone conference call line for audio.
- Online chat. Women answered questions in real time using a private online text chat room.
- Online message board or email. For focus groups, women answered questions on an online discussion board for 7 to 10 days. For interviews, women answered questions throughout 10 days using email.
In focus groups and interviews, the research team asked women about the Zika virus and making decisions about safety during pregnancy.
What were the limits of the study?
The study included women with home internet access. Most women were white and had high education levels. Findings may differ for people from other backgrounds or without home internet access.
Future research could include people from different backgrounds.
How can people use the results?
Researchers can consider the results when choosing ways to ask people about their experiences.
Professional Abstract
Objective
To compare the volume and content of data from in-depth interviews (IDIs) and focus groups (FGs) conducted in person versus online
Study Design
| Design Elements | Description |
|---|---|
| Design | Empirical analysis |
| Data Sources and Data Sets | 171 women randomized to a data collection method (IDI or FG), then assigned to a data collection modality (in person, online video, online chat, email or online message board) |
| Analytic |
Quantitative: one-way ANOVA, chi-square tests Qualitative: inductive and a priori thematic analyses |
| Outcomes | Average word count, frequency of thematic codes applied, presence of sensitive disclosures and dissenting opinions |
IDIs and FGs provide data on people’s thoughts and experiences in their own words. Few studies have examined how data might differ between IDIs and FGs conducted in person versus online. Data may vary in volume, measured by word count, or content, measured by the frequency of thematic codes applied.
In this study, researchers compared the volume and content of data collected through IDIs or FGs conducted in person and through three online modalities. Researchers randomly assigned women to either IDIs or FGs. Next, researchers assigned women to one of four groups:
- In person. Researchers conducted IDIs and FGs in an office conference room.
- Online video. Women used their personal computers with webcams to sign in to a private online video chat room and used a telephone conference line for audio.
- Online chat. Researchers typed questions, and women responded in a private online text chat room in real time.
- Online message board or email. For IDIs, the interviewer emailed women three to five questions at a time during a 10-day period. For FGs, the moderator posted three to five questions on an online discussion board each day for 7 to 10 days. Women responded at their convenience.
Women in the IDIs and FGs discussed the Zika virus and medical risks during pregnancy.
The study included 171 women in North Carolina who had been pregnant in the previous three years and hoped to become pregnant in the next three. Of these, 82% were white, 4% were black, and 7% were of another race; 6% were Hispanic. The median age was 32, and 77% had a bachelor’s degree or higher.
Results
Average word counts were similar for IDIs and FGs in person and using online video. In-person IDIs had a higher average word count than IDIs using online chat or email (p<0.01). For FGs, average word count was higher in person compared with online chat or message boards (p<0.05) and in online video compared with online chat groups (p<0.05).
Frequency of thematic codes applied and frequency of disclosing sensitive information did not differ across modalities. For FGs, women in online chat groups were more likely to express a dissenting opinion than women in online video or in-person groups (p<0.05).
Limitations
All participants in the study were women with home internet access. Most women were white and had high education levels. Findings may differ for participants from other backgrounds or for people without home internet access.
Conclusions and Relevance
In this study, the thematic content of data generated using in-person and online modalities did not differ significantly. Researchers can consider these results when determining ways to collect qualitative data from patients.
Future Research Needs
Future research could include populations from different backgrounds.
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented, and the researchers made changes or provided responses. The comments and responses included the following:
- Reviewers asked for a better rationale for the fourth study objective, which focused on the clinical content area, as it did not fit well within this methods study. The researchers agreed, stating that they had originally proposed to include the clinical topic in their results but that this did not fit with the rest of the report. The researchers removed this content but stated that they were developing a manuscript for publication that would address the clinical objective.
- Reviewers noted that the project focused on two measures of focus group and interview data—the volume of data collected and the range of themes—but did not explore these issues in depth, such as analyzing the volume of data per participant within focus groups or how often different focus group participants repeated themes. The researchers agreed that additional analyses were possible. However, since their goal was to provide comparable data from both individual interviews and focus groups, they did not analyze within-group interactions in focus groups that would not apply to individual interviews. They cited other studies that analyzed issues specific to focus groups.
- Reviewers said that the scale of the study was impressive for a qualitative study but restricting the population studied to women of childbearing age in a small geographical region limited the generalizability of the findings. The researchers explained that most qualitative studies, and focus groups in particular, tend to target a restricted population, but this study considered a population with a relatively wide pool of potential participants. The researchers argued that a degree of homogeneity among participants helped the study avoid confounders when comparing data gathered using different modes of engagement. They suggested that future research can build on this base using other populations.