Results Summary
What was the research about?
Quitting smoking is hard for many people. But effective programs are available to help people who want to quit. Decide2Quit (www.decide2quit.org) is a website that provides information and support for smokers who want to quit.
In this study, the research team wanted to test ways to encourage people to use the Decide2Quit website and quit smoking. The team compared two approaches, used alone and together:
- Emails based on machine learning versus standard emails. People received emails for six months encouraging them to use the website. The research team looked at two ways to tailor emails. The first used machine learning, where computers tailored emails based on people's website use and ratings of emails from a previous study. The second, standard emails, tailored messages based on people's answers to questions on the website about how ready they were to quit smoking.
- Peer recruitment guide versus no guide. The research team looked at whether a guide helped people recruit peers who were smokers to use the website. The guide included ways to recruit others, such as a private Facebook message, an email form on the website, and word of mouth.
What were the results?
People who received machine-learning-based emails and the recruitment guide had higher rates of quitting smoking than the other three groups. The four groups did not differ in the number of people who visited the website.
Between the two types of e-mails, people didn’t differ in quit rates, number of cigarettes smoked per day, or number of website visits.
People who got the guide had higher quit rates and a lower number of cigarettes smoked per day than people who didn’t get it. But the number of their recruits who used the website—and how fast they visited it—didn’t differ.
Who was in the study?
The study included 1,487 adult smokers. Of these, 88 percent were non-Black, and 12 percent were Black. About 52 percent were over age 45, and 84 percent were women.
What did the research team do?
The research team assigned people in two steps. First, the team assigned people by chance to receive machine-learning-based or standard emails. Next, the team assigned people to receive the peer recruitment guide or no guide. All peers received the recruitment guide, and they were assigned to either email type by chance.
As a result, the four groups in the study received:
- Machine-learning-based emails only
- Machine-learning-based emails with peer recruitment guide
- Standard emails only
- Standard emails with peer recruitment guide
The research team tracked people’s website use. At the start of the study and six months later, people completed surveys about smoking.
Smokers and a community health advocate helped design the study.
What were the limits of the study?
People self-reported if they quit smoking. Results may have differed if the research team confirmed that people quit.
Future research could look at how the recruitment guide improved smoking quit rates.
How can people use the results?
Researchers can use the results when considering ways to encourage use of websites to quit smoking.
Professional Abstract
Objective
To compare the effectiveness of (1) machine-learning-based versus standard email messaging and (2) access versus no access to a peer recruitment guide for increasing the use of a smoking cessation website and improving smoking outcomes
Study Design
| Design Element | Description |
|---|---|
| Design | Randomized controlled trial |
| Population | 1,487 adults who were current smokers with internet access |
| Interventions/ Comparators |
|
| Outcomes |
Number of visits to smoking cessation website, number of peers recruited to use the website, average number of days required to recruit others to use the website, smoking quit rates, number of cigarettes smoked per day |
| Timeframe | 6-month follow-up for study outcomes |
This randomized controlled trial included two comparisons:
- Machine-learning-based messaging versus standard messaging. Researchers compared two ways to tailor health messages to encourage use of a smoking cessation website. Machine-learning-based messaging tailored information based on (1) metadata from the participants’ website usage and (2) ratings of the messages from current participants and from participants in a previous study. Standard messaging tailored emails based on participants’ answers to questions on the website about their readiness to quit smoking.
- Peer recruitment guide versus no guide. Researchers sought to determine whether providing a peer recruitment guide helped participants recruit their peers to use the website. The guide included a Facebook plug-in and email forms to help automate recruitment.
Group assignment occurred in two steps. First, researchers randomized participants to receive either machine-learning-based messaging or standard messaging. In each group, participants received email messages over six months that encouraged them to engage with the online smoking cessation website. Next, researchers randomized participants to receive the peer recruitment guide or no recruitment guide. All peers recruited by participants were assigned to receive the peer recruitment guide and were randomly assigned to either messaging type.
Participants completed surveys about smoking and about when they first recruited peers. Researchers tracked website use.
The study included 1,487 adult smokers. Of these, 88% were non-Black, and 12% were Black. About 52% were over age 45, and 84% were female.
Adult smokers and a community health advocate helped design the study and interpret results.
Results
Participants who received machine-learning-based messaging and the participant recruitment guide had higher (p=0.05) smoking quit rates compared with the other three groups. The four groups did not differ significantly in the number of participant visits to the website.
Quit rates, number of cigarettes smoked, and number of participant visits to the website did not differ significantly between the two messaging approaches.
Compared with participants who did not receive the peer recruitment guide, those who did had higher (p<0.0001) smoking quit rates and smoked a lower (p=0.040) number of cigarettes per day; visits to the website did not differ significantly. In addition, the number of peers the participants reported recruiting and the speed of recruitment did not differ significantly between the groups.
Limitations
Participants self-reported smoking cessation status; social desirability bias may have affected these reports.
Conclusions and Relevance
In this study, machine-learning-based messaging with the recruitment guide were associated with higher quit rates but did not affect the number of peers recruited, the speed of recruitment, or the number of times participants used the website.
Future Research Needs
Future research could look at how the recruitment guide improved smoking quit rates.
Final Research Report
View this project's final research report.
Journal Citations
Results of This Project
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers questioned the study conclusions because the study design included a nonrandomized participant cohort recruited by their peers, potentially biasing the results. The reviewers went on to suggest the possibility that motivation to quit smoking was high among the peer-recruited individuals not because of the peer recruitment tools used but because those recruited were asked by someone they knew. The researchers acknowledged the potential bias by adding the concern to their study limitations. They also added comments in the report indicating that just being recruited by their peers could add to individuals’ motivation to quit smoking.
- The reviewers noted that the researchers’ first aim was to reach and recruit a greater proportion of African American smokers than typically participated in research studies, but there were no strategies implemented to target their recruitment. The reviewers asked for more information on how the researchers tried to engage African American smokers to participate in the study. The researchers stated that they did not make any additional efforts to recruit African American smokers beyond the use of the peer recruitment materials. They just counted on repeating the success of their pilot study in recruiting African American smokers. The researchers also acknowledged that one strategy that might have helped recruit hard-to-reach populations—providing incentives for peer recruitment—had been dropped early on because of concerns that such a practice would not be sustainable in a real-world implementation of the intervention.