Results Summary
What was the research about?
Pap tests screen for cervical cancer in women. Current guidelines recommend that women ages 21–29 get a Pap test once every three years because they are at low risk for cervical cancer. But many women get Pap tests more often. Having more tests can cause anxiety and lead to unneeded procedures.
In this study, the research team tested two ways to help clinics make sure women get Pap tests as recommended but not more often:
- Clinician app. Clinicians, such as doctors and nurses, received an app for their phones. The app helped clinicians figure out how often patients should get Pap tests based on guidelines and patient traits such as age.
- Clinician app and patient website. Clinicians received the app. Their clinics also received tablet computers with a website that had information for patients about when women should get Pap tests. Clinic staff asked patients to view the website in the waiting room.
The research team compared the rate of Pap tests per woman receiving care for clinics that used the app, the app plus the website, or neither.
What were the results?
During the study, the rate of Pap tests per woman didn’t differ based on whether women received care at a clinic that had the app, the app and the website, or neither.
When the research team compared the rate of Pap tests before and after the study started, the clinics that got both the app and the website had more of a decrease than clinics with the app only or clinics with neither.
Who was in the study?
The study included claims for 84,823 women who received care at 1 of 42 women’s health clinics in Southern California. Of these women, 81 percent were Latina; 8 percent were white, 5 percent were Asian or Pacific Islander, 4 percent were black, and 2 percent were other races. Spanish was the primary language for 61 percent of women; English was the primary language for 34 percent; and 6 percent spoke another language. The average age was 26.
What did the research team do?
The research team assigned 14 clinics by chance to use either the app or the app plus the website for 18 months. Then the team compared these 14 clinics with 28 clinics that didn’t have the app or website.
The research team looked at how many Pap tests women received in the 18 months before and the 18 months during the study.
Members of advocacy organizations helped create the patient education website and recruit clinics.
What were the limits of the study?
The research team followed patients for only 18 months. Results may have differed if the study tracked patients for longer. The team also couldn’t track which clinics used the app or website. Some clinics may not have used the app or website as much as others.
Future studies could track Pap test rates for longer and look at app and website use by clinic.
How can people use the results?
Clinics can use the results when considering ways to help women get Pap tests as recommended.
Professional Abstract
Objective
To determine the effects of two interventions on adherence to three-year-interval screening for cervical cancer using Pap smears among women ages 21–29
Study Design
| Design Element | Description |
|---|---|
| Design | Randomized controlled trial |
| Population | 84,823 women ages 21–29 receiving care at 1 of 42 reproductive health clinics in California |
| Interventions/ Comparators |
|
| Outcomes |
18-month Pap rate, defined as the number of Pap claims a woman had between July 2015 and December 2016, accounting for the timing of clinic visits during that period |
| Timeframe | 18-month follow-up for primary outcome |
This randomized controlled trial of clinics compared the effects of a clinician mobile app, the app plus a patient education website, and no intervention on rates of Pap screening. Current guidelines recommend that women ages 21–29 have Pap tests every three years, but many women have these tests more frequently. The phone-based app provided clinicians with appropriate cervical cancer screening intervals for patients based on guidelines and patient characteristics such as age or pregnancy status. The patient education website described the risks and benefits of different screening intervals for cervical cancer; patients could access the website via tablet computers in the clinic waiting room.
Researchers randomly assigned 14 reproductive health clinics in California to either the clinician app alone or the app plus a patient education website. Then researchers matched the intervention clinics with 28 clinics that received no intervention, based on characteristics such as patient age and public or private clinic status.
Researchers used claims data to calculate Pap screening rates. They compared rates during the 18-month intervention period with the 18 months prior to the study start. Researchers looked at whether intervention period rates and changes in rates differed between clinics with one of the two interventions and clinics with no intervention.
The study included claims data for 84,823 women receiving care at 1 of 42 reproductive health clinics in Southern California. Of the women, 81% were Latina; 8% were white, 5% were Asian or Pacific Islander, 4% were black, and 2% were other races. The primary language was Spanish for 61% of study participants, English for 34%, and another language for 6%. The average age was 26.
Members of advocacy organizations helped develop the patient education website and recruit participants.
Results
The 18-month Pap rate for the intervention period did not differ between the clinics that received one of the interventions and clinics that received no intervention. However, compared with clinics that received no intervention, clinics that received the clinician app plus patient education website had a greater decrease in the rate of Pap tests from the 18-month period before the study to the 18-month intervention period (p=0.02). The change in rates for clinics that received the clinician app alone was similar to that of clinics that received no intervention.
Limitations
Researchers followed patients for only 18 months. Results may have been different if the study had been able to track patients for longer. The study also could not track use of the app or patient education website by specific clinic site.
Conclusions and Relevance
Use of the mobile app or the app plus a patient education website did not affect the Pap test rate. However, compared with clinics that received no intervention, clinics that received the clinician app plus the patient education website had a greater decrease in the rate of Pap tests during the intervention period.
Future Research Needs
Future research could follow patients for three years to determine Pap test guideline adherence and provider and patient use of the interventions by clinic.
Final Research Report
View this project's final research report.
Journal Citations
Results of This Project
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers asked why the 18-month follow-up period was broken into three 6-month segments and questioned the validity of extrapolating from the 18-month period to 3 years. The researchers explained that they used 6-month intervals to see if an observed change was consistent over time or if there was a lag period before an effect was seen. The researchers said they extrapolated from 18 months because their study had to be completed in a restricted time frame. The reviewers continued to express concern about this extrapolation to 3-year outcomes, and the researchers removed reference to 3 years from the report.
- The reviewers questioned the exclusion of women from the study who had screening at intervals averaging more than 30 months, many of whom were likely not having screening enough and suggested conducting statistical analyses including these women. The researchers agreed that many of the women at such sites were likely underscreened but observed that the purpose of their study was to examine overscreening, not underscreening. The researchers said they excluded clinics that were already screening at the recommended 3-year intervals. Since they excluded those clinics from the beginning of the study, the researchers had no access to data from the sites and could not perform any additional analyses of these patients.
- The reviewers noted that control participants were not assigned within clinics or practices where the researchers conducted interventions and suggested that the non-random assembly of control groups from outside practices might have introduced bias. The researchers said they designed the study in this way because practices would not participate unless they received some kind of intervention, so the researchers assigned the practices they recruited to one or the other of their two interventions. They then randomly sampled clinics that matched the intervention sites on a set of specific characteristics.
- The reviewers expressed concern that the study gave little consideration to subgroups within the study population and suggested conducting additional exploratory, after-the-fact analyses. The reviewers said they did do subgroup analysis based on patients’ primary language and race, but they did not find other patient characteristics in their dataset that they could analyze for heterogeneity.
- The reviewers wondered about providers’ familiarity with current guidelines regarding cervical screening and asked if there was any information regarding providers’ attitudes and practices before the intervention. The researchers pointed out that they were not able to conduct a formal assessment of providers’ knowledge prior to them receiving the intervention, because the assessment may have comprised an intervention itself. Instead, the study included formal group-based provider orientations, so that all providers received the same information regarding the provider tool.
Conflict of Interest Disclosures
Project Information
Key Dates
Study Registration Information
^Anna-Barbara Moscicki was affiliated with University of California, San Francisco when this project was funded.