Results Summary
What was the project about?
A stepped-wedge study is a type of clinical trial that can help researchers learn which treatments are effective. Researchers compare groups of patients, such as patients at different clinics, before and after they receive a new treatment. All groups start with usual care. Then, researchers assign groups by chance to the new treatment at different times during the study. This process continues until all groups receive the new treatment.
Because a stepped-wedge study can take a long time, patients may leave before the study ends, resulting in missing data. Missing data make it hard to know if the new treatment works. To address this problem, researchers need methods to design such studies and know how many patients to enroll.
In this project, the research team developed a new method to find out how many patients should be in stepped-wedge studies to get accurate results.
What did the research team do?
To create the new method, the research team used a statistical approach called generalized estimating equation. This approach uses data from different groups over time to estimate how the treatment will work.
Then the research team used data from real stepped-wedge studies to create different scenarios. For example, scenarios had different health outcomes and varying patterns of missing data. The team tested how the new method worked in each scenario.
Patients, clinical researchers, and people working in health systems helped design the method.
What were the results?
The new method helped the research team find out the number of patients to include in different stepped-wedge design scenarios. For instance, the method helped them figure out how many patients should be included when the group sizes changed after some patients left the study.
Compared with current methods, the new method worked well even when data didn’t meet all the requirements for the statistical approach.
The research team created a computer program to use the method.
What were the limits of the project?
The new method assumes that data are missing by chance and patients who leave a study aren’t different from patients who stay. It may not work if data aren’t missing by chance, such as if patients with a certain health problem drop out of a study. The study developed the new method only for stepped-wedge trials with a single treatment.
Future studies can develop methods to help design stepped-wedge studies with more treatments.
How can people use the results?
Researchers can use the new method to find out how many patients to include when designing stepped-wedge studies to compare treatments.
Professional Abstract
Background
The stepped-wedge trial is a type of cluster randomized clinical trial in which all patient groups, or clusters, start in the control group with the standard treatment. Then, at predefined intervals, researchers randomly assign groups to receive the intervention. This process continues until all groups are receiving the intervention. Stepped-wedge trials help researchers compare treatment outcomes between the intervention and standard treatment over time.
Stepped-wedge trials often take a long time to complete and have a high risk of attrition and missing data. Researchers need to account for such issues when estimating sample size and designing stepped-wedge trials. Sample size affects the power of a study and whether researchers get valid results about a treatment’s effect. Existing sample size estimation methods for designing stepped-wedge trials do not adequately account for issues such as small cluster sizes or attrition. Further, methods for analyzing categorical and count outcomes in stepped-wedge trials are limited. Improvements in sample size estimation could help researchers design stepped-wedge trials for rigorous comparative effectiveness research.
Objective
To develop new methods for stepped-wedge trial design and sample size estimation
Study Design
| Design Element | Description |
|---|---|
| Design | Theoretical development; simulation studies |
| Data Sources and Data Sets |
Data from 5 stepped-wedge trials:
|
| Analytic Approach |
|
| Outcomes |
|
Methods
The research team used a statistical approach called generalized estimating equation (GEE) modeling to develop a sample size estimation method for different types of stepped-wedge trials.
Using simulation analysis, the research team tested the method’s performance with different trial designs, informed by review of actual trials. They tested the method under different scenarios with varying missing data patterns, cluster sizes, and duration of follow-up. The team also tested whether the method helped analyze treatment effect using outcomes that were continuous, binary, categorical, or counts. The team compared the new method’s performance with existing sample size estimation methods.
Patients, clinical researchers, and members of health system leadership provided input to help design the methods.
Results
The research team developed formulas for calculating sample size for different stepped-wedge trial designs. The new sample size estimation method demonstrated good performance in simulated studies. Type 1 error was maintained under different scenarios, such as missing data and randomly varying cluster sizes.
Compared with existing methods, the GEE sample size method worked when deviations from assumed data distributions occurred. When cluster sizes were small, bias correction or increasing cluster sizes improved power.
The research team developed an R code to apply the GEE method.
Limitations
The new method assumes data to be missing at random. If data are not missing at random, the method may not work well. The study developed sample size estimation formulas only for stepped-wedge trials with a single intervention.
Conclusions and Relevance
The new sample size estimation method can accommodate different issues that arise from repeated measurements and attrition. Researchers can use the method when planning a stepped-wedge trial to improve the validity of results.
Future Research Needs
Future research could develop the method further for stepped-wedge trials with three or more interventions and address issues such as unequal length of follow-up.
Final Research Report
This project's final research report is expected to be available by December 2022.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers noted that not all of the cross-sectional designs discussed in the report would result in all participants receiving the intervention, despite the researchers’ claims that this is a particular advantage of stepped-wedge research designs like those discussed in the report. The researchers revised the report to specify that all participants would receive the intervention only in closed-cohort, stepped-wedge trials where additional participants are not added to the cohort over time.
- The reviewers asked the researchers to describe further how suggestions from their stakeholder advisory panel tied into the sample size and analytic design research in this study. The researchers explained that stakeholder input was not necessarily directly related to sample size calculation or stepped-wedge methods, but to how this research could affect the timeliness or usefulness of study results.
- The reviewers questioned if the methods developed were based on independence estimating equations and the researchers were inconsistent in their acknowledgments of the potential limitations using this approach. The researchers clarified the report by referring to generalized estimating equations instead of independence estimating equations throughout. They also provided additional simulation results comparing their method using generalized estimating equations to other analytic methods and demonstrating that the researchers’ methods are robust, even in cases where the distribution of scores does not follow a normal distribution.