• Download Summary

What was the project about?

Cluster randomized trials, or CRTs, are research studies that compare treatments among different groups of patients, or clusters. An example of a cluster is a group of people who receive care at a single clinic.

One type of CRT is a stepped-wedge CRT. These CRTs compare patients’ health before and after a new treatment. In stepped-wedge CRTs, all groups start with the standard treatment. Then, each group switches to the new treatment at a specific time during the study. By the end of the study, all groups are receiving the new treatment.

In stepped-wedge CRTs, group characteristics, such as how clinics follow up with patients, can affect how well a treatment works. It’s hard to figure out if changes in a patient’s health are due to the treatment or group characteristics.

In this study, the research team wanted to improve how to plan and analyze stepped-wedge CRTs for studying the effect of treatments.

What did the research team do?

The study had two parts. In the first part, the research team looked at ways to measure how well treatments work in stepped-wedge CRTs. The team also looked at how to analyze data from stepped-wedge CRTs in ways that account for group characteristics.

In the second part, the research team looked at which statistical methods got accurate results when using data from stepped-wedge CRTs. The team first used a computer program to create test data that looked like data from a stepped-wedge CRT. The team created the test data using nine scenarios; each scenario had a different set of conditions. For example, the number of patient groups varied across each scenario. Using the test data, the team compared six statistical methods for analyzing data from stepped-wedge CRTs.

The research team also created a statistical program to help plan and analyze stepped-wedge CRTs.

What were the results?

The research team found that all six statistical methods worked for some scenarios; none worked for all nine scenarios.

The new statistical program helps calculate the number of patients that should be in a study.

What were the limits of the project?

The study focused on stepped-wedge CRTs that group patients at a single level, such as by clinic.

Future research could extend the methods to stepped-wedge CRTs that group patients at multiple levels, such as patients receiving care at clinics within a region.

How can people use the results?

Researchers may be able to use these results to improve the design and analysis of stepped-wedge CRTs.

Background

In cluster randomized trials (CRTs), researchers, examine the effectiveness of interventions that are offered to groups of patients, such as patients receiving care in a particular clinic.

One type of CRT is a stepped-wedge CRT in which each group starts in the control condition and switches to the treatment at pre-selected time points. This way, all groups receive the treatment, and all groups serve as a control group at some point during the study.

However, these groups often differ by characteristics such as a clinic’s patient volume, standards of care, or patient risk factors. These differences make it hard to determine if changes in a patient’s health are due to the treatment or to group characteristics. Current analysis methods and statistical software for do not adequately address the effect of group characteristics and the longitudinal design of stepped-wedge CRTs in determining patients’ health outcomes.

Objective

To develop and test statistical methods and create a software package for planning and analyzing stepped-wedge CRTs

Study Design

Methods

The researchers first developed a new causal framework for analyzing stepped-wedge CRTs, which included five ways to measure causal effects that account for the role of clinic practices in determining health outcomes. For example, the team defined average causal effect for a provider as the average outcome for treatment and control conditions as if all study participants were treated at a single clinic.

Then, using simulated data, the researchers evaluated two semi-parametric generalized estimating equation (GEE) methods and four linear mixed model (LMM) methods for analyzing stepped-wedge CRTs. The simulated data mimicked stepped-wedge CRTs for nine scenarios, varying by conditions such as the number of groups. The team compared the six analytic methods using two approaches. The model-based approach relied on the assumptions of each method. The design-based approach used the assumptions of the stepped-wedge CRT design.

Finally, the researchers developed a computer program, swCRTdesign, to facilitate the design and analysis of stepped-wedge CRTs.

Results

Each method worked under some scenarios, but none worked for all nine scenarios.

• Model-based approach. For each of the nine scenarios, both LMM and GEE methods gave unbiased estimates of treatment effects. LMM methods produced a low Type I error rate, while the Type I error rate for GEE methods decreased as the number of groups increased from 20 to 40.
• Design-based approach. LMM and GEE methods had varying levels of bias across the nine scenarios. Type I error rates for LMM methods were inflated for some scenarios. GEE methods produced similar Type I errors for all nine scenarios.

Limitations

The study focused on methods for stepped-wedge CRTs with single-level grouping. In the design-based approach, the researchers assumed equal sample size across groups, which may not reflect real-world conditions.

Conclusions and Relevance

Researchers can use the measures of causal effects, analytic methods, and the computer program to improve the design and analysis of stepped-wedge CRTs.

Future Research Needs

Researchers could extend the methods to stepped-wedge CRTs with multiple levels of grouping such as by clinic and geographic region.