• Download Summary

What was the project about?

A rare disease is one that affects fewer than 200,000 people in the United States. Because few people have these diseases, clinical studies on treatments can be hard to conduct. One way to study rare disease treatments is with an snSMART study.

snSMART studies have two stages. In the first stage, researchers assign patients to a treatment by chance. In the second stage, patients may stay with the same treatment or switch treatments. Patients stay on the same treatment if it’s working well. If the treatment isn’t working, researchers assign patients by chance to a new treatment.

snSMARTs can help researchers learn more from a smaller number of patients than a standard clinical study. But most current methods for analyzing snSMARTs use data only from the first stage, which can lead to inefficient results.

In this project, the research team developed and tested new methods that use data from both stages to analyze snSMARTs. The team compared results from the new methods to actual treatment effectiveness to see

• Bias, or whether results are too high or too low
• Efficiency, or how big the difference is between the results and actual treatment effectiveness

What did the research team do?

The research team created two Bayesian and two frequentist methods. Bayesian methods use findings from previous studies in the analysis, but frequentist methods don’t.

Using a computer program, the research team created test data sets with 45–300 patients that looked like data from an snSMART study. Using the test data, the team compared the new and current methods to see which worked better to find the best

• First stage treatment
• Sequence of treatments

Clinicians that treat rare diseases, a patient with a rare disease, and a family member gave input on the study.

What were the results?

All methods worked well enough to see which first stage treatment or sequence of treatments worked best. To find the best first stage treatment, the Bayesian methods using data from both stages worked better than methods using only first stage data and the frequentist methods. To find the best treatment sequence, the Bayesian method was the most efficient but could result in biased estimates.

The research team also created an online sample size calculator that figures out how many patients should be in an snSMART study.

What were the limits of the project?

The research team compared the methods using test data. Bayesian methods may not get accurate results if assumptions about data aren’t met. The new methods don’t apply to designs using a placebo, or a pill that doesn’t contain medicine. Also, the snSMART design is most appropriate to study diseases with symptoms that are stable over time and don’t change.

Future studies could test the methods when data are missing. Researchers could also develop methods to analyze data from an snSMART with a placebo.

How can people use the results?

Researchers can consider using the new methods and the online calculator when designing and analyzing snSMART studies.

Background

Clinical studies of rare diseases often have small sample sizes. A small n sequential multiple assignment randomized trial (snSMART) is a two-stage design for small samples. In the first stage, snSMART studies randomize patients to one of two or more treatments. In the second stage, patients who respond to their initial treatment continue with that treatment while patients who do not respond are re-randomized to another treatment. Most existing statistical methods for analyzing snSMART designs only use data from the first stage, which can lead to inefficient estimates.

Objective

To develop new methods for designing and analyzing snSMARTs that increase efficiency and have low bias

Study Design

Methods

Using data from both stages of a snSMART studying three treatments, researchers developed two models:

• Bayesian joint stage model (BJSM)
• Log Poisson joint stage model (LPJSM)

The models estimate the best first stage treatment based on response rates or the percentage of patients who responded to each of three treatments, from both stages. Researchers compared simulation estimates from the new models with two models that relied on data from only the first stage:

• Bayesian first stage model (BFSM)
• First stage maximum likelihood estimates (FSMLE)

To estimate response rates for treatment sequences, researchers extended the new models to allow for the second stage treatment effect to depend on the first stage treatment:

• BJSM with multiple linkage parameters (BJSMM)
• LPJSM with multiple parameters (LPJSMM)

To identify the optimal treatment sequence, researchers compared BJSMM and LPJSMM with an existing method, weighted and replicated regression method (WRRM).

Clinicians who treat rare disease, one patient with a rare disease, and one family member gave input on the study.

Results

Optimal treatment. When statistical assumptions held, BJSM had low bias and was most efficient, as measured by the lowest root mean square error. BJSMM did not result in a substantial reduction in bias or error from the BJSM. The LPJSM, BFSM, and FSMLE all had negligible bias but were less efficient.

Treatment sequence. BJSMM can result in small, non-negligible bias but provides the most efficient estimates. LPJSMM and WRRM result in negligible bias but are less efficient compared with BJSMM.

Applet. Researchers created an applet to calculate required sample sizes and statistical power for snSMART designs. The applet uses assumptions of BJSM to determine the best first stage treatment.

Limitations

Researchers tested the models using simulation analyses. Statistical assumptions for the new models may not hold in clinical trials but can be tested by comparing results from alternative models presented in the study. The models do not apply to snSMARTs with a placebo.

The snSMART design is not ideal for studying diseases that change over time.

Conclusions and Relevance

The new models were more efficient than existing models. The new Bayesian models worked well for identifying the optimal treatment and the best treatment sequence.

Future Research Needs

Future research could assess the robustness of the methods for missing data. Researchers can also develop methods to analyze data from an snSMART with a placebo.