Results Summary
What was the research about?
Research studies called clinical trials test treatments to see if they are safe and effective for patients. When designing clinical trials, researchers must plan to include enough patients with different traits for the study to have accurate results. Once the study starts, researchers must follow the plan. Sometimes, early results from a trial show that a group of patients with a certain trait may have more benefits or harms from the treatment than other groups. For example, the treatment may not work for patients with a history of heart disease. In the standard trial design, researchers can’t change the plan to stop enrolling these patients once the trial starts.
In this study, the research team compared the standard trial design with more flexible approaches known as adaptive enrichment designs. These designs set up rules that allow researchers to change the study plan. For example, if early results show a treatment doesn’t work for patients with heart disease, researchers can stop enrolling these patients in the trial. The team compared the trial designs using data from four completed trials.
What were the results?
Compared with the standard designs used in the example trials, the adaptive enrichment designs
- Had more accurate results
- Needed fewer patients in the overall trial
- Needed more patients to show that a treatment had a benefit or harm for a specific group
What did the research team do?
The research team developed different methods for designing clinical trials. To test these methods, the team used information from the four completed studies. Then the team created a computer program to help researchers choose a study design for future clinical trials. The program compares the standard and adaptive enrichment designs to predict
- How many patients would need to take part in a clinical trial to have accurate results
- How many patients with a specific trait would need to take part in the trial to show that the treatment has a benefit or harm for that group
What were the limits of the study?
The study tested the methods with only four example trials. The adaptive enrichment designs allowed researchers to look at the effect of treatment on only one specific group of patients in the clinical trial.
Future research could develop study designs that focus on the benefits or harms for more than one group of patients.
How can people use the results?
Researchers can use the computer program to choose a trial design and plan for the number of patients needed to take part in the trial.
Professional Abstract
Objective
(1) To develop new clinical trial designs, called adaptive enrichment designs, that can determine which patients can benefit from a new medical treatment; (2) To evaluate the new designs using simulations of trials involving treatments for HIV, stroke, and heart failure; and (3) To develop free, user-friendly software implementing these designs
Study Design
| Design Elements | Description |
|---|---|
| Design | Simulation studies to compare standard designs with proposed adaptive enrichment designs |
| Data Sources and Data Sets |
Simulations based on 4 RCT data sets:
|
| Analytic Approach | For the simulation studies, researchers compared the performance of adaptive enrichment and standard designs. |
| Outcomes |
Primary: maximum sample size and average sample size Secondary: performance metrics to assess precision gain, including estimator bias, variance, mean squared error, coverage of confidence intervals |
Standard designs for randomized controlled trials (RCTs) typically exhibit reduced statistical power to detect if a treatment differentially benefits or harms a subpopulation of study participants. Adaptive enrichment designs can address this issue by including rules for changing the trial protocol based on interim data, thus preserving a study’s ability to detect these effects.
In this study, researchers developed new adaptive enrichment designs to optimize sample size for use in clinical trials examining time-to-event and other delayed outcomes. To reduce error and improve the precision of the analysis results, researchers also explored the addition of new prognostic baseline variables, which are characteristics that correlate with the progression or resolution of a disease.
Researchers evaluated new adaptive enrichment designs in three simulation RCT data sets and assessed the value of prognostic baseline variables in one simulation RCT data set. They compared the performance of these new designs to standard designs considering
- Maximum sample size, which is the number of participants in the study if enrollment does not stop
- Average sample size, which is the average number of participants enrolled
- Precision gain, which is the improvement in estimating the population-level treatment effect due to adjusting for chance imbalances between study arms in baseline variables
Then the researchers developed a free, open source software program to assist researchers in designing clinical trials. Users can input their own study scenarios and performance criteria into the program to evaluate and compare numerous standard and adaptive enrichment design choices. The program works with clinical trials that have continuous, binary, or time-to-event primary outcomes.
Results
The adaptive enrichment design reduced the expected sample size and increased the maximum sample size in the simulation studies that used data from two of the four simulated trial types. Results were as follows:
- Stroke trial 1: New prognostic baseline variables led to a 12% precision gain compared with the standard variables.
- Stroke trial 2: Adaptive enrichment design reduced expected sample size by 32% (adaptive = 981 versus standard design = 1,443 patients), but maximum sample size was 22% larger (adaptive = 1,762 versus standard design = 1,443 patients).
- Cardiac resynchronization device trial: Adaptive enrichment design reduced expected sample size by 25% (adaptive = 1,525 versus standard design = 1,818 patients), but maximum sample size was 5% larger for the adaptive enrichment design (2,154 versus 1,818 patients).
- HIV trial: Adaptive enrichment design did not provide any benefit. The time to observing the first outcome was too long for meaningful changes to enrollment to be useful.
Limitations
The proposed adaptive enrichment designs require that researchers know the populations of interest before the trial starts. For example, they could identify these populations by the severity of disease or a genetic marker. Using the software program requires knowledge of statistical methods used to design and analyze standard clinical trials.
Conclusions and Relevance
The new methods proposed in the study offer researchers additional options for designing rigorous clinical studies that respond to emerging differential treatment effects. The accompanying software allows researchers to compare the advantages and disadvantages of standard and adaptive enrichment designs to help them plan and conduct clinical trials.
Future Research Needs
Future methodological research could investigate other types of adaptive enrichment designs that allow for sample size modifications, dose selections, varying proportions of subpopulations, or multiple subpopulations.
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented, and the researchers made changes or provided responses. Those comments and responses included the following:
- Reviewers noted that in adaptive enrichment trial designs, researchers had many design decisions to make. Those decisions could have affected the validity of the study, such as when to include interim analyses and how many, and researchers could have provided in the report additional, useful guidance about how to approach such decisions. The researchers responded that they would prefer not to provide general guidance on these study-specific choices. Instead, the software the researchers created provides a range of options that clinical investigators can explore as part of designing an adaptive enrichment trial.
- Reviewers asked whether the results of the sensitivity analyses could be presented in a table with quantitative findings clearly provided. The researchers explained that this might require substantial space to explain the simulation step, and that a qualitative presentation through plots, would be more useful to the report overall.