AT-1: Specify planned adaptations, decisional thresholds, and statistical properties of those adaptations

The adaptive clinical trial design must be prospectively planned and the design must be clearly documented in the study protocol before trial enrollment begins, including at a minimum:

• All potential adaptations, including timing;
• Interim trial findings that will be used in determining each adaptation;
• Statistical models and decisional thresholds to be used; and
• Planned analyses of the trial endpoint(s).

The description of the design should be sufficiently detailed that it could be implemented from the description of procedures. This specification should include a statistical analysis plan (SAP) in which all necessary detail is provided regarding planned interim and final analyses.

Additionally, the statistical properties of adaptive clinical trial designs should be thoroughly investigated over the relevant range of important parameters or clinical scenarios (e.g., treatment effects, accrual rates, delays in the availability of outcome data, dropout rates, missing data, drift in participant characteristics over time, subgroup-treatment interactions, or violations of distributional assumptions). Statistical properties to be evaluated should include Type I error, power, and sample size distributions, as well as the precision and bias in the estimation of treatment effects.

Public comments

No comments.

AT-2: Specify the structure and analysis plan for Bayesian adaptive randomized clinical trial designs

If a Bayesian adaptive design is proposed, the Bayesian structure and analysis plan for the trial must be clearly and completely specified. This should include any statistical models used either during the conduct of the trial or for the final analysis, prior probability distributions and their basis, utility functions associated with the trial’s goals, and assumptions regarding exchangeability (of participants, of trials, and of other levels). Specific details should be provided as to how the prior distribution was determined and if an informative or non-informative prior was chosen. When an informative prior is used, the source of the information should be described. If the prior used during the design phase is different from the one used in the final analysis, then the rationale for this approach should be indicated. Computational issues, such as the choice of software, the creation and testing of custom software, and software validation, should be addressed as well. Software used for Bayesian calculations during trial design, trial execution, and final analysis must be functionally equivalent. When feasible, software or programs should be made available to relevant stakeholders for evaluation and validation.

Public comments

AcademyHealth commends PCORI’s Standard AT-2 on the Bayesian trial structure, namely the request that researchers provide specific details about how the prior distribution was determined and if an informative or non-informative prior was chosen. To further improve this Standard, however, AcademyHealth would recommend a few minor changes: First, in the sentence regarding computational issues, we recommend changing “be addressed as well” to “specified” to make PCORI’s request more explicit. Furthermore, since the items that follow this sentence (e.g., software used for Bayesian calculations during trial design and trial execution) are very specific, appropriate requirements, PCORI should consider enumerating them clearly as a bulleted list of documentation requirements. This would also provide an additional sense of consistency with other reporting requirements included in the Methodology Standards.

Lisa Simpson, AcademyHealth, , 04/11/2016 - 4:47pm

AT-3: Ensure that clinical trial infrastructure is adequate to support planned adaptation(s) and independent interim analyses

The infrastructure and processes for trial implementation must be able to support the planned adaptation. The study plan must clarify who will perform the analyses to inform adaptation while the study is ongoing and who will have access to the results. The interim analyses should be performed and reviewed independent from the investigators who are conducting the trial. Trial investigators should remain blinded to changes in treatment allocation rates as this information provides data regarding treatment success. Trials with more complicated requirements, such as frequent interim analyses, require thorough testing prior to trial initiation. Such testing should involve the trial’s data collection and data management procedures, the implementation of the adaptive algorithm, and methods for implementing the resulting adaptation(s). The impact on the trial’s operating characteristics of delays in collecting and analyzing available outcome data should be assessed.

Public comments

No comments.

AT-4: When reporting adaptive randomized clinical trials, use the CONSORT statement, with modifications

The following sections of the 2010 CONSORT statement can be used to report key dimensions of adaptation:

• Adaptation of randomization probabilities (sections 8b and 13a);
• Dropping or adding study arms (sections 7b and 13a);
• Interim stopping for futility and superiority (sections 7b and 14b);
• Sample size re-estimation (sections 7a and 7b);
• Transitioning of stages (e.g., seamless Phase II/III designs) (sections 3a, 7a, 7b, and 16); and
• Modification of inclusion and exclusion criteria (sections 4a and 13a).

CONSORT sections 16, 20, and 21 may also be expanded to report additional aspects of an adaptive trial.

If the trial incorporates adaptations other than those listed above, the authors should use their judgment as to where in the CONSORT structure to include both design details and the associated results. All possible adaptations included in the prospective design, even if they did not occur, should be included in the report.

Public comments

In bullet three, AcademyHealth recommends that PCORI add “or adverse outcomes” after the full “Interim stopping” clause.

Lisa Simpson, AcademyHealth, , 04/11/2016 - 4:47pm

General feedback on the Standards for Adaptive Trial Designs

Public comments

These standards raise awareness of adaptive designs; however it is not clear why non-adaptive Bayesian designs seem to be removed from this section. We suggest PCORI includes additional language which recommends providing convergence information and assessments of sensitivity of priors (prior-to-posterior).

Eli Lilly and Company , , 03/30/2016 - 2:25pm