Project Summary

It can be challenging for decision makers and clinical providers when the results of randomized controlled trials conflict with each other. Currently, there are no standard methods and guidelines that can be used by decision makers in this situation, even though they are often tasked with deciding whether the treatment of interest should be available, for whom the treatment may work and whether another trial is needed. 

This study aims to develop and apply causal inference and machine learning methods to answer two key questions decision makers may ask when faced with conflicting results from randomized trials: 

  • To what extent can the conflicting results be explained by differences in study populations or differences in how participants followed the study protocol? 
  • Given an individual’s unique characteristics, will they benefit from the treatment of interest, or could the treatment of interest be harmful? 

The results of this project will help decision makers decide 

  • Whether a treatment should be available 
  • Which populations may benefit from the treatment 
  • What information may be valuable to collect in future trials of the treatment of interest 
  • What population should be prioritized for enrollment in future trials of the treatment of interest 

In this study, researchers will reanalyze data from two large trials of preterm birth and 17-alpha-hydroxyprogesterone caproate (17P), a weekly injection that may prevent preterm birth in individuals with a previous preterm birth. The first trial found 17P substantially reduced the risk of preterm birth, but the second confirmatory trial found there was no effect. The methods and solutions the study team develops can be applied to any pair of conflicting randomized trials. 

This study aims to: 

  1. Develop methods and software to understand the extent to which conflicting results from randomized trials can be explained by differences in study populations 
  2. Develop methods and software to understand the extent to which conflicting results from randomized trials can be explained by differences in how participants followed the study protocol
  3. Develop methods and software that can be used to evaluate whether a given individual may benefit from or be harmed by the proposed treatment 

The study team will use causal inference and machine learning methods to conduct its analyses. For each aim, researchers will develop the necessary methods and then apply each method to the example of the trials of 17P and preterm birth. 

The research team has assembled a Stakeholder Advisory Panel (SAP) to help ensure the proposed research is patient centered. Panel members have a range of expertise relevant to the study objectives and will offer a range of critical perspectives. Members include physicians, epidemiologists and statisticians, clinical trialists, individuals with leadership positions in clinical societies, individuals working with national organizations that inform policy and research priorities, and individuals who serve on US Food and Drug Administration advisory committees. Through the SAP and the study’s investigator team, this study will also identify two patients with experience with preterm birth to serve as additional members of the SAP. 

All SAP members will be involved in study planning, study conduct and disseminating results and have played a role in co-developing the research questions and study design. The study team will meet with the SAP at the start of the study and then every six-months. The investigator team will work closely with SAP members to ensure the output can be used by stakeholders. The SAP will also be heavily involved in the interpretation and dissemination of the results to clinicians, patients and regulators. Finally, individual SAP members will also be involved with methods development. 

The expected output of this work includes open-source software code with corresponding documentation and user friendly tutorials that can be used by different types of stakeholders including regulators, clinical trialists, epidemiologists, clinicians, clinical societies and patients. The study team will also disseminate results through publications targeting both statistical and clinical audiences and will present key findings as conference presentations. The SAP will also provide guidance on how to best disseminate study findings to clinicians, regulators, patients and other stakeholders.

Project Information

Ellen Caniglia, Sc.D.
Enrique Schisterman, Ph.D.
The Trustees of The University of Pennsylvania
$1,049,373 *

Key Dates

36 months *
April 2024

*All proposed projects, including requested budgets and project periods, are approved subject to a programmatic and budget review by PCORI staff and the negotiation of a formal award contract.


Project Status
Award Type
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: April 23, 2024