Results Summary

What was the project about?

Randomized controlled trials, or RCTs, look at how well treatments work. But people who take part in RCTs may differ from patients who receive care in clinics. For instance, patients who take part in RCTs may be less likely to smoke or may have fewer health problems. These differences can affect how well a treatment works. As a result, a treatment may work differently for a patient receiving care in a clinic than it did for patients who took part in the RCT.

Researchers can use statistical methods to account for differences in patient traits and behaviors. In this project, the research team developed and tested new methods to account for these differences. They used the methods to apply RCT results to patients receiving care in clinics.

What did the research team do?

The research team combined data from an RCT with data from patients receiving care in clinics. The team looked at two ways of combining the data:

  • Nested design. The RCT data were inserted, or nested, within data from patients receiving care in clinics.
  • Non-nested design. The team added data from patients receiving care in clinics to the RCT data.

The research team then created three types of statistical methods to analyze the data from both study designs. The team used the methods to look at how well the treatment would work for the patients receiving care in clinics. The team tested the accuracy of the methods using data created by a computer and data collected from patients.

Patients and healthcare providers helped advise the study team.

What were the results?

All three types of methods worked correctly if enough data were included from the patients receiving care in clinics. If enough data weren’t available, then the findings from the RCT couldn’t be applied to clinic patients.

What were the limits of the project?

The methods need further development if data from patients receiving care in clinics are incomplete or have errors. To use the methods, researchers need to make sure that they have data on the patient traits that may affect how well a treatment works.

Future studies could explore ways to use the methods with other health problems and different data sets.

How can people use the results?

Researchers can use these methods to apply RCT results to patients receiving care in clinics.

Final Research Report

View this project's final research report.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:

  • The reviewers questioned the applicability of the methods tested in this study. They noted that if a target population includes patients who would not have been eligible for the randomized trial, then the assumptions resulting from the randomized trial would not be generalizable to that target population. The researchers disagreed, stating that as long as the factors that made that target population ineligible were not factors that affected the trial assumptions, investigators could make inferences from the randomized trial to the target population.
  • The reviewers noted that the report often refers to a target population in a way that readers might think that each study can only have one target population. The researchers agreed that this would be an incorrect assumption and revised the report to more clearly explain that the methods can be applied to any population, and that any one analysis would require the investigator to choose a specific group or population to test.
  • The reviewers suggested that the researchers eventually provide an annotated version of the statistical codes from the appendix of this report on a public platform to make their methods more accessible. The researchers agreed and explained that they had already posted annotated statistical codes from this study on the GitHub.com platform.

Project Information

Issa J. Dahabreh, MD, ScD
Harvard TH Chan School of Public Health
$1,151,953
10.25302/07.2021.ME.150227794
Making Better Use of Randomized Trials: Assessing Applicability and Transporting Causal Effects

Key Dates

September 2015
October 2022
2015
2021

Study Registration Information

Tags

Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: October 20, 2022