Results Summary
What was the research about?
Sometimes, people with a certain trait respond differently to the same medicine. For example, women might, on average, benefit more from a medicine than men or have worse harmful side effects. If researchers group everyone together when they analyze study data, they may miss learning about which people are more likely to benefit or have harmful side effects from a medicine.
In this study, the research team grouped people based on their likelihood of having benefits or side effects from a certain medicine. The team used data from research studies of two medicines:
- Pioglitazone treats diabetes in patients who have had a stroke but can increase the risk of breaking a bone.
- Anthracycline treats women with breast cancer but can harm the heart.
What were the results?
Grouping data on people based on their likelihood of having benefits or side effects from each medicine showed differences in how the medicines affected them. The group of patients with a low risk of breaking a bone was more likely to benefit from pioglitazone. Patients in the group less likely to benefit from anthracycline had more heart problems.
What did the research team do?
The research team used statistical methods to analyze the data from two studies. The studies looked at benefits and side effects of medicines for different groups of people. The groups included patients with a
- Low or high chance of benefitting from the medicines
- Low or high chance of having side effects from the medicines
In each group, the team then looked at the actual benefits and side effects patients had from the medicines.
What were the limits of the study?
The research team used data from past studies. The studies didn’t include some health data, such as history of fracture and smoking, that could affect how well the medicines work or their side effects. Having this information might have changed how the team grouped the patients. Some groups of people may have been too small for the statistical methods to work well.
Future research could further explore ways of grouping patients to study differences in the effects of medicines.
How can people use the results?
Researchers can use the statistical methods to better understand how likely patients are to have benefits or harms from medicines.
Professional Abstract
Objective
To examine the value of assessing heterogeneity of treatment benefits and harms in guiding clinical choices across patients
Study Design
| Design Elements | Description |
|---|---|
| Design | Empirical analysis and predictive modeling using secondary data from preexisting data sets of 2 RCTs |
| Data Sources and Data Sets |
Study 1: secondary data from RCT on the effect of pioglitazone for patients with insulin resistance and stroke, secondary data from the Women’s Estrogen for Stroke Trial Study 2: secondary data from RCT on the effect of anthracycline for patients with breast cancer |
| Analytic Approach | Risk stratification, Cox proportional hazards survival analysis, predictive modeling |
|
Study 1: relative and absolute risk of recurrent stroke or MI within 5 years, bone fracture Study 2: relative and absolute risk of breast cancer mortality and cardiotoxicity |
Understanding heterogeneity of treatment effect (HTE), or variation in patients’ likelihood of experiencing treatment-related benefits and harms, can help clinicians and patients choose appropriate therapies. In this study, researchers conducted two studies of different ways to model treatment benefits and harms to detect HTE. They used data from randomized controlled trials (RCTs) to identify benefit-harm tradeoffs for patients with different levels of risk from the primary treatment benefit or major harm.
Study 1 included data from an RCT on the effect of pioglitazone in patients with insulin resistance who have had a stroke. Pioglitazone can prevent recurrent strokes or myocardial infarction (MI) but increases the risk of bone fractures. Researchers used Cox proportional hazards regression analysis to model the risk of stroke or MI and the risk of bone fracture for each participant. Then they assigned participants to risk quantiles and compared the absolute benefit across risk quantiles. Researchers validated the model using data from the Women’s Estrogen for Stroke Trial.
Study 2 included data from an RCT on the effect of anthracycline in women with breast cancer. Anthracycline reduces the risk of breast cancer recurrence but can have toxic effects on the heart. Researchers used a Cox proportional hazards model to predict cardiotoxicity. To estimate 5- and 10-year mortality from breast cancer for each patient, researchers used an externally validated risk-prediction tool, called PREDICT, version 2. They used these models to predict the risk of cardiotoxicity across patient groups stratified by their likelihood of benefit from anthracycline (lower mortality from breast cancer).
Results
In study 1, researchers reported significant HTE in absolute benefit of pioglitazone.
- Patients at low risk of stroke derived less benefit from pioglitazone in preventing a stroke or MI than those at high risk of stroke.
- Patients at low risk of fracture were less likely to experience harms from pioglitazone than those at high risk of fracture.
- Patients at low risk of fracture had the most benefit in stroke prevention.
In study 2, researchers found significant HTE in absolute benefit of anthracycline. Patients with high anticipated anthracycline-related benefit had a lower risk of cardiotoxicity than those with low anticipated benefit.
Limitations
In study 1, information on certain risk factors, such as history of fracture, smoking, and bone density measures, was not available, which may have affected risk modeling and HTE detection. In addition, the sample size was small for certain subgroup analyses.
In study 2, the overall sample size was small, which could limit the generalizability of the results. Cardiotoxicity was assessed only during the first year, so data on longer term cardiac outcomes were not available.
Conclusions and Relevance
Stratifying risk helped researchers detect clinically significant HTE in treatment-related benefits and harms, which is information that could help patients and clinicians personalize treatment decisions.
Future Research Needs
Future research could apply risk stratification in analyzing HTE using data from RCTs on additional treatments. It could also refine the current methods with routine collection of information, including long-term follow-up, relating to treatment benefits and harms.
Final Research Report
View this project's final research report.
Journal Citations
Results of This Project
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented, and the researchers made changes or provided responses. The comments and responses included the following:
- The reviewers generally found the report to be comprehensive and mostly requested clarifications on the statistical methods used and about the text in the figures. The researchers added details to the text and to figure captions.
- A reviewer questioned the need to demonstrate that treatment would reduce risk less for a low-risk group than for a high-risk group. The researchers responded that high-risk patients are often but not always more likely to benefit from treatment. They strongly disagreed with the suggestion that they did not need any analysis to assess how absolute treatment effects vary in relation to initial risk. The researchers added a section to the introduction to discuss the concept of heterogeneity of treatment effects. They also added that there are strong theoretical reasons and good empirical evidence that treatment effects are not necessarily proportional across levels of risk. Therefore, the researchers argued that risk-stratified analyses are necessary to evaluate harm-benefit trade-offs.
- A reviewer argued that the study’s stratification of risk, a continuous measure, into discrete groups was arbitrary. Each stratum encompassed a great deal of heterogeneity of risk. They stated that patients with risk scores closer to the middle might be more like patients in the other stratum, rather than patients within their own stratum, but at the other end of the risk score continuum. The researchers responded that it is customary to categorize patients into subgroups when analyzing heterogeneity of treatment effects in clinical trials. The researchers agreed that dividing patients into discrete groups has some disadvantages, but even so, using risk strata can help illustrate the clinical importance of differences in treatment effects. In the report, the researchers acknowledged the lack of consensus on how to optimally group continuous measures and suggested that this is an area warranting future methodological research.