Developing Software for Using Bayesian Regression to Evaluate Heterogeneity of Treatment Effects in Data from Randomized Controlled Trials

• Download Summary

What was the research about?

Comparative effectiveness research compares two or more treatments to see which one works better for which patients. Sometimes, groups of people respond differently to the same treatment. For example, women might, on average, receive more benefit from a treatment than men do. If researchers group women and men together when they analyze study data, they may miss this difference and overlook some of the benefits of a treatment.

Researchers can analyze data on the effects of a treatment in many ways. Each way has strengths and weaknesses. Bayesian regression is one method that allows researchers to consider various factors in their analysis, such as patients’ ages, sex, or health problems. This method can help researchers understand how different groups of people respond to a treatment. But it requires advanced computer programs that are not readily available to all researchers.

In this study, the research team wanted to make it easier for researchers to use Bayesian regression.

What were the results?

The research team created a new type of software. The software helps researchers use Bayesian regression to understand how different groups of people respond to a treatment. The team also provided

• Instructions on how to use the software
• A detailed example using data from a past study on heart disease
• Tips to help researchers know how best to analyze data and report the results

What did the research team do?

First, the research team identified the best ways to use Bayesian regression to understand how different groups of people respond to a treatment. The research team then designed the new software. During the study, the team worked with a panel of experts in statistics. The panel gave input on the best ways to use Bayesian regression. The panel also gave feedback on the software.

To find out how best to analyze data and report results, the team talked with the panel. The team also reviewed studies on methods to understand how different groups of people respond to a treatment.

What were the limits of the study?

The software only works for randomized controlled trials, a type of study where researchers assign the treatment a patient receives by chance. Future research can look at how to develop software for other types of studies.

How can people use the results?

Researchers can download the software from this study for free. The software may help researchers to see when treatments work well for some groups of people but not for others.

Objective

(1) To develop recommendations for assessing heterogeneity of treatment effects (HTE) using Bayesian regression along with a corresponding user-friendly, open-source, validated software suite to perform the analyses; (2) To develop recommendations for the choice of treatment-effect scale for the assessment of HTE

Study Design

The term HTE describes a situation in which individuals have different responses to a treatment; for example, some receive a benefit while others do not or are harmed. HTE can occur across a variety of patient characteristics, such as demographics, health behavior, genetics, or comorbidity. Researchers often use subgroup analysis to determine whether HTE exists, but this approach is imprecise and may elevate error rates. Using Bayesian regression models helps overcome these limitations by formally including prior information about the subgroup in the estimation process and then updating the likelihood that people in a subgroup will respond a certain way to treatment as more information becomes available. However, many researchers may not be familiar with Bayesian models, which require complex software.

For objective 1, the research team developed specific analytic recommendations for using Bayesian regression to evaluate HTE. Then the team developed a free, user-friendly software package, called beanz, for comprehensive Bayesian HTE analysis. A panel of leading statisticians and methodologists provided input on the analytic recommendations and the software package.

For objective 2, the research team conducted a literature review and elicited guidance from the expert panel to develop recommendations related to selecting the appropriate treatment-effect scale for HTE analysis and reporting of results. However, the beanz software currently provides HTE analysis only on commonly used relative scales (e.g., risk ratios or odd ratios).

Results

The recommendations on how to model HTE using Bayesian regression cover determining which model to use, choosing prior probability distributions for estimating HTE, and assessing model adequacy.

The beanz software allows researchers who may not be skilled in the use of advanced Bayesian software to explore these models and estimate HTE. To accompany beanz, the team developed a user’s manual and a detailed case study using data from a large clinical trial called SOLVD, which tested a drug to treat congestive heart failure.

The recommendations on selecting the appropriate treatment-effect scale for HTE analysis and reporting of results include the following:

• Evaluate qualitative HTE (i.e., when one or more subgroups have a treatment effect in the opposite direction of the overall treatment effect) in a prespecified manner for important subgroups (e.g., men versus women).
• Evaluate HTE on different scales (e.g., multiplicative and additive) because HTE might be present on one scale but not on another.
• Use different scaling, if appropriate, for modeling and for reporting results. Physicians and patients typically prefer results reported in terms of absolute magnitude of benefit or harm; however, modeling can use whichever scale best fits the data.
• Interpret statistically significant interactions cautiously, given that statistical significance may not translate into clinical significance.

Limitations

Researchers can use the proposed Bayesian methods for HTE analyses in randomized controlled trials, but not in observational studies without accounting for confounding due to the influence of expected treatment outcomes on treatment selection. These methods focus on patient-level factors of HTE and do not address provider-level factors or variations in the treatment itself, as occurs in pragmatic trials. Researchers cannot use the methods for evidence synthesis, for example, for meta-analyses of multiple randomized controlled trials. The methods also have limited usefulness when a priori specification of HTE factors is not possible.

Conclusions and Relevance

The Bayesian approach offers an effective, practical framework for evaluating HTE. The beanz software and the accompanying guidelines and recommendations make the approach more accessible to researchers.

Future Research Needs

Future research could further test and validate the beanz software and work to develop Bayesian HTE methods for observational studies such as those using data from electronic health records, individualized treatment effect estimation, pragmatic trials, or device and procedure trials where variations in provider-level characteristics and other treatment-related factors are major sources of the HTE.