Background: Both behavioral prevention and medical research have frequently shown prominent subgroup interactions in treatment effect. However, subgroup analyses of clinical trial data is controversial, largely due to the high likelihood of error and lack of replicability when a lengthy list of interactions with treatment are tested. Machine learning techniques, such as random forests (RFs), provide a principled approach to exploring a large number of predictors and identifying replicable sets of predictive factors. In recent innovations these machine learning techniques have been used specifically to uncover subgroups with differential treatment responses. We are proposing an improvement on this approach.
- Aim 1: Develop comprehensive methodology for reliably estimating patient subgroup treatment effects using optimal person-specific counterfactual RF machines for heterogeneous and potentially confounded data.
- Aim 2: Develop free software to implement these new methods in a wide array of comparative effectiveness research applications using the R software platform.
- Aim 3: Work with stakeholders (public health department, infectious disease clinicians) and patient groups (men who have sex with men, other sexually transmitted infection clinic patients) with interpretation of clinical findings and the feasibility (stakeholder and patient attitudes and concerns) of incorporating data collection and predictive modeling as part of clinic procedures.
Methods: We will evaluate the methods developed using both simulation and data from Project Aware, a comparative effectiveness study that investigated whether HIV risk reduction counseling at the time of HIV testing was an effective strategy. Results from this study showed that such risk reduction counseling does not have an impact and therefore should not be universally provided. This same data, however, can tell us whether there are subgroups in which counseling was effective and whether there were any subgroups, in addition to men who have sex with men, for which counseling had negative consequences.
Patient/Stakeholder Engagement: We have met with sexually transmitted disease clinic personnel and patients to develop our study proposal and will continue to interact with them to receive feedback throughout the research process. In consultation with patients and other stakeholders, we will identify the strengths and weaknesses of the methods developed and create guidelines for how to present the methods and their potential value to future patients.
Anticipated Impact: The methods developed in this study will have wide applicability to nearly all comparative effectiveness research by providing a method and software to identify for whom specific interventions work. These methods have promise to become the backbone of patient information–focused feedback systems, providing patient-specific estimates of expected level of success with alternative treatments.