Comparative effectiveness research studies often use secondary data sources, such as electronic health records. Secondary data include detailed clinical information but may be missing data on potential confounders, which are variables that affect both treatment choices and outcomes. Not accounting for confounders in an analysis can lead to biased results about the effectiveness of a treatment.
In such situations, researchers can use case-only studies, which compare each patient’s outcomes before and after treatment. Case-only studies eliminate the influence of confounders that remain stable over time. However, case-only studies do not identify heterogeneity of treatment effects. Also, they typically report results in terms of relative risk rather than absolute risk, as current methods for estimating absolute risk in case-only studies are limited. Information about absolute risk, like the number of patients who need to be treated for at least one patient to have a benefit or harm from treatment, can help in clinical decision making.
To improve methods and provide clear guidance for designing and analyzing case-only studies
|Theoretical development; simulation studies
|Data Sources and Data Sets
|Simulated data based on data from Beth Israel Deaconess Medical Center (N=120,000)
Develop guidance about choosing and implementing the appropriate case-only design to address the research question of interest
Simulation analysis to compare findings from 6 case-only designs:
- Unidirectional case-crossover
- Unidirectional fixed-effect case-time control
- Unidirectional Mantel-Haenszel incidence rate ratio
- Bidirectional case-crossover
- Bidirectional self-controlled case series
- Bidirectional Mantel-Haenszel incidence rate ratio
Validity (bias) and efficiency (mean squared error; mean-variance ratio for the estimator)
Methods and Results
To develop guidance for selecting and analyzing a case-only design, the research team analyzed simulated data and compared the validity and efficiency of findings from six case-only designs. The team examined the potential bias when statistical assumptions of case-only designs were and were not met. For example, case-only designs assume that confounders and the probability of receiving treatment do not change over time. Based on the simulation results, the team described data and design considerations for case-only studies that worked when different statistical assumptions were relaxed.
To estimate absolute risk and heterogeneity of treatment effects, the research team developed new methods that account for differences in changes over time in the probability of treatment and health outcomes.
The research team provided guidance about how to select an appropriate case-only design to best address the research question of interest, corresponding statistical assumptions, and data considerations.
Doctors and patients provided input that helped in designing the study.
The new methods are only appropriate for categorical treatment status, such as a binary indicator for receiving a particular treatment.
Conclusions and Relevance
The new methods calculate absolute risk, which can aid in clinical decision making. The methods guidance can help researchers select an optimal design and develop an analytic plan for case-only studies.
Future Research Needs
Future research could focus on developing methods for continuous treatment status, such as varying the treatment dosage.