Objective

To assess the performance of privacy-preserving analytic methods in patient-centered outcomes research

Study Design

Privacy-preserving analytic methods offer new ways to address patient privacy, data security, data control, and regulatory requirements when collaborating across multiple research sites. These methods analyze summary-level patient data from research sites rather than individual-level patient data and may increase researchers’ willingness and ability to collaborate in multisite research studies. Summary-level data include the same patient information as individual-level data for statistical analysis, but the data are transformed and summarized to exclude sensitive information and preserve patient privacy.

In this study, the research team developed a set of privacy-preserving analytic methods made up of data sharing methods, confounding scores, and confounding adjustment approaches. The team compared the performance of privacy-preserving methods that use summary-level data with those using individual-level data.

The research team conducted multiple simulations to generate individual-level data that resembled patient data from an observational study. Next, the team transformed the individual-level simulated data to create three different summary-level data sets representing increasing levels of summarization: risk-set data, summary-table data, and effect-estimate data.

The research team then analyzed each data set using different privacy-preserving methods to estimate treatment effects on health outcomes. To control for confounding in the analyses, the team applied confounder summary scores such as propensity scores (PSs) and disease risk scores (DRSs). The team incorporated these scores into the analyses for confounding adjustment via matching, stratification, or weighting. To evaluate the performance of the various privacy-preserving methods on survival outcomes, the team compared the treatment effect estimates with the true treatment effect used to generate the simulated data.

The research team also tested the performance of the privacy-preserving methods using real data from two comparative effectiveness studies on obesity and rheumatoid arthritis from three sites within a clinical data research network. The team applied the same analytic methods used with the simulated data.

To design and implement this study, the research team worked with patients, health system administrators, other researchers, and experts in governance and regulatory compliance.

Results

Both simulation and empirical studies showed that the privacy-preserving methods produced results that were identical or highly comparable to results obtained from individual-level data analysis in most of the scenarios examined.

Simulation studies

• Matching and weighting, when used to adjust for confounding, produced less biased results than stratification in all three summary-level data analyses.
• In simulation scenarios with outcomes that are uncommon, PS-based methods performed slightly better than DRS-based methods in validity (when compared with the true treatment effect) and precision of point estimates.
• All privacy-preserving methods produced similar results to those from the individual-level data analyses with slight variation in PS-based analysis of effect estimates in settings with infrequent exposure.

Empirical analysis

• The confounding adjustment approach affected the comparability between the summary-level data analyses and the individual-level data analyses.
• Risk-set data sharing generally performed better in estimating the treatment effect, while summary-table and effect-estimate data sharing more often produced variation in settings with outcomes that are uncommon and small sample sizes.

Limitations

The research team did not examine other privacy-preserving methods, such as the distributed regression approach. Lack of homogeneity in the data can affect the comparability of the summary data across multiple sites and could affect the decision to combine site-level data.

Conclusions and Relevance

Privacy-preserving methods using summary-level data do not affect the bias and precision of treatment effect estimates compared with results of pooled individual-level data analyses. These methods may be an alternative to sharing individual-level data when privacy concerns arise in multisite research studies.

Future Research Needs

Future research could assess the performance of privacy-preserving methods in studies with more sites and partitioned data on the same patient from different databases.