Skip to main content
Patient-Centered Outcomes Research Institute
Patient-Centered Outcomes Research Institute
  • Blog
  • Newsroom
  • Find It Fast
  • Help Center
  • Subscribe
  • Careers
  • Contact Us

PCORI

Patient-Centered Outcomes Research Institute

Search form

  • About Us
    Close mega-menu

    About Us

    • Our Programs
    • Governance
    • Financials and Reports
    • Procurement Opportunities
    • Our Staff
    • Our Vision & Mission
    • Contact Us

    Fact Sheets: Learn More About PCORI

    Download fact sheets about out work, the research we fund, and our programs and initiatives.

    Find It Fast

    Browse through an alphabetical list of frequently accessed and searched terms for information and resources.

    Subscribe to PCORI Email Alerts

    Sign up for weekly emails to stay current on the latest results of our funded projects, and more.

  • Research & Results
    Close mega-menu

    Research & Results

    • Explore Our Portfolio
    • Research Fundamentals
    • Research Results Highlights
    • Putting Evidence to Work
    • Peer Review
    • Evidence Synthesis
    • About Our Research

    Evidence Updates from PCORI-Funded Studies

    These updates capture highlights of findings from systematic reviews and our funded research studies.

    Journal Articles About Our Funded Research

    Browse through a collection of journal publications that provides insights into PCORI-funded work.

    Explore Our Portfolio of Funded Projects

    Find out about projects based on the health conditions they focus on, the state they are in, and if they have results.

  • Topics
    Close mega-menu

    Topics

    • Addressing Disparities
    • Arthritis
    • Asthma
    • Cancer
    • Cardiovascular Disease
    • Children's Health
    • Community Health Workers
    • COVID-19
    • Dementia and Cognitive Impairment
    • Diabetes
    • Kidney Disease
    • Medicaid
    • Men's Health
    • Mental and Behavioral Health
    • Minority Mental Health
    • Multiple Chronic Conditions
    • Multiple Sclerosis
    • Obesity
    • Older Adults' Health
    • Pain Care and Opioids
    • Rare Diseases
    • Rural Health
    • Shared Decision Making
    • Telehealth
    • Transitional Care
    • Veterans Health
    • Women's Health

    Featured Topic: Women's Health

    Learn more about the projects we support on conditions that specifically or more often affect women.

  • Engagement
    Close mega-menu

    Engagement

    • The Value of Engagement
    • Engagement in Health Research Literature Explorer
    • Influencing the Culture of Research
    • Engagement Awards
    • Engagement Resources
    • Engage with Us

    Engagement Tools and Resources for Research

    This searchable peer-to-peer repository includes resources that can inform future work in patient-centered outcomes research.

    Explore Engagement in Health Literature

    This tool enables searching for published articles about engagement in health research.

    Research Fundamentals: A New On-Demand Training

    It enables those new to health research or patient-centered research to learn more about the research process.

  • Funding Opportunities
    Close mega-menu

    Funding Opportunities

    • What & Who We Fund
    • What You Need to Know to Apply
    • Applicant Training
    • Merit Review
    • Awardee Resources
    • Help Center

    PCORI Funding Opportunities

    View and learn about the newly opened funding announcements and the upcoming PFAs in 2021.

    Tips for Submitting a Responsive LOI

    Find out what PCORI looks for in a letter of intent (LOI) along with other helpful tips.

    PCORI Awardee Resources

    These resources can help awardees in complying with the terms and conditions of their contract.

  • Meetings & Events
    Close mega-menu

    Meetings & Events

    • Upcoming
    • Past Events

    January 2021 Board of Governors Meeting

    The Board approved funding for a new research study relating to kidney health and a new funding allocation for PCORnet. Learn more

    Confronting COVID-19: A Webinar Series

    Learn more about the series and access recordings and summary reports of all six sessions.

    2020 PCORI Annual Meeting

    Watch recordings of all sessions, and view titles and descriptions of the posters presented at the virtual meeting.

You are here

  • Research & Results
  • Explore Our Portfolio
  • Statistical Methods to Preserve Patie...

This project has results

Statistical Methods to Preserve Patient Privacy When Sharing and Analyzing Data

Sign Up for Updates to This Study  

Results Summary and Professional Abstract

Results Summary

Results Summary

Download Summary

What was the research about?

Sometimes a study can get better results using data from different sites. In these cases, researchers may want to share patient data, including personal and private information such as dates of birth and addresses. However, researchers may not want to share data across sites because of worries about patient privacy. Some statistical methods can change patients’ sensitive individual data into summary data that hides individuals’ personal information. These privacy-protecting methods, or PPMs, make it safe to share data across sites. But researchers don’t know if PPMs produce accurate results.

In this study, the research team compared combinations of PPMs with methods that use patients’ individual data.

What were the results?

PPMs provided similar results to methods that used patients’ individual data. Some PPMs worked better than others.

What did the research team do?

The research team used a computer program to create data that mimicked real patient data. The team used three ways to change the individual data into summary data. Then they analyzed the summary data using a combination of PPMs. The team compared the results they got using summary data with results from individual data. Finally, the team tested the PPMs using real-world data.

Patients, researchers, and hospital staff provided input during this study.

What were the limits of the study?

The research team studied only some PPMs and types of data. Results may differ with other PPMs or data.

Future research could test PPMs in other research studies with data from different sources.

How can people use the results?

Researchers can use the results when considering how to share data across research sites.

Professional Abstract

Professional Abstract

Objective

To assess the performance of privacy-preserving analytic methods in patient-centered outcomes research

Study Design

Design Element Description
Design Simulation studies, empirical analysis
Data Sources and Data Sets Simulated data to resemble comparative effectiveness research data, empirical data from the administrative and clinical databases of Kaiser Permanente & Strategic Partners Patient Outcomes Research to Advance Learning (PORTAL) network
Analytic Approach Combination of 3 data sharing methods (risk-set, summary-table, effect-estimate), 2 confounding risk scores (propensity score and disease risk score), and multiple confounding adjustment approaches (matching, stratification, weighting)
Outcomes Treatment effect estimates for binary and survival outcomes

Privacy-preserving analytic methods offer new ways to address patient privacy, data security, data control, and regulatory requirements when collaborating across multiple research sites. These methods analyze summary-level patient data from research sites rather than individual-level patient data and may increase researchers’ willingness and ability to collaborate in multisite research studies. Summary-level data include the same patient information as individual-level data for statistical analysis, but the data are transformed and summarized to exclude sensitive information and preserve patient privacy.

In this study, the research team developed a set of privacy-preserving analytic methods made up of data sharing methods, confounding scores, and confounding adjustment approaches. The team compared the performance of privacy-preserving methods that use summary-level data with those using individual-level data.

The research team conducted multiple simulations to generate individual-level data that resembled patient data from an observational study. Next, the team transformed the individual-level simulated data to create three different summary-level data sets representing increasing levels of summarization: risk-set data, summary-table data, and effect-estimate data.

The research team then analyzed each data set using different privacy-preserving methods to estimate treatment effects on health outcomes. To control for confounding in the analyses, the team applied confounder summary scores such as propensity scores (PSs) and disease risk scores (DRSs). The team incorporated these scores into the analyses for confounding adjustment via matching, stratification, or weighting. To evaluate the performance of the various privacy-preserving methods on survival outcomes, the team compared the treatment effect estimates with the true treatment effect used to generate the simulated data.

The research team also tested the performance of the privacy-preserving methods using real data from two comparative effectiveness studies on obesity and rheumatoid arthritis from three sites within a clinical data research network. The team applied the same analytic methods used with the simulated data.

To design and implement this study, the research team worked with patients, health system administrators, other researchers, and experts in governance and regulatory compliance.

Results

Both simulation and empirical studies showed that the privacy-preserving methods produced results that were identical or highly comparable to results obtained from individual-level data analysis in most of the scenarios examined.

Simulation studies

  • Matching and weighting, when used to adjust for confounding, produced less biased results than stratification in all three summary-level data analyses.
  • In simulation scenarios with outcomes that are uncommon, PS-based methods performed slightly better than DRS-based methods in validity (when compared with the true treatment effect) and precision of point estimates.
  • All privacy-preserving methods produced similar results to those from the individual-level data analyses with slight variation in PS-based analysis of effect estimates in settings with infrequent exposure.

Empirical analysis

  • The confounding adjustment approach affected the comparability between the summary-level data analyses and the individual-level data analyses.
  • Risk-set data sharing generally performed better in estimating the treatment effect, while summary-table and effect-estimate data sharing more often produced variation in settings with outcomes that are uncommon and small sample sizes.

Limitations

The research team did not examine other privacy-preserving methods, such as the distributed regression approach. Lack of homogeneity in the data can affect the comparability of the summary data across multiple sites and could affect the decision to combine site-level data.

Conclusions and Relevance

Privacy-preserving methods using summary-level data do not affect the bias and precision of treatment effect estimates compared with results of pooled individual-level data analyses. These methods may be an alternative to sharing individual-level data when privacy concerns arise in multisite research studies.

Future Research Needs

Future research could assess the performance of privacy-preserving methods in studies with more sites and partitioned data on the same patient from different databases.

Final Research Report

View this project's final research report.

Journal Articles

Related Articles

BMC Medical Research Methodology

Privacy-protecting estimation of adjusted risk ratios using modified Poisson regression in multi-center studies

Statistical Methods in Research

Inverse probability weighted Cox model in multi-site studies without sharing individual-level data

American Journal of Epidemiology

Validity of Privacy-Protecting Analytical Methods That Use Only Aggregate-Level Information to Conduct Multivariable-Adjusted Analysis in Distributed Data Networks

Pharmacoepidemiology and Drug Safety

Comparison of privacy-protecting analytic and data-sharing methods: A simulation study

Journal of Comparative Effectiveness Research

Stakeholders' views on data sharing in multicenter studies

More on this Project  

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented, and the researchers made changes or provided responses. The comments and responses included the following:

  • Reviewers raised concerns about how representative the viewpoints of the stakeholders selected for interviews could be expected to be. In particular, reviewers noted that the policies and procedures of the institutional stakeholders’ organization, Kaiser Permanente, could influence the institutional stakeholders’ views. The researchers stated that they did not believe organizational policies or practices would impact more scientific discussions. However, they addressed the reviewers’ concerns by discussing the selective nature of their stakeholders and the potential impact on their Aim 1 findings in their limitations section.
  • Reviewers suggested adding a brief summary of the strengths and limitations of each methodological approach described to stakeholders. The researchers added a table to the discussion section summarizing the strengths and limitations of the analytic approaches considered in this study. The table also provides ranks of the various methods under different scenarios, so that it can guide stakeholders who might be choosing among these methods for their own analyses.
  • Reviewers asked the researchers to provide more of a summary of qualitative findings from their interviews with stakeholders and patients. The researchers explained that they would have liked to be able to summarize findings, but there was considerable variability between and within different stakeholder groups regarding priorities and preferences for data sharing. The researchers felt that this variability was an interesting finding in itself.
  • Reviewers suggested adding a glossary of key terms, primarily noting the interchangeable use of the terms, privacy and confidentiality. The researchers instead aimed to improve clarity by largely omitting the term, confidentiality, assuming other terms were less confusing.

Conflict of Interest Disclosures

View the COI disclosure form.

Videos

Patients Fill Research Holes (below)
Patients and other stakeholders help researchers ask important questions and find critical information they might otherwise miss.

Project Details

Principal Investigator
Darren Toh, ScD, BPHAR, MS
Project Status
Completed; PCORI Public and Professional Abstracts, and Final Research Report Posted
Project Title
Privacy-Preserving Analytic and Data-Sharing Methods for Clinical and Patient-Powered Data Networks
Board Approval Date
September 2014
Project End Date
April 2019
Organization
Harvard Pilgrim Health Care, Inc.
Year Awarded
2014
State
Massachusetts
Year Completed
2018
Project Type
Research Project
Health Conditions  
Allergies and Immune Disorders
Immune Disorders
Muscular and Skeletal Disorders
Arthritis
Nutritional and Metabolic Disorders
Obesity
Intervention Strategies
Drug Interventions
Other Clinical Interventions
Funding Announcement
Improving Methods for Conducting Patient-Centered Outcomes Research
Project Budget
$1,420,273
DOI - Digital Object Identifier
10.25302/06.2020.ME.140311305
Study Registration Information
HSRP20152296
Page Last Updated: 
August 28, 2020

About Us

  • Our Programs
  • Governance
  • Financials and Reports
  • Procurement Opportunities
  • Our Staff
  • Our Vision & Mission
  • Contact Us

Research & Results

  • Explore Our Portfolio
  • Research Fundamentals
  • Research Results Highlights
  • Putting Evidence to Work
  • Peer Review
  • Evidence Synthesis
  • About Our Research

Engagement

  • The Value of Engagement
  • Engagement in Health Research Literature Explorer
  • Influencing the Culture of Research
  • Engagement Awards
  • Engagement Resources
  • Engage with Us

Funding Opportunities

  • What & Who We Fund
  • What You Need to Know to Apply
  • Applicant Training
  • Merit Review
  • Awardee Resources
  • Help Center

Meetings & Events

January 21
Cycle 1 2021 Broad PFA Applicant Town Hall
February 2
PCORI 2021 and Beyond: Opportunities for Funding and Involvement in Patient-Centered Research
February 9
Board of Governors Meeting: February 9, 2021

PCORI

Footer contact address

Patient-Centered Outcomes
Research Institute

1828 L Street, NW, Suite 900
Washington, DC 20036
Phone: (202) 827-7700 | Fax: (202) 355-9558
[email protected]

Subscribe to Newsletter

Twitter Facebook LinkedIn Vimeo

© 2011-2021 Patient-Centered Outcomes Research Institute. All Rights Reserved.

Privacy Policy | Terms of Use | Trademark Usage Guidelines | Credits | Help Center