• Download Summary

What was the project about?

Researchers often combine patient health data from different sources, such as claims and health records, to get a fuller picture of patients’ health. To combine these data, researchers use personal information, or PI, such as names and social security numbers.

Using PI to link data can put patient privacy at risk. Researchers can use computer software that hides PI to protect privacy, but they must make decisions about how much PI to hide. For example, they must decide how much PI to look at or how many records to review to make sure data are linked accurately.

In this project, the research team created and tested a new user interface called MiNDFIRL. MiNDFIRL can be used with record linkage, or RL, software to help researchers use less PI while maintaining accuracy.

What did the research team do?

The research team got input from RL experts and patients to design features of MiNDFIRL. The team then led two large studies with data analysts:

• In study 1, the team looked at how hiding different amounts of PI affected analysts’ ability to accurately link data.
• In study 2, the team tested ways to use less PI. For example, one way was to hide all PI but let analysts click to see only important parts of the PI. Another way had an image of a meter that showed how privacy risk increases when analysts use more PI to link data sets.

The research team also tested MiNDFIRL in two case studies with 12 data analysts at two medical schools. The case studies checked how much PI was needed to accurately link real patient health data. The team also tested if using MiNDFIRL increased the amount of time analysts took to link data.

What were the results?

The studies showed that hiding more PI made it harder to accurately link data. They also showed which features helped data analysts accurately link the data using the least amount of PI. Adding these features reduced the use of PI from 100 percent to 8 percent with the same level of accuracy. In the two case studies, MiNDFIRL helped analysts link real data using only 30 percent of PI. Using MiNDFIRL didn’t increase the time analysts took to link data.

What were the limits of the project?

MiNDFIRL can only be used with RL software. Future studies could add software so that MiNDFIRL can be used by itself to link data.

How can people use the results?

Researchers can consider using MiNDFIRL with RL software to help researchers accurately link patient data while protecting patient privacy.

Background

Record linkage (RL)—linking a patient’s records across multiple databases such as electronic health records (EHRs) and insurance claims—can help improve comparative effectiveness research by providing a more complete picture of a patient’s health. But accurately linking patient records requires using personally identifiable information, such as names and social security numbers, which raises concerns about patient privacy. Statistical methods can mask personally identifiable patient data, but use of these data can decrease RL accuracy. Researchers also face challenges in using RL, including the need to manually review complex linkages, refine linkage models, and verify RL accuracy.

To facilitate effective use of RL, researchers can benefit from software support that can balance accuracy and patient privacy. Software with effective user interfaces can also help researchers identify choices that increase privacy risks.

Objective

To design effective user interfaces that can support human decision making in RL to facilitate accurate linking of patient records while protecting patient privacy

Study Design

Methods

The research team developed an open-source user interface prototype software called MiNDFIRL, or Minimum Necessary Disclosure For Interactive Record Linkage, through iterative processes of design, development, and evaluation. The team obtained input for different software features from multiple studies conducted with diverse stakeholders, including RL experts, researchers who use RL, clinicians, privacy experts, and patients, as well as from two case studies with data analysts at medical schools.

The team also conducted two large-scale controlled experiments that examined how different designs affect data analysts’ ability to manually link records accurately.

• Study 1 examined the effect of varying levels of masking of personally identifiable information on data analysts’ ability to manually link records accurately.
• Study 2 examined the effect of a clickable interactive interface to promote ethical behavior in RL decision making. For example, the interface displayed privacy risks in different ways, such as a meter that showed the potential increase in risk when an additional, personally identifiable data field was disclosed

Results

The two controlled experiments demonstrated that MiNDFIRL can substantially limit the amount of personally identifiable information during manual review (7.85% compared with 100% disclosure) without negatively affecting the quality of RL or completion time. Results from the expert review showed that allowing analysts some access to personally identifiable information yielded better RL accuracy compared with no access, and better patient privacy compared with access to all personally identifiable information. Both case studies showed that allowing analysts access to 30% of the personally identifiable information was sufficient to support their decision making in RL.

Limitations

MiNDFIRL is a prototype software and needs to be fully developed for use in research. The research team did not investigate automated RL algorithms.

Conclusions and Relevance

MiNDFIRL can help protect patient privacy by assisting researchers in using the minimum amount of information necessary to accurately link records during human decision making in RL.

Future Research Needs

Future research could focus on incorporating automated RL algorithms to fully develop MiNDFIRL into a comprehensive RL system.