• Download Summary

What was the project about?

Electronic health records, or EHRs, have data on patient traits, health problems, and treatments. Researchers can use EHR data to study how treatments work or which patient traits affect health outcomes. But EHR data can have errors.

The best way to get accurate EHR data is to closely review patients’ original records. But reviewing all patient records isn’t possible when many patients are in a study. In such cases, researchers can review and correct records for a few patients and use the revised records to adjust data for all patients. But existing methods for using revised records don’t address some kinds of errors, such as errors that are related. For example, errors in a treatment starting date can lead to mistakes in the data on length of treatment.

In this project, the research team created and tested new methods to improve the accuracy of EHR data. The new methods corrected records from some patients. Then the team used the corrections to address related errors for all patients.

What did the research team do?

The research team reviewed and corrected a sample of records from an EHR data set. Then the team used the revised records to create four new methods for fixing errors in EHR data that weren’t reviewed.

Next, the research team created test data using a computer program. The team compared how well the four methods worked to improve the accuracy of results using the test data. The team also made a computer program to help other researchers use the new methods.

Lastly, the research team applied one method, called Raking, to real EHR data. The team looked at whether weight gain during pregnancy was related to children’s obesity. The team used real EHR data from 10,335 children and their mothers and reviewed 996 EHRs. The team analyzed the data with and without the Raking method to see how well it corrected data errors.

Patients, caregivers, and doctors gave input throughout the study.

What were the results?

Using test data, all four methods improved the accuracy of EHR data. The Raking method did a good job correcting data errors while making fewer assumptions than the other methods.

With real EHR data, when data weren’t corrected using Raking, a child whose mother gained 10 kilograms (22 pounds) more than recommended during pregnancy had a 24 percent increase in risk of childhood obesity. After using the Raking method to correct data errors, the risk of childhood obesity was 30 percent. When using data that were likely to have errors, Raking helped improve the accuracy of results.

What were the limits of the project?

The new methods can correct only some errors in EHR data. For example, the methods can’t correct errors like mistakes in diagnosis codes.

Future research could develop additional methods that could work for different types of errors.

How can people use the results?

Researchers can use the new methods to improve the accuracy of results in studies using EHR data.

Background

Researchers can use data from electronic health records (EHRs) to study health outcomes. However, errors in these data may lead to inaccurate study results.

The gold standard for ensuring data accuracy is data validation, in which trained personnel compare EHR-derived data samples with the original source documents, such as medical charts. However, validating all patient records is time consuming and not feasible for large samples. In such cases, researchers can validate data for a subset of records and use the validated data to adjust estimates in the larger unvalidated data set. But existing methods do not address correlated errors between some variables. For example, errors in treatment initiation date could generate errors in treatment duration, leading to biased results.

Objective

To develop and test statistical methods that address errors and improve the accuracy of EHR data

Study Design

Methods

The research team developed methods to address correlated errors in statistical analyses of EHR data. To do this, the team first manually validated data for a subsample of patients. The team then developed four new methods using the validated data to reduce bias caused by correlated errors:

• Multiple imputation (MI)
• Regression calibration (RC)
• Generalized raking (Raking)
• Sieve maximum likelihood estimation (SMLE)

Then the team conducted simulations to compare the robustness and efficiency of the four methods and created open-source software for all four methods.

Based on the simulation results, the research team further tested the Raking method using real patient data from 10,335 mother–child pairs, of which 996 pairs were validated by chart review, looking at whether weight gain during pregnancy predicted risk for childhood obesity.

Patients, caregivers, and doctors gave input throughout the study.

Results

With simulated data, all four methods helped correct correlated errors in outcome and exposure variables, but performance varied across scenarios. MI worked well in complicated error settings, such as correlated errors in multiple variables, but required correct model specification. RC and SMLE helped address errors in some, but not all, scenarios. Although less efficient, Raking was the most robust method.

Using Raking with validated patient data, the research team found that a 10-kilogram weight gain above the recommended amount during pregnancy was associated with a 30% increased risk of childhood obesity, versus a 24% increased risk in analyses that only used unvalidated data. With Raking, error-prone variables, such as the mother’s smoking and insurance status, had a stronger association with childhood obesity.

Limitations

The methods may not work if validated data include errors from incorrect data entry or erroneous diagnoses. The methods do not address other challenges with EHR data, such as confounding.

Conclusions and Relevance

Researchers and healthcare administrators can use this study’s methods to improve EHR data quality and reduce bias in studies using EHR data.

Future Research Needs

Future studies could test the methods with data from different EHR systems, which may vary in data availability and missingness. Researchers could also develop new methods that work better with specific types of errors in data.