Results Summary

What was the project about?

Electronic health records, or EHRs, have data on patient traits, health problems, and treatments. Researchers can use EHR data to study how treatments work or which patient traits affect health outcomes. But EHR data can have errors.

The best way to get accurate EHR data is to closely review patients’ original records. But reviewing all patient records isn’t possible when many patients are in a study. In such cases, researchers can review and correct records for a few patients and use the revised records to adjust data for all patients. But existing methods for using revised records don’t address some kinds of errors, such as errors that are related. For example, errors in a treatment starting date can lead to mistakes in the data on length of treatment.

In this project, the research team created and tested new methods to improve the accuracy of EHR data. The new methods corrected records from some patients. Then the team used the corrections to address related errors for all patients.

What did the research team do?

The research team reviewed and corrected a sample of records from an EHR data set. Then the team used the revised records to create four new methods for fixing errors in EHR data that weren’t reviewed.

Next, the research team created test data using a computer program. The team compared how well the four methods worked to improve the accuracy of results using the test data. The team also made a computer program to help other researchers use the new methods.

Lastly, the research team applied one method, called Raking, to real EHR data. The team looked at whether weight gain during pregnancy was related to children’s obesity. The team used real EHR data from 10,335 children and their mothers and reviewed 996 EHRs. The team analyzed the data with and without the Raking method to see how well it corrected data errors.

Patients, caregivers, and doctors gave input throughout the study.

What were the results?

Using test data, all four methods improved the accuracy of EHR data. The Raking method did a good job correcting data errors while making fewer assumptions than the other methods.

With real EHR data, when data weren’t corrected using Raking, a child whose mother gained 10 kilograms (22 pounds) more than recommended during pregnancy had a 24 percent increase in risk of childhood obesity. After using the Raking method to correct data errors, the risk of childhood obesity was 30 percent. When using data that were likely to have errors, Raking helped improve the accuracy of results.

What were the limits of the project?

The new methods can correct only some errors in EHR data. For example, the methods can’t correct errors like mistakes in diagnosis codes.

Future research could develop additional methods that could work for different types of errors.

How can people use the results?

Researchers can use the new methods to improve the accuracy of results in studies using EHR data.

Final Research Report

This project's final research report is expected to be available by May 2023.

Journal Citations

Related Journal Citations

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:

  • The reviewers applauded the high quality of this study report but requested that the researchers summarize some of the more technical details for a general clinical scientist. The researchers added non-technical summary tables to multiple technical chapters.
  • The reviewers asked the researchers to explain some elements of their approach, such as use of time-to-event data as well as the use of the missing-at-random assumption. The researchers explained that their research was meant to solve problems created by error-prone time-to-event data. They also explained that they assumed the validation sample in their work was a probabilistic sample and therefore missing data could be expected to be missing at random.

Conflict of Interest Disclosures

Project Information

Bryan E. Shepherd, PhD; Pamela A. Shaw, PhD
Vanderbilt University Medical Center
$1,050,000
Statistical Methods and Designs for Addressing Correlated Errors in Outcomes and Covariates in Studies Using Electronic Health Records Data

Key Dates

August 2016
December 2022
2017
2022

Study Registration Information

Tags

Has Results
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: January 12, 2023