Results Summary

What was the research about?

Many healthcare systems use electronic health records. Researchers use data from these records in their studies. Some records have missing or incorrect data. When this happens, people might not be able to trust a study’s results. The research team wanted to:

  • Create guidance to judge whether data that a study used were high quality
  • Find new ways to display the quality of data
  • Learn why researchers don’t always report the quality of data that they used in studies

What were the results?

The research team developed guidance to help people judge whether data in research studies were high quality. The guidance included ways to report quality. High-quality data are complete, believable, and reliable.

The research team found that the guidance helped researchers from six large healthcare systems judge and report data quality from electronic health records.

The study members created new ways to show data quality in pictures or graphs.

The research team found that cost, time, and lack of guidance were the primary reasons that researchers did not report on data quality.

Who was in the study?

About 100 people joined the study. The study members included healthcare workers, patient advocates, and policy makers. They also included project managers, people who work with healthcare data, and researchers. All the people in the study were interested in the quality of data that research studies use.

What did the research team do?

The research team held two in-person meetings and monthly online meetings for the study members. The team then used information from these meetings to write guidance about how to measure data quality. The team made sure that all study members agreed on the guidance.

The team used a website to ask for feedback from other people interested in data quality about the new guidance. The team tested the guidance using electronic health records from six large healthcare systems.

The research team asked study members for ideas about pictures and graphs that could show data quality. The research team also surveyed other researchers to find out what kept them from reporting on data quality.

What were the limits of the study?

The study included about 100 people interested in the quality of data used in research studies. Other people may have different ideas about looking at data quality.

People who took part in the study were interested in the use of electronic health records for research. The study didn’t include people who use other types of research data, such as data from science laboratories or from social media. People who use other types of data may have different ideas about reporting on data quality.

How can people use the results?

Having common guidance about measuring and reporting the quality of research data can help people understand whether data that studies used are high quality and trustworthy. Figuring out why researchers don’t report the quality of their data may lead to new ideas about how to better share the quality of data with everyone.

Final Research Report

View this project's final research report.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Reviewers’ comments and the investigator’s changes in response included the following:

  • The awardee provided more information about the Data Quality Collaborative (DQC) and its work in identifying key data quality recommendations.
  • Based on reviewer recommendations, the awardee highlighted key study results involving harmonized data quality terms and recommendations by reorganizing the report by the three distinct categories of study findings.
  • The reviewers requested that the investigator clarify the description of the factor analyses completed on the survey data, including replacing a more-technical table with a more-intuitive figure.
  • The awardee added a discussion of the relevance and potential impact of this study on patient-centered outcomes research. The results  improved the ability to assess data quality of a specific data set.

Conflict of Interest Disclosures

Project Information

Michael G. Kahn, MD, PhD
University of Colorado Denver
$1,002,995 *
Building PCOR Value and Integrity with Data Quality and Transparency Standards

Key Dates

September 2013
July 2017

Study Registration Information

Final Research Report

View this project's final research report.

Journal Articles


Has Results
Award Type
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: January 20, 2023