Final Research Report

View this project's final research report.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:

  • The reviewers commented that the report seemed to present a predictive model for hemoglobin A1c (HbA1c) improvements based on analyses from electronic health records. The reviewers expressed concern because electronic health record data would be prone to potential bias because of unmeasured confounders and missing data for predictors and outcomes. The researchers revised their description of the electronic health records analysis to clarify that their probability models were meant to be explanatory rather than predictive. In other words, they were answering the question, “What happened?” rather than, “What will happen?”
  • The reviewers asked whether the researchers tested their probability model for external validity, or the application of the model in a new population to see how well it predicts the same outcome. The researchers initially aimed to test external validity by applying the model they built using data from the randomized controlled trial to community data from patients’ electronic health records. The reviewers pointed out that the two data sets were not equivalent: the trial participants were more homogenous because of strict trial inclusion criteria compared to the community participants. In addition, the follow-up periods for the randomized controlled trial were more consistent, with shorter intervals, than the follow-up periods for the data collected from community electronic health records. The researchers therefore built a separate probability model for reaching the desired HbA1c levels for community patients using data from the electronic health records cohort of patients. The researchers tested neither probability model with a new population, and they identified the lack of testing as a limitation of the study.
  • The reviewers noted that the researchers did not follow current standards for missing data when they used the most recent preceding available observation of HbA1c, or the Last Observation Carried Forward, to replace missing HbA1c data at the study’s endpoint. The researchers contended that this technique was valid in this situation because HbA1c is a summary measure of glycemic control over a three-month period and that this technique was conservative because HbA1c was likely to have improved if the study endpoint measure was collected. The reviewers disagreed with this contention and requested the researchers to acknowledge that their methods to address missing data were potentially biased, which was a limitation of the study.

Conflict of Interest Disclosures

Project Information

Marcia Anne Testa, BA, MPH, MPHIL, PhD
Harvard University School of Public Health
$2,025,244
10.25302/03.2021.CE.13046756
Benchmarking the Comparative Effectiveness of Diabetes Treatments Using Patient-Reported Outcomes and Socio-Demographic Factors

Key Dates

September 2013
February 2021
2013
2021

Study Registration Information

Tags

Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
Populations Populations PCORI is interested in research that seeks to better understand how different clinical and health system options work for different people. These populations are frequently studied in our portfolio or identified as being of interest by our stakeholders. View Glossary
Intervention Strategy Intervention Strategies PCORI funds comparative clinical effectiveness research (CER) studies that compare two or more options or approaches to health care, or that compare different ways of delivering or receiving care. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: March 4, 2022