Results Summary

What was the research about?

Researchers can use data from health registries or electronic health records to compare two or more treatments. Registries store data about patients with a specific health problem. These data include how well those patients respond to treatments and information about patient traits, such as age, weight, or blood pressure. But sometimes data about patient traits are missing.

Missing data about patient traits can lead to incorrect study results, especially when traits change over time. For example, weight can change over time, and the patient may not report their weight at some points along the way. Researchers use statistical methods to fill in these missing data.

In this study, the research team compared a new statistical method to fill in missing data with traditional methods. Traditional methods remove patients with missing data or fill in each missing number with a single estimate. The new method creates multiple possible estimates to fill in each missing number.

What were the results?

The new method worked better than traditional methods. Patients with certain traits were more likely to have missing data. Accounting for such traits in the analysis made the new method work even better.

What did the research team do?

The research team first developed computer software, which created test data that mimicked a study about patients living with HIV. The data included patient traits that change over time and are often missing, such as body mass index, a measure of weight and height. The team used the test data to compare the new method of filling in missing data with traditional methods.

What were the limits of the study?

When the research team compared the methods, they looked at the test data only in certain situations. For example, because patients with certain traits were more likely to have missing data, they compared results with and without those patients. Findings may differ if the team looked at the test data in other situations.

Future research could compare methods for handling missing data in other situations.

How can people use the results?

Researchers can use the new method to get results that are more accurate in studies with missing data.

Final Research Report

View this project's final research report.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented, and the researchers made changes or provided responses. The comments and responses included the following:

  • Reviewers requested additional information about the missingness models, simulations, and model assumptions. The reviewers noted that it was hard to keep track of the specifics of each simulation study and the results obtained. The researchers added an appendix including a table that summarizes the three main simulation studies, their assumptions, the methods evaluated, and the findings.
  • Reviewers asked for additional details about the scenarios used to test the model simulations, to help readers better understand the limitations to the generalizability of these models. The researchers responded by adding discussion of the proportion of missing records and the missing data mechanism. However, they disagreed with the reviewers’ contention that the discrepancies between simulated and observed data limited the generalizability of their findings regarding performance of missing data methods.
  • Reviewers noted that the simulations of time-dependent categorical variables did not appear to perform well and asked for additional discussion of the impact of this poor performance on conclusions that could be made from these simulations. The researchers acknowledged that the performance of the simulation tool was not optimal in generating covariates but did not consider this an important issue because the simulation of survival times was excellent. They also noted excellent recovery of true parameter estimates of the relationship between the time-dependent covariates and outcome, which are critical to gauging how well missing data methods handle time-varying covariates.

Conflict of Interest Disclosures

Project Information

Manisha Desai, PhD
Stanford University
$796,625
The Handling of Missing Data Induced by Time-Varying Covariates in Comparative Effectiveness Research Involving HIV Patients

Key Dates

September 2013
January 2019
2013
2018

Study Registration Information

Tags

Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: January 20, 2023