Results Summary

What was the project about?

Researchers often use data from patients’ health records to compare treatments. But many things—not just treatments—affect patients’ health. To figure out whether changes in patients’ health result from treatment or something else, researchers can use statistical methods called instrumental variables, or IVs. IV methods account for factors that affect health but aren’t in patients’ health records, such as eating habits. Existing IV methods work well when looking at health outcomes that are measured using certain types of scales, such as blood pressure. But existing methods don’t work as well to measure the time until a health event occurs, particularly when an event, like death, has not occurred for many patients in the study.

In this study, the research team created and tested a new IV method to more accurately estimate how a treatment relates to the time until a health event.

What did the research team do?

The research team created 2SRI-F, a new IV method. The team then created a test data set to mimic patient health records. The test data set included information on patients’ health and treatments. Using the test data, the team compared findings using the new IV method with findings from existing methods.

The research team also tested the new method in three studies using data from patients with blood vessel diseases who had health records in the Vascular Quality Initiative, or VQI. Studies 1 and 2 looked at the effect of treatment type on time to death. Study 3 estimated the effect of treatment on time until having an amputation or until a patient had to start treatment again. In all three studies, the research team compared findings from the 2SRI-F method to those from existing methods.

Patients, physicians, and the VQI director helped the research team design the study.

What were the results?

Compared with existing methods, the 2SRI-F method had more accurate results. The results from the 2SRI-F method were more similar to results found in randomized trials. Researchers consider these trials the best way to compare how well treatments work.

What were the limits of the project?

Current computer programs cannot make estimates for the new methods that are as precise as researchers would like. These programs are not designed to run the 2SRI-F method.

Future research could improve computer programs to analyze data using the 2SRI-F method.

How can people use the results?

Researchers can use the results when designing studies using data from patient health records.

Final Research Report

View this project's final research report.

Journal Citations

Related Journal Citations

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:

  • The reviewers noted that other methods would be useful to examine via simulations, and that the researchers should present these other methods or discuss them at the end of the report. The researchers responded that their principal comparison approaches focused on the Cox (proportional hazards) regression model because it is heavily used and therefore, the report would be of more interest to readers. The reviewers pointed out that the researchers mentioned less commonly used methods, including the accelerated failure time model. In response to reviewer comments, the researchers added references to this model in their discussion section.
  • For the simulation results, the reviewers suggested adding information about root mean square errors (RMSE), which is a way of comparing predicted values against observed values. The researchers acknowledged that they did not report RMSEs for all simulations because they considered bias to be a primary concern, with statistical precision being secondary. The researchers did add appendix tables with results that provided RMSE for the simulations.
  • The reviewers asked for additional details to allow readers to replicate simulation results after one reviewer failed to replicate the results. The researchers explained that they believed the simulations the reviewer performed did not work because the reviewer did not run analyses for the Cox model. The researchers noted that their code is freely available on GitHub and use of this code should allow for perfectly reproduced simulations.

Conflict of Interest Disclosures

Project Information

Alistair James O’Malley, BSc (Hons), MS, PhD
Dartmouth College
Advancing Patient Centered Outcomes Research in Survival Data with Unmeasured Confounding to Improve Patient Risk Communication

Key Dates

September 2015
January 2020

Study Registration Information


Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
Intervention Strategy Intervention Strategies PCORI funds comparative clinical effectiveness research (CER) studies that compare two or more options or approaches to health care, or that compare different ways of delivering or receiving care. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: April 11, 2024