Results Summary

What was the research about?

Comparative effectiveness research compares two or more treatments to see which one works better for which patients. Electronic healthcare data are useful for this type of research. These data come from medical records and insurance claims. The data include information about how well patients respond to treatments. But many things—not just treatments—affect whether a patient’s health improves.

How well a patient responds to a treatment may depend on the patient’s age or what medicines the patient takes. It could also depend on what other health problems a patient has and how severe those problems are. Or a doctor may suggest one treatment instead of another because of a patient’s personal situation and health. Researchers need ways to determine whether changes in a patient’s health result from a certain treatment or something else.

Different statistical methods help researchers account for the various things that can affect treatment results. But researchers don’t know which methods work best. This study compared several methods. The team looked at how well the methods worked to predict patients’ responses to treatment, taking into account their personal situations and health. The team then created a computer program to help researchers use the methods.

What were the results?

No statistical method worked best in all cases to predict how well patients responded to treatment, taking into account their personal situations and health. Each method had pros and cons. But the research team found that one combination of two methods worked well in many cases.

What did the research team do?

The team used electronic healthcare data from three studies. The researchers first used the data from the studies to test several statistical methods. Then, they made more data sets based on the real data to test the methods further. The team created a computer program to help researchers use the methods.

During the study, patients gave input to the research team about key problems they have had in getting health care and what questions they see as most important for research to answer.

What were the limits of the study?

The research team could not say for sure which method will work best in specific cases to account for the things that could affect treatment results. Future research could continue to look at which methods might work best when doing a study using electronic healthcare data.

How can people use the results?

Researchers can use the results to understand which statistical methods might be useful when doing a study using electronic healthcare data. The software that the team developed can help researchers use the statistical methods in their research.

Final Research Report

View this project's final research report.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.

Peer review identified the following strengths and limitations in the report:

  • Reviewers commended the researchers on a methodologically sound study focused on improving statistical methods for confounder adjustment needed for causal inference models in patient-centered outcomes research. Many of the comments were requests for investigators to provide explanations of their highly-technical methods and results in language that would be understood by the general clinical researcher.
  • Reviewers questioned the generalizability of the statistical approaches tested in this study, as simulation studies are often plagued with limited generalizability. The researchers acknowledged this problem, but responded that they used three different datasets for the simulations, which would improve the overall generalizability. One of the specific advantages of their simulation approach, the authors stated, was the preservation of the complex relationships among baseline covariates, which would not be possible in other simulation frameworks.

Conflict of Interest Disclosures

Project Information

Sebastian Schneeweiss, MD, MS, ScD
Brigham and Women's Hospital
$1,080,576 *
Causal Inference for Effectiveness Research in Using Secondary Data

Key Dates

September 2013
July 2018

Study Registration Information

Final Research Report

View this project's final research report.

Journal Articles


Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
Intervention Strategy Intervention Strategies PCORI funds comparative clinical effectiveness research (CER) studies that compare two or more options or approaches to health care, or that compare different ways of delivering or receiving care. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: January 20, 2023