Results Summary

What was the research about?

Comparative effectiveness research compares two or more treatments to see which one works best for which patients. Information from health insurance claims could be useful for this type of research. These claims include data on how well patients respond to treatments. But many things—not just treatments—affect whether patients’ health improves.

How well patients respond to treatments could depend on patients’ ages or medicines they take. It could also depend on how many health problems a patient has and how severe the problems are. Also, a doctor may suggest one treatment instead of another because of a patient’s personal situation and health. Researchers need ways to figure out whether changes in patients’ health result from treatment or something else.

Comparing treatments is hard in small studies with only a few patients. When there are few patients in a study, researchers can study only a few events. An event is an outcome related to the health problem or treatment researchers are studying. When there are few events and many things that could affect treatment results, it is hard to figure out what causes changes in patients’ health. To address this problem, researchers use different statistical methods to account for all the things that could affect treatment results. But researchers don’t know which methods might work best in studies with few events. In this study, the research team compared several methods to see which ones worked best.

What were the results?

The research team found that certain statistical methods worked better than others to account for all the things that could affect treatment results in studies with few events.

What did the research team do?

The research team wanted to see which statistical methods worked best to account for things that could affect treatment results. To do this, the team made a test set of health insurance claims using real patient data. The team made sure the set had only a few events and many things other than treatment that could affect the results. The team also made sure the test set had information on what happened after each patient got treatment. The test set made it possible to see which methods worked best.

During the study, patients gave input to the research team about the issues that are important to them in research that uses health insurance claims.

What were the limits of the study?

This study compared different statistical methods using data created by the research team. Studies using different data may have different results. Also, the results may not apply to all types of data.

How can people use the results?

This study can help researchers understand which statistical methods to use when doing a study with few events and many things that could affect treatment results. Knowing which methods work best can help researchers use health insurance claims to get information that patients can use to choose between treatment options.

Final Research Report

View this project's final research report.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Reviewers’ comments and the investigator’s changes in response included the following: In response to reviewer requests, the awardee added figures to an appendix showing results for simulation 1 as well as for the other two covariate specifications. The awardee also added a list of variables to the appendix.

  • Responding to reviewer feedback, the awardee clarified that simulation 1, unlike simulation 2, is generalizable because it included prevalent and rare outcome scenarios.
  • The awardee expanded the discussion of how results differ across the simulation scenarios and provided guidance for researchers on determining which results are applicable for their own studies.

Conflict of Interest Disclosures

Project Information

Jessica M. Franklin, PhD
Brigham and Women's Hospital
Methods for Comparative Effectiveness and Safety Analyses in a High-Dimensional Covariate Space with Few Events

Key Dates

September 2013
December 2017

Study Registration Information


Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
Intervention Strategy Intervention Strategies PCORI funds comparative clinical effectiveness research (CER) studies that compare two or more options or approaches to health care, or that compare different ways of delivering or receiving care. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: March 4, 2022