Results Summary

What was the research about?

Comparative effectiveness research compares two or more treatments to see which one works better for which patients. In some studies, researchers assign patients by chance to several treatments or to have or not have a treatment. But approaches that assign patients by chance are not always suitable. For example, assigning patients to a new treatment may not be good medical care.

For this reason, researchers sometimes do studies using data collected when patients and their doctors choose the treatments. Data from such studies are observational data. When using observational data for research, it can be hard to know if the effect of a treatment is because of the treatment or other factors, such as patients’ age, gender, or health history. In these cases, researchers use statistical methods to understand the effect of the treatment. Depending on the study’s focus and design, some methods work better than others.

In this study, the research team developed guidance for researchers to help them choose methods for their study.

What were the results?

The research team created an online guide. The guide explains the differences between various methods for doing research and gives options for analyzing observational data. It also includes links to other articles and websites. The guide could help researchers choose the method that would be the right fit for their study.

What did the research team do?

First, the research team searched for published articles that describe methods for using observational data to analyze the effects of treatments. Next, they analyzed data to compare the methods to see how they did for finding out treatment effects. Finally, the research team used these results to develop the online guide for other researchers.

What were the limits of the study?

The research team looked only at certain types of methods and studies. For example, the team looked only at articles for studies that compared two treatments. Searching for other types of studies may have led to different ideas for the guide.

Future studies could expand the guide to include other methods. The team could also update the article list in the guide as new research becomes available.

How can people use the results?

Researchers using observational data to compare treatments can use the guide to decide on the design and analysis for their research. Results from studies that use the guide can help patients and doctors compare treatments.

Final Research Report

View this project's final research report.

Journal Citations

Stories and Videos

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:

  • Reviewers generally found the study to be very strong and the report to be clearly written but could not see the connection between the systematic review and the development of the decision tool. The researchers explained that instead of summarizing the articles described in the systematic review, they used the information from the review to develop the final structure of the decision tool.
  • Reviewers asked how the research accomplished one of the original objectives of the systematic review, which was determining which methodological technique works best when. The researchers revised the report to explain that they focused instead on the need to align the research question, methods, and interpretation in observational research.

Conflict of Interest Disclosures

Project Information

Douglas Landsittel, PhD
University of Pittsburgh at Pittsburgh
Modeling Strategies for Observational Comparative Effectiveness Research - What Works Best When?

Key Dates

December 2013
October 2018

Study Registration Information


Has Results
Award Type
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: March 4, 2022