Final Research Report

View this project's final research report.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:

  • The reviewers asked how the presence or absence of patient preferences and self-selection for treatments might interact with treatment effects in different ways. The researchers responded that they could not conduct all possible simulations and relied on their stakeholders to decide which would be most relevant. The researchers added that when they allow many parameters to vary, it would become complicated to describe simulation results and to tease out the import of findings. The researchers noted that this is a limitation of the study.
  • The reviewers said that the researchers’ method of adjusting for additional variables was limited in the types of variables it could accommodate whereas a method that they cited did not present such restrictions. The researchers explained that their goal was not to adjust for additional variables or covariates, but to be able to separate data into subgroups via stratification. The researchers said they cited the work the reviewers suggested, but said the method was better suited for analysis than for using when developing a new method.
  • The reviewers asked how the study’s two-stage trial design for estimating preference effects would differ from what would be obtained in a fully randomized preference trial where participants are asked their treatment preference but then randomized to treatment regardless of preference. The researchers explained that while a fully randomized preference trial would provide an unbiased estimate of preference effect, such a design would increase the bias in estimating selection effect. Also, treatment effect would only be valid for the groups where researchers randomized participants to their preferred treatment. The researchers also noted that it seems unethical to ask participants their treatment preference but ignore that information when assigning treatment groups. 
  • The reviewers asked the researchers to address the potential causal effects related to assigning patients to treatment based on randomization versus patient preference. The reviewers noted that causal definitions of selection and preference effects would be important to consider when discussing methods for a two-stage randomized trial design. The researchers responded that addressing the causal-related definitions of selection and preference was beyond the scope of the study but did add to the discussion that consideration of these other effects would be important to generalizability of the approach.

Conflict of Interest Disclosures

Project Information

Denise Esserman, MS, PhD
Yale University
$515,511
10.25302/10.2020.ME.151132832
Expansion of Methods for Two-Stage Trial Designs for Testing Treatment, Self-Selection and Treatment Preference Effects

Key Dates

July 2016
March 2020
2016
2020

Study Registration Information

Tags

Has Results
Award Type
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: March 4, 2022