|This project's final research report is expected to be available by June 2019.|
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- Reviewers noted that the report’s conclusions overstated the success of the intervention since most of the differences favoring the intervention were not statistically significant. The researchers explained that since the nonsignificant findings were in one direction, it was reasonable to assume that the intervention had some effect. However, with further urging from reviewers, the researchers removed the assertions that the intervention affected outcomes despite the lack of statistically significant differences.
- Reviewers pointed out that a major finding of the study related to the reachability of study participants, rather than the intervention outcomes. The researchers acknowledged this point and added language to their report describing their lessons learned about contacting and engaging participants. The researchers argued further that despite the very low participation and completion rates for the intervention and control groups, the results could be generalizable to other populations.
- Reviewers expressed concern about the potential for post-randomization selection bias. The study randomized participants before inviting them to the program itself, which meant that people who entered the program could differ from those who did not, as well as from those participants randomized to the uninvited group. Therefore, it is possible that the benefits seen in the intervention group were because the participants who were easier to contact to issue the invitation to were also those who would respond better to the intervention. The researchers acknowledged this potential confounder but were not comfortable concluding that the observed benefit among program recipients was due entirely to the ease of contacting such recipients about participating in the program. However, the researchers revised their conclusions to acknowledge the potential effect of selection bias for as-treated analyses.
Conflict of Interest Disclosures
View the COI disclosure form.