Results Summary and Professional Abstract
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers criticized the report for focusing the results on significant secondary outcomes rather than presenting the primary outcome, informed decision making, first. The reviewers noted the importance of starting with the primary outcome in presenting and interpreting results, especially when some secondary outcomes were exploratory. The researchers reorganized to describe the primary outcome results first, even though there were no significant differences between interventions on informed decision making. The researchers also stated that they disagreed that the secondary outcomes were not meaningful even though many were exploratory. They chose to continue to highlight the secondary outcomes but acknowledged that the results should lead to more definitive hypothesis testing for these outcomes.
- The reviewers noted that the number of measurements participants completed at each data collection was quite large and burdensome and may have contributed to the high attrition rate. The number of related outcome measures could also lead to false positive significant tests due to multiple testing and should be acknowledged. The researchers noted that they did not collect all of the outcomes at each time point, but they added a note about the potential for false positives and the importance of viewing the significant secondary analysis results as exploratory, in their limitations section.
- The reviewers noted that the level of attrition differed for different intervention groups, and that there were a number of baseline measures that differed more than 10 percent between groups. Both of these considerations indicated missing data was unlikely to be missing at random, creating an imbalance between treatment groups and affecting the interpretation of available cases analyses. In response the researchers used multiple imputation to account for missing data and found that their results remained robust. Further, the researchers disagreed with the reviewers that a 10 percent difference on baseline characteristics indicated that there was an imbalance in the groups at baseline, before the attrition problems occurred. The researchers stated that they did not find literature defining a 10 percent difference as the threshold for poor balance between groups. They did add a note in their study limitations that the small imbalance at baseline may have affected the study’s power.
Conflict of Interest Disclosures
View the COI disclosure form.
Other Health Services Interventions
Training and Education Interventions