Results Summary and Professional Abstract
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers suggested that PCORI Methodology Standards require that the group concept mapping (GCM) approach used in this study be treated as a complex intervention, which would necessitate the inclusion of a specific table describing the approach. The researchers did not complete the suggested table. They reviewed the standards for a complex intervention and concluded that the GCM approach did not fit the construct of a complex intervention or evaluation. They noted that GCM is not an intervention and does not have comparator groups.
- The reviewers criticized the researchers’ conclusions that GCM brainstorming was more comprehensive and more efficient than patient interviews, pointing out that interviews provided rich contextual information that could not be captured in concept mapping. The researchers agreed with this aspect of individual interviews. However, as their focus in this study was to understand which method garnered more patient-important outcomes, the depth of information from individual interviews was not a factor in their results. The researchers revised the report to clarify that GCM brainstorming surpassed individual interviews when the broader context was not relevant to the research question.
- The reviewers noted that participants in the GCM brainstorming sessions likely varied in their ability to move individual ideas into higher domains. The reviewers ask how the researchers dealt with the potential for bias in the GCM brainstorming sessions if some participants were much more capable of performing the higher-level analyses. They also worried that the cognitive limitations of some patients made GCM a less appropriate approach and the results obtained from the study less generalizable. The researchers agreed that individuals varied in their ability to engage in the GCM sessions and agreed that the challenges that participants had with GCM limited the generalizability of the study. The researchers noted in their report that they anticipated this issue by having members of their patient and stakeholder advisory board facilitate the sessions.
- The reviewers asked why the researchers excluded people with complications, like amputations and blindness, from participating. The researchers explained that their goal was to compare two different methods for identifying outcomes important to patients, GCM versus individual interviews. They believed that people who had already experienced serious irreversible complications from diabetes might have a perspective that differed too markedly for comparison from those who had not. The researchers added an explanation of this decision in the sampling strategy section of the methods.
Conflict of Interest Disclosures
View the COI disclosure form.