Results Summary and Professional Abstract
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers asked for context and examples of patient-centered outcomes research in which the methods developed in this study could eventually be applied. The reviewers also asked whether the expected applications justified overcoming the substantial challenges to implementing these methods. The researchers replied that they envision their research will lead to data registries that support cohort queries, in which they classify users into related groups for analysis. The researchers also envision that the data registries will be used to train certain machine learning models. The researchers noted that their approach would not be suitable for rare disease studies because the methods used to achieve privacy would not work where only a small number of contributing patients are available. The researchers added that the different types of data registries described in their report would apply in different settings. This would depend, however, on the availability of more individualized information, such as individual privacy preferences, that could be built into the machine learning models and improve the accuracy of the methods. The researchers agreed that there are still substantial challenges to implementing their proposed methods but said they believe this is an important research direction that can make health data more accessible for broader-scale research.
- The reviewers asked for more information on how the stakeholder panels engaged and the points that emerged from them. The researchers provided more details about how the panels were engaged. They noted that the panels convinced them of the variability in patient privacy preferences and therefore of the need to develop flexible and customizable methods that take different patient preferences into account. The researchers commented that they believe large-scale studies, such as patient surveys are needed to obtain a better understanding of patient privacy preferences.
- The reviewers requested more information in the report about how the researchers met PCORI Methodology Standards regarding controlling for missing data. These would normally involve multiple imputation as the most reliable method to control for missing data. The researchers acknowledged that there were missing data issues with some of the datasets, but they chose to apply simple imputation methods with the assumption that data were missing at random, rather than the more complicated multiple imputation method. The researchers explained that their interest was in testing their privacy algorithms rather than conducting comparative effectiveness research. Thus, they did not feel that their imputation method had an impact on the study’s results.
Conflict of Interest Disclosures
View the COI disclosure form.