I am concerned that the limitations imposed by the time line of 90 days may be impractical. I have conducted large scale surveys of the patient population and have found that the analysis alone—figuring out what is true and relevant—can take a considerable amount of time and needs to percolate. Hence, I have concerns about the tight timeline creating a “rush to judgment”. Maybe a generous reading of “conduct or receipt” of findings could help here.
I would want patients involved in the peer review. My involvement as a patient in appraisal of research tells me that there can be a considerable gap between the research design and what is of importance to patients. There also can be a considerable gap between the findings and the conclusions of a study, which may simply reflect intellectual bias of the researchers.
I would select patients that have experience and knowledge to hold their own in the assessment. Frequently, unempowered, unprepared patients are just by-standers and window dressing in peer review. Get patients with experience to avoid tokenism. I would want patient concerns placed front and center. Each time, someone should ask was the patient perspective considered in terms of adequately expressing uncertainty and the room for patient values and preferences provided in conclusions, was the question important to patients, were the outcomes important to patients, do the findings ring true to patients.
I am concerned about the narrowness of the peer review here, which appears to be limited to one methodologist. Doesn’t this create a risk for bias? I am also concerned that there may be an appearance of suppression or content driven modifications required that are perceived as “policing of the science”. This is not what PCOR stands for, but all appearances should be addressed. Perhaps an open peer review with all comments published would address this concern.
Absolutely essential.
Appropriate.
This is appropriate.
This is appropriate.
I believe that the anonymity of peer review is damaging to the integrity of science. Transparency is key. I suggest using an open peer review process where the reviewers know the authors and the authors know who the reviewers are, and, most importantly, the whole thing, comments and all, is published. This lifts the veil of secrecy and allows public scrutiny of all. BMJ has recently adopted this and added patient peer reviews, all published along- side.
One unique option to consider would be post-publication peer review like that offered by F-1000. In this system, the entire peer review process is open, public, transparent, and occurs real time. This would address the problem of having a PCORI peer review that conflicts with the “real publication” peer review.
Neither Well nor Poorly
I see it as a work in progress. It is right where it should be at this stage, but should be modified to address significant public comments.
Neither Clear nor Unclear
There is a need for further development here as to both "who" peer reviews and "how" the peer review process is done, for example post-publication.
The time frame seems tight to do the job right.
Neither Agree nor Disagree
I think you need some quality control, but really need to think through the process, how it might be perceived, and the goals. Data that is incorrect is not useful. So taking the time to ensure the submission is accurate is important.
I would select patients that have experience and knowledge to hold their own in the assessment. Frequently, unempowered, unprepared patients are just by-standers and window dressing in peer review. Get patients with experience to avoid tokenism. I would want patient concerns placed front and center. Each time, someone should ask was the patient perspective considered in terms of adequately expressing uncertainty and the room for patient values and preferences provided in conclusions, was the question important to patients, were the outcomes important to patients, do the findings ring true to patients.
I am concerned about the narrowness of the peer review here, which appears to be limited to one methodologist. Doesn’t this create a risk for bias? I am also concerned that there may be an appearance of suppression or content driven modifications required that are perceived as “policing of the science”. This is not what PCOR stands for, but all appearances should be addressed. Perhaps an open peer review with all comments published would address this concern.
One unique option to consider would be post-publication peer review like that offered by F-1000. In this system, the entire peer review process is open, public, transparent, and occurs real time. This would address the problem of having a PCORI peer review that conflicts with the “real publication” peer review.