|This project's final research report is expected to be available by February 2020.|
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented, and the researchers made changes or provided responses. The comments and responses included the following:
- Reviewers wondered how the analyses took into account the transition period from fee-for-service into managed-care plans, since the effects of the transitions were likely to change during the three years that the researchers observed. The researchers said they did not expect the transition to managed care to be disruptive since the provider networks stayed the same. Even so, the researchers ran new analyses that compared the three years after transition separately to see if there were differences in patterns of healthcare use. These analyses suggested differences in service use for three variables between the first year, and the second and third years after the transition. The researchers described this in their report.
- Reviewers noted that for aim 3, the report presented 23 adjusted comparisons, even though the multiple comparisons are more likely to introduce false-positive findings. The reviewers suggested focusing on the magnitude of differences rather than statistical significance and asked the researchers to define what they would consider clinically important effect sizes. The researchers said that there are no established metrics for clinical significance in the use of services as there are for clinical outcomes. They did agree that it would be more clinically relevant if the presentation of differences in service use was in proportion to baseline service use.
- Reviewers suggested that measuring service in a binary way—whether patients used services or not, rather than frequency of use—would limit the study’s ability to measure the differences between groups in the frequency of service use. The researchers disagreed that this should be considered a limitation. They said that their data do allow them to count service use episodes. However, in general they considered any use of the service to be a more important outcome, since any use indicates that patients had access to the service. The researchers declined to make any changes to the report on this score.
Conflict of Interest Disclosures
View the COI disclosures form.