Final Research Report

View this project's final research report.

More to Explore...

Stories and Videos

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

  • Peer reviewers commented about the report and researchers made changes or provided responses. Those comments and responses included the following: The reviewers felt that the study’s focus on advanced care planning in later years was important but noted several methodological concerns. The reviewers asked for additional justification for why the researchers used different types of analyses for the primary and secondary outcomes.  Reviewers also asked researchers to further explain why they changed the primary endpoint from three months to one month after the study had started.
  • The reviewers noted that originally the primary study endpoint was three months but that it changed to one month once researchers found statistically significant differences in those interim analyses. The investigators indicated that they made this change, and the decision to stop the study early, with input and approval from their stakeholder partners.
  • The investigators determined the benefit of the intervention website based on a statistically significant difference between groups on the investigators’ new, unvalidated measure. This measure had no clearly stated prespecified score range for a clinically meaningful difference. Relative to baseline, the intervention group outcome scores did not change; rather, the comparison group scores worsened. Therefore, the reviewers felt that the investigators had not shown that the intervention actually benefited participants. The researchers responded that because this was a new measure and there was no comparison intervention, they were unable to provide insight on what a clinically meaningful difference would be. The investigators did not address the reviewer comment that the intervention group score had not changed at one month relative to baseline. The researchers also could not suggest reasons for the decreased score on the outcome measure for the control group.
  • The reviewers expressed concern that the investigators had described the analyses as intent-to-treat, when the investigators used only complete cases—data from participants who completed outcome surveys—in the analyses. The researchers revised the methods section to indicate that they conducted a complete case analysis at one month and three months.
  • Overall, the reviewers felt that investigators had not justified their conclusion that the intervention benefits study participants. The reviewers concluded that a number of methodological flaws limited the validity of the results and the conclusions drawn from them.  Without further research, the reviewers stated that readers should consider the report’s conclusions with caution.

Given the disagreement between peer reviewers and the researchers about the study’s interpretability, the researchers provided a response to peer-review comments:

The concerns raised in the PCORI peer-review summary only reflect aim 2, which composed one-third of this project. The key issues we found with the peer-review process of our aim 2 were that:

  • Our project had been judged by current PCORI Methodology Standards, which had been instituted after our research began in 2013.
  • The measures and endpoints used to evaluate the one- and three-month outcomes did not change from the original PCORI-approved statistical analysis plan.
  • There are no validated measures of successful aging-in-place, which is why our patient partners and stakeholders helped us to create a new, as of yet unvalidated, measure. 
  • We identified a statistically significant difference in those subjects who used the intervention compared to subjects in the control arm. Reviewers asked for us to speculate on why the control arm worsened, but we do not feel it is scientifically rigorous to guess.

We feel that we have created a valuable tool, PlanYourLifespan.org, that has demonstrated efficacy in helping older adults plan for their health and support needs that typically occur in the fourth quarter of life. We have published four articles with results in peer-reviewed journals and received additional PCORI funding for both an open science pilot and further dissemination and implementation.

Conflict of Interest Disclosures

Project Information

Lee Lindquist, MD, MPH, MBA
Northwestern University at Chicago
$1,904,872
10.25302/4.2019.IH.12114259
Advance Planning for Home Services for Seniors

Key Dates

May 2013
August 2018
2013
2018

Study Registration Information

Tags

Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
Populations Populations PCORI is interested in research that seeks to better understand how different clinical and health system options work for different people. These populations are frequently studied in our portfolio or identified as being of interest by our stakeholders. View Glossary
Funding Opportunity Type
Intervention Strategy Intervention Strategies PCORI funds comparative clinical effectiveness research (CER) studies that compare two or more options or approaches to health care, or that compare different ways of delivering or receiving care. View Glossary
Research Priority Area
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: March 4, 2022