Results Summary and Professional Abstract
Related PCORI Dissemination and Implementation Project
Final Research Report
View this project's final research report.
Results of This Project
To Stay at Home, Older Adults Should Plan Ahead
A narrative about a Chicago team of researchers who are working with older adults to develop a tool that helps them have a voice in what happens as their health needs grow.
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
- Peer reviewers commented about the report and researchers made changes or provided responses. Those comments and responses included the following: The reviewers felt that the study’s focus on advanced care planning in later years was important but noted several methodological concerns. The reviewers asked for additional justification for why the researchers used different types of analyses for the primary and secondary outcomes. Reviewers also asked researchers to further explain why they changed the primary endpoint from three months to one month after the study had started.
- The reviewers noted that originally the primary study endpoint was three months but that it changed to one month once researchers found statistically significant differences in those interim analyses. The investigators indicated that they made this change, and the decision to stop the study early, with input and approval from their stakeholder partners.
- The investigators determined the benefit of the intervention website based on a statistically significant difference between groups on the investigators’ new, unvalidated measure. This measure had no clearly stated prespecified score range for a clinically meaningful difference. Relative to baseline, the intervention group outcome scores did not change; rather, the comparison group scores worsened. Therefore, the reviewers felt that the investigators had not shown that the intervention actually benefited participants. The researchers responded that because this was a new measure and there was no comparison intervention, they were unable to provide insight on what a clinically meaningful difference would be. The investigators did not address the reviewer comment that the intervention group score had not changed at one month relative to baseline. The researchers also could not suggest reasons for the decreased score on the outcome measure for the control group.
- The reviewers expressed concern that the investigators had described the analyses as intent-to-treat, when the investigators used only complete cases—data from participants who completed outcome surveys—in the analyses. The researchers revised the methods section to indicate that they conducted a complete case analysis at one month and three months.
- Overall, the reviewers felt that investigators had not justified their conclusion that the intervention benefits study participants. The reviewers concluded that a number of methodological flaws limited the validity of the results and the conclusions drawn from them. Without further research, the reviewers stated that readers should consider the report’s conclusions with caution.
Given the disagreement between peer reviewers and the researchers about the study’s interpretability, the researchers provided a response to peer-review comments:
The concerns raised in the PCORI peer-review summary only reflect aim 2, which composed one-third of this project. The key issues we found with the peer-review process of our aim 2 were that:
- Our project had been judged by current PCORI Methodology Standards, which had been instituted after our research began in 2013.
- The measures and endpoints used to evaluate the one- and three-month outcomes did not change from the original PCORI-approved statistical analysis plan.
- There are no validated measures of successful aging-in-place, which is why our patient partners and stakeholders helped us to create a new, as of yet unvalidated, measure.
- We identified a statistically significant difference in those subjects who used the intervention compared to subjects in the control arm. Reviewers asked for us to speculate on why the control arm worsened, but we do not feel it is scientifically rigorous to guess.
We feel that we have created a valuable tool, PlanYourLifespan.org, that has demonstrated efficacy in helping older adults plan for their health and support needs that typically occur in the fourth quarter of life. We have published four articles with results in peer-reviewed journals and received additional PCORI funding for both an open science pilot and further dissemination and implementation.
Conflict of Interest Disclosures
View the COI disclosure form.
Training and Education Interventions
Low Health Literacy/Numeracy