PCORI funded the Pilot Projects to explore how to conduct and use patient-centered outcomes research in ways that can better serve patients and the healthcare community. Learn more.

Background

When completing health surveys, older adults often require help from someone else. A person who completes a survey on someone else’s behalf is called a proxy respondent. Researchers who analyze survey responses generally note whether or not the survey was completed by a proxy respondent. But researchers often don’t know whether the proxy respondent filled out the survey for the patient or just helped the patient do it. For example, a proxy respondent might help by reading questions aloud to a patient and then recording their answers. Also, researchers often don’t know who the proxy respondent was—a family member, friend, or someone else. So researchers don’t know what effect different proxy respondents might have on survey answers, including for patient-centered outcomes research.

Project Purpose

This study had three goals:

• See how well patients’ and proxy respondents’ answers to survey questions about the patient’s disease history and medical care matched Medicare claims information
• Learn whether the patient’s and the proxy respondent’s answers were the same or different for specific health outcomes
• Identify whether any differences in answers continued or changed over time

After completing the first and second goals, the research team realized that the third goal was not possible in a two-year project. Instead, the team decided to study how well patients’ responses about their own health behavior matched what their spouses reported on their behalf.

Findings

Responses from both patients and proxy respondents were most likely to match Medicare claims on health conditions that needed invasive treatments or constant monitoring, such as diabetes or glaucoma. Survey responses from both proxy and patient respondents were least likely to match the Medicare claims for care that took place often (such as doctor’s visits). Responses from both proxy respondents and patients also tended to be different from Medicare claims for diseases or treatments that doctors and patients might define differently (for example, arthritis or minor surgery in a doctor’s office).

Patients who filled out the survey themselves tended to report having fewer health conditions and receiving less health care than they actually did. People who filled out the survey for patients tended to report more health conditions and healthcare use than was actually recorded in the Medicare claims.

When husbands and wives filled out the survey for each other, the difference between their answers and the Medicare claims was about the same as it was when patients filled out the survey themselves. Both groups tended to report fewer health conditions and less healthcare use than was recorded in Medicare claims.

When people who were not patients’ spouses filled out the survey, their responses were more likely to be similar to the Medicare claims. This was particularly true for sons-in-law and daughters-in-law.

Patients who scored high on thinking ability (the ability to process information) tended to complete their surveys with fewer differences from the Medicare claims. This finding is similar to results the researchers had found in other studies.

In interviews, married couples who answered survey questions about each other were more likely to give similar answers about some activities than about others. The answers tended to agree on activities done every day, health conditions, hospital stays, surgeries, preventive treatments, and how well a person could move around. Spouses’ responses differed more often when asked to rate health or report on visits to the doctor or dentist.

Limitations

First, the research team was able to report only on health conditions or treatments that were in both the AHEAD survey and Medicare records. Second, respondents who completed the survey themselves had good thinking abilities. People with lower thinking abilities may have had more errors. Finally, the research team did not know why someone served as a proxy respondent for a spouse, parent, or in-law. These reasons might have provided more insight into the differences and similarities of the survey responses.

Conclusions

Researchers should record who is filling out their study’s health surveys so they can compare responses of people who complete surveys themselves to responses given by spouses, children, and other proxy respondents. Future health surveys should also include a short list of questions to assess respondents’ mental status. This is especially important if the survey responses cannot be compared with medical records or Medicare claims.

Sharing the Results

The research team published journal articles about the research (see below).

PCORI funded the Pilot Projects to explore how to conduct and use patient-centered outcomes research in ways that can better serve patients and the healthcare community. Learn more.

Background

Someone else (called a proxy respondent) often needs to complete health surveys for older adults. The best practice is to use a binary marker for whether a proxy respondent was used. However, binary markers do not differentiate between self-respondents, self-respondents assisted by others, and proxy respondents, or the relationship between the proxy/assistor and the target person. Because patient-centered outcomes research assumes that all responses are equally reliable and valid, the researchers evaluated respondent status effects.

Project Purpose

There were three objectives of this study. The first was to evaluate the concordance of disease histories, procedures, and health services use between the survey reports versus the Medicare claims data across respondent status categories. The second objective was to evaluate the cross-sectional effects of respondent status categories on these health outcomes. The third objective was to evaluate the longitudinal effects of respondent status categories.

After completing the first and second objectives, the research team decided to drop the third objective because they determined that it was not feasible. They replaced it with a qualitative component (the “pilot study”), which involved interviewing husbands and wives about their own health and health behavior, as well as that of their spouses, and then directly comparing the concordance of those self-reports with spousal proxy reports.

Study Design

The first and second objectives used a prospective closed cohort design with biennial survey interviews linked to Medicare claims. The pilot study involved a qualitative study of older married couples.

Participants, Interventions, Settings, and Outcomes

For the first two objectives, researchers used survey data on participants 70 years old or older who completed the survey on Assets and Health Dynamics among the Oldest Old (AHEAD), a nationally representative sample linked to Medicare claims. Outcome measures included disease histories (e.g., diabetes, chronic objective pulmonary disease), procedures (e.g., cataract surgery, flu and pneumonia immunizations), and health services use (e.g., emergency departments, physicians, hospitals), with comparable data from the Medicare claims. For the pilot study, researchers used a convenience sample of married couples with the same outcome measures. Husbands and wives had to be at least 65 years old with both agreeing to participate. AHEAD participants were interviewed in their homes or by telephone.

Data Sources

Baseline (1993) and follow-up (1995, 1998, 2000, 2002, 2004, 2006, 2008, 2010) survey data were available for 3,661 AHEAD participants in 1993, yielding 12,313 observations linked to Medicare claims for 1991–2010. For the pilot study, the research team used a convenience sample of married couples taken from the University of Iowa’s volunteer registry. The interviews included the same survey items used in AHEAD, with some additional cognitive interviewing queries and probes.

Data Analysis

To measure concordance between survey reports, researchers used kappa coefficients and prevalence and bias adjusted kappa coefficients. Researchers estimated the net effects of respondent status on the outcomes using multivariate logistic, negative binomial, and linear regression models as appropriate, with propensity score adjustments for selection bias.

Finding

Concordance was greatest for the high salience outcomes and for those involving invasive treatments or constant monitoring like cataract surgery, hospitalizations, flu shots, mammography, diabetes, and glaucoma. Concordance was lowest for more frequent and less memorable outcomes like physician visits, and for those where medical and lay definitions differed, like arthritis and outpatient surgery. As a single group, proxy respondents were significantly less likely than self-respondents to underreport, but proxy respondents were significantly more likely to over-report disease histories, medical procedures, and health services use. These effects were essentially off-setting and therefore would have been masked if the analysis had focused on a summary concordance count rather than under- or over-reporting separately.

When proxy respondents were subdivided, the researchers found that spousal proxies were similar to self-respondents on discordance. Non-spousal proxies, particularly daughters, daughters-in-law, sons, and sons-in-law, however, had less discordance than self-respondents and spousal proxies, mainly due to reduced underreporting. This was likely due to the better cognitive function among children than among their elderly parents and is consistent with the age and aging effects found by the same researchers.

Among self-respondents for whom objective cognitive performance tests were available, cognition was associated with greater accuracy and fewer errors in self-reports relative to Medicare claims, even after adjustment for all covariates. Most importantly, it was mental status rather than short- or long-term (episodic) memory (word recall) that had the largest and most consistent associations with improved accuracy and reduced error. This confirms the researchers’ finding that cognitive function, even among self-respondents, is one of the main drivers of the concordance between survey reports and Medicare claims data.

Among the 25 married couples in the pilot study, there was good concordance for most of the instrumental activities of daily living, health conditions, hospitalization, surgery, preventative service, and mobility questions, but only poor concordance for health ratings (e.g., self-rated health) and physician and dental visits. These results question the validity of routinely using spousal proxies in health surveys to obtain health ratings or the number of physician and dental visits among older adults even in this convenience sample where the average educational attainment was quite high and the couples were married for several decades.

Limitations

The range of health outcomes for which concordance could be assessed was limited by the ability to crosswalk the AHEAD survey questions with the Medicare claims data. The cognitive function of the self-respondents was relatively high, which may have led to underestimation of its importance. Finally, data on the reasons for the use of proxy respondents were sparse and crude.

Conclusions

Future studies that include proxy respondents should include a set of indicator variables to compare and contrast spousal, child, and other family member proxy respondents with self-respondents. Future health surveys should include a brief mental status screener, especially in situations where the survey reports cannot be confirmed by linkage to Medicare claims or chart review.