Results Summary
What was the research about?
Patient-reported outcome measures are surveys that ask patients about their health or well-being. These surveys may include questions about sleep, depression, or pain. Many of the surveys now in use don’t focus on a specific health problem.
The research team wanted to create and test a process for adapting patient-reported outcome measures for specific health problems. To test this process, the team developed surveys for two health problems, one for heart failure and one for knee arthritis.
What were the results?
The survey for heart failure had 86 questions in 18 categories. The new categories based on patient concerns were
- Health behaviors
- Illness burden
- Independence
- Life satisfaction
- Symptoms
The survey for knee arthritis had 76 questions in 14 categories. The new categories based on patient concerns were
- Independence
- Life satisfaction
- Symptoms
Testing the surveys with patients showed that
- Patients understood the questions
- Patients answered the questions similarly when they retook the surveys a few days later
- The surveys accurately measured patients’ thoughts and feelings about the health problems and changes over time
Who was in the study?
The study included 847 patients with heart failure. Of these patients, 66 percent were white, 23 percent were African American, and 11 percent were other races. The average patient age was 56, and 44 percent of patients were female.
The study also included 979 patients with knee arthritis. Of these patients, 77 percent were white, 14 percent were African American, and 9 percent were other races. The average patient age was 57, and 63 percent of patients were female.
What did the research team do?
First, the team conducted focus groups with patients living with heart failure or knee arthritis. These patients gave feedback on what concerns related to their specific condition were missing from the general surveys now in use. Then the team wrote questions to fill those gaps. Next, the team interviewed some of these patients about whether the new questions were easy to understand and made changes based on patient recommendations. Finally, the team sent the surveys to patients across the United States. Three to seven days later, the team asked some of the patients to retake the surveys to see if they answered questions similarly. Other patients completed a follow-up survey after three months.
Patient advisors and doctors gave the research team feedback throughout the study.
What were the limits of the study?
The patients in the focus groups didn’t represent all the racial or ethnic backgrounds of people in the United States. People of other races or ethnicities may have different concerns. Future research could test the surveys with patients of different racial and ethnic backgrounds. Researchers could also create shorter versions of the surveys for use in health care.
How can people use the results?
Researchers can use the process developed in this study to create patient-reported outcome measures for other specific health problems.
Professional Abstract
Objective
To incorporate condition-specific concerns into generic patient-reported outcome measures
Study Design
Design Element | Description |
---|---|
Design |
Survey development and testing |
Data Sources and Data Sets |
Patient advisors and expert panel, 61 focus group participants with heart failure, 68 focus group participants with knee osteoarthritis, 10 cognitive interview participants with heart failure, 10 cognitive interview participants with knee osteoarthritis, 786 field-test survey respondents with heart failure, 911 field-test survey respondents with knee osteoarthritis |
Analytic Approach |
Psychometric analysis to evaluate survey reliability, validity, and sensitivity to change |
Outcomes |
Patient-reported outcome measures of heart failure and knee osteoarthritis |
Researchers developed two disease-specific surveys, one for heart failure and the other for knee osteoarthritis, using Patient-Reported Outcomes Measurement Information System (PROMIS®) methods and recommendations from the International Society for Pharmacoeconomics and Outcomes Research’s Task Force. First, researchers conducted focus groups with 61 patients with heart failure and 68 patients with knee osteoarthritis to identify their priority outcome domains. Next, researchers drafted new survey items to address gaps between the priority domains and domains in existing PROMIS item banks. Researchers refined the new items based on the results of cognitive interviews with 10 patients with each condition and added the items to the generic measures.
Researchers field tested the new measures to evaluate reliability, validity, and sensitivity to change. To assess reliability and validity, the sample included 600 patients per condition from an online panel. The average ages of heart failure and knee osteoarthritis patients were 54 and 52, respectively. Among heart failure patients, 45% were female, 67% were white, 19% were African American, and 14% were other races. Among knee osteoarthritis patients, 60% were female, 70% were white, 18% were African American, and 12% were other races. Three to seven days after the initial surveys, researchers asked 100 patients per condition in this sample to retake the surveys to evaluate test-retest reliability.
To assess sensitivity to change, the sample included 311 patients with knee osteoarthritis and 186 patients with heart failure from nine clinics. The average ages of heart failure and knee osteoarthritis patients were 56 and 65, respectively. Among heart failure patients, 42% were female, 55% were white, 39% were African American, and 6% were other races. Among knee osteoarthritis patients, 68% were female, 87% were white, 8% were African American, and 5% were other races. Patients in this sample completed a follow-up survey three months after baseline.
Patient advisors and clinicians who had experience with these conditions provided input throughout the study.
Results
Results suggested that both measures had good reliability, validity, and sensitivity to change.
Heart failure. The final measure included 86 items across 18 domains.
- Reliability and validity. Internal consistency reliability of the domains, measured by Cronbach’s alpha, ranged from 0.52 to 0.94. Test-retest reliability of the domains, measured by Pearson’s r, ranged from 0.81 to 0.98. All domains had correlations greater than 0.6 with PROMIS Global Health Physical and Mental Health summary scores, indicating acceptable concurrent validity.
- Sensitivity to change. For 8 of 18 domains, patients who reported experiencing meaningful positive change in the domain had better domain-change scores than patients who reported negative or no change (p< 0.05).
Knee osteoarthritis. The final measure included 76 items across 14 domains. The new domains were independence, life satisfaction, and symptoms.
- Reliability and validity. Internal consistency reliability of the domains, as measured by Cronbach’s alpha, ranged from 0.67 to 0.95. Test-retest reliability of the domains, as measured by Pearson’s r, ranged from 0.88 to 0.98. All domains had correlations greater than 0.6 with PROMIS Global Health Physical and Mental Health summary scores, indicating acceptable concurrent validity.
- Sensitivity to change. Patients who reported experiencing meaningful positive change in the domain had better domain-change scores than patients who reported negative or no change (p< 0.05).
Limitations
Focus group participants did not represent the educational, racial, or ethnic backgrounds of all people in the United States, which may limit the generalizability of the priority domains identified for the measures. Also, the full final instruments may be too long for ongoing data collection in clinical settings.
Conclusions and Relevance
Using two high-impact health conditions, researchers created disease-specific patient-reported outcome measures. These measures added items about patient concerns to existing generic patient-reported outcome measures.
Future Research Needs
Researchers could use this study’s methodology to adapt generic patient-reported outcome measures for other specific conditions. Additional studies could also test the knee osteoarthritis and heart failure measures developed for this study with patients of different educational, racial, or ethnic backgrounds and create short forms of the measures for use in clinical settings.
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Reviewers’ comments and the investigators changes in response included the following:
- In general, the reviewers found the report to be a comprehensive description of the patient-reported outcomes validation study.
- The reviewers questioned the limited use of crosswalks, which are tables describing equivalent and differing items within two sets of items such as outcome measures. The reviewers questioned why the investigator used so few crosswalks between existing disease-specific measures and the newly created disease-specific items for PROMIS measures. The investigator explained that such crosswalks were possible only with measures of the same constructs. The investigator also explained that the measures the study developed were not consistently of the same constructs, limiting the use of crosswalks.
- Some reviewers expressed concern that adding disease-specific items to the widely accepted PROMIS measures would dilute the value of generic measures that span disease and condition areas. The investigator responded by stating that it intended the new items to expand, not limit, the usability of PROMIS measures by including items that would speak to disease-specific experiences.