Results Summary
PCORI funded the Pilot Projects to explore how to conduct and use patient-centered outcomes research in ways that can better serve patients and the healthcare community. Learn more.
Background
Understanding what matters to patients is important for providing patient-centered care. The Patient-Reported Outcomes Measurement Information System (PROMIS®) is a survey that asks patients what they are able to do in daily life and how they feel. PROMIS has been used for many health conditions. For people with substance use disorders, doctors need to understand how patients are feeling over extended periods in order to best treat them. Short and simple surveys like PROMIS may help doctors quickly understand how patients are feeling compared with their previous visits and plan treatment accordingly.
Project Purpose
The project’s goal was to determine whether the PROMIS alcohol use survey would help doctors and patients understand patients’ alcohol use and quality of life. The project also tested how PROMIS compares with other alcohol surveys that doctors already use and whether PROMIS tracks patients’ changes over time.
Methods
The researchers enrolled 225 people in the study. Participants were 18 years old or older and had recently joined an outpatient substance use treatment program at the University of Pittsburgh Medical Center. People with health conditions that might affect their ability to think clearly were not in the study.
Participants answered questions from PROMIS on a computer three times: when the study started, one month later, and three months later. The survey asked about alcohol use, emotions, sleep, pain, and other aspects of health.
To help researchers compare PROMIS with other surveys, participants took five other surveys that asked about alcohol use. They also answered questions about their personal characteristics (such as age and gender) and health history.
Researchers asked a smaller group of participants to take part in interviews twice during the study. During the interviews, the researchers gave participants the results of their PROMIS surveys and asked whether
- The results would help them with treatment
- They would be comfortable talking to their doctor about the results
- The survey asked about topics that were important to their treatment
- They had suggestions for improving the PROMIS survey design
Researchers also interviewed participants’ doctors about the PROMIS survey results. After the interviews, participants and their doctors rated how important each part of the survey was.
Researchers examined each participant’s scores on the surveys and any change in scores from the beginning to the end of the study. They then examined whether the PROMIS alcohol use survey measured alcohol use in the same way that the other five surveys did. Researchers also examined any links between participants’ personal characteristics—such as their age, gender, or health history—and their initial alcohol use or how it changed over time.
Findings
Participants’ survey answers did not change very much during the three-month study. The survey responses showed tighter links between personal characteristics and how serious someone’s alcohol use disorder was than between personal characteristics and the likelihood of reducing or stopping alcohol use quickly. Researchers also found that the PROMIS survey results were similar to other alcohol use surveys that doctors use. Talking to patients during the interviews and presenting them with graphs displaying scores for many different parts of their health, including alcohol use, were useful ways to get across important information about what to focus on in treatment. Participants and doctors both said that alcohol use was the most important topic in the survey. The most important nonalcohol topics were depression, anxiety (worry), and emotional support.
Limitations
To use the PROMIS alcohol use survey, researchers needed to be at the clinic, so it was difficult to plan times that worked for everyone. Doctors and other staff may be too busy to give patients surveys, which could make it hard to use them at other clinics and doctors’ offices.
This was not a randomized controlled study. Doing such a study might provide better evidence about whether this kind of feedback is helpful in practice.
Conclusions
The short, simple PROMIS survey gave results similar to other alcohol use surveys that doctors use. Patients and doctors agreed that reviewing the PROMIS scores in a graph format and seeing patients’ responses to certain survey items helped them figure out what to focus on when making plans for treatment. This study also showed that taking the PROMIS alcohol use survey on a computer gave an accurate picture of patients’ alcohol use.
Sharing the Results
The research team has written articles for medical journals (see below) about the results of the study.
Professional Abstract
PCORI funded the Pilot Projects to explore how to conduct and use patient-centered outcomes research in ways that can better serve patients and the healthcare community. Learn more.
Background
Assessments of health status and health-related quality of life should have excellent psychometric properties; be brief; be easy to read, administer, score, and interpret; and be meaningful in the clinical encounter. The goal of this study was to identify, test, and evaluate patient-centered outcomes instruments that meet these standards in a challenging clinical setting: outpatient treatment for substance-use disorders. Successful treatment of alcohol- and other substance-use disorders requires monitoring over extended periods of time (given the relapsing nature of such disorders). Brief and efficient assessments from the patient perspective can be a critical element in such monitoring.
Project Purpose
Key objectives of this research were to: (1) investigate the validity of the Patient-Reported Outcomes Measurement Information System (PROMIS®) alcohol-use item banks and evaluate the real-world applicability of PROMIS measures in an addiction medicine setting; (2) demonstrate the convergent validity of the PROMIS alcohol item banks with existing legacy instruments; (3) illustrate the responsiveness to change of the PROMIS alcohol-use item banks when assessing treatment outcome; and (4) examine the impact of presenting patients and providers with a graphical display of patients’ PROMIS-generated health-status profiles.
Study Design
A three-month prospective observational study.
Participants, Interventions, Settings, and Outcomes
The research team enrolled men and women 18 years of age and older who had begun outpatient treatment for substance use within the past 30 days. Participants with dementia or another major cognitive impairment, any major medical condition that may have a significant impact on the central nervous system, or a history of any psychotic disorder were excluded.
Subjects completed PROMIS item banks for alcohol use, as well as 15 additional item banks from eight other PROMIS domains, including emotional distress, sleep, and pain. Subjects completed assessments at intake, one-month follow-up, and three-month follow-up. Participants also completed legacy measures: the Alcohol Use Disorders Identification Test (AUDIT); the Concern/Cut-down, Anger, Guilt, and Eye-opener Questionnaire; the six-item set of the National Institute on Alcohol Abuse and Alcoholism “recommended alcohol questions;” and the Comprehensive Alcohol Expectancies Questionnaire (CAEQ) . Participants also answered questions about their demographic and clinical characteristics (e.g., presence of medical conditions) and use of all substances both in their lifetimes and in the past 30 days.
A subsample of clinicians and their patients completed ratings of the clinical importance of the PROMIS health domains and qualitative interviews to elicit feedback regarding the content, format, and clinical utility of the patients’ assessment results.
The setting was a substance-abuse treatment clinic at the University of Pittsburgh Medical Center.
Data Sources
The study utilized mixed (qualitative and quantitative) methods. Computerized items (from the PROMIS item banks and legacy measures) were displayed one at a time using Assessment Center, the PROMIS electronic testing platform. The PROMIS item banks were administered as computerized adaptive tests (CATs).
Participants whose total scores on the AUDIT were 20 or greater were also invited to participate in interviews at two time points. Interviewed patients were given the results of their CAT assessments in both bar graph form and as a table of individual items and responses. Their clinicians were also invited to participate in concurrent interviews, which included a review of their patients’ responses and scores. The interviews focused on whether interviewees thought the information would be helpful in treatment, their comfort in discussing the information with their clinician, and the relevance of the content domains to their treatment. In addition, their comments and suggestions for improvement of the formats were solicited. Following the interview, patients and clinicians were asked to rate the importance of the health domains assessed. Each domain was rated on a scale from 1 (less important) to 10 (more important).
Data Analysis
To investigate convergent validity, researchers computed Pearson correlations between the PROMIS alcohol banks and the legacy measures. Researchers also examined the PROMIS equivalents of conventional thresholds and ranges of severity on the relevant legacy measures.
Researchers examined the relationship of the intercepts and slopes of both binary (any drinking in the last 30 days) and continuous (severity of drinking) alcohol-use measures to other relevant demographic and clinical variables to investigate which of these variables were associated with initial status (intercept) or rate of change (slope) of alcohol use.
Findings
There were only small changes over time in the current sample across both the target problem of alcohol use and the other domains of the PROMIS health-status profile. There were more correlates of initial severity of drinking (age, gender, emotional support, cognitive concerns, and sleep disturbance) than correlates of rate of change in severity (gender, participation in an interview providing feedback on the PROMIS health status profile at intake). The importance ratings revealed that depression, anxiety, and lack of emotional support were rated highest of the nonalcohol-related domains among both patients and clinicians. General alcohol use was considered most important by both patients and clinicians. Evidence of convergent validity with the legacy measures was good (ranging from .79 for the full AUDIT to .56 for the CAGE).
Limitations
Assessment research is difficult to integrate into the clinical encounter, and several possible barriers to implementation were identified through the current study. Research staff needed to be available at the clinic to adequately accommodate the schedules of patients and clinicians and to generate the interview reports. Ideally, clinic staff or clinicians themselves would administer the assessments, but given staffing needs and time concerns, this may not always be possible. Incorporating the assessment into patient web portals would give patients the flexibility to choose the time and location.
The current work did not use a randomized controlled design. A more robust demonstration of the impact of feedback would be valuable to justify routine clinical use of such feedback.
Conclusions
Both therapists and patients agreed that their review of the graphical display of scores, as well as individual item responses, helped them to identify areas of greatest concern and was useful for treatment planning. The presence of a feedback interview and use of a graphical display of patients’ comprehensive health status is a clinically relevant and straightforward intervention, and the ability to demonstrate its impact on severity of drinking provides support for its use in clinical practice. The work demonstrates the validity of assessment using PROMIS tools and CATs, which require only four to six items in each health domain. This efficiency makes it feasible to incorporate a comprehensive health-status profile within the substance-use treatment setting, providing important prognostic information.
Although there is a need for additional validation of the impact of feedback, PROMIS assessments may be useful within a clinical treatment setting.