Results Summary
What was the research about?
During care transitions, patients move from one care setting to another, such as from the hospital to home. If not done well, these care transitions can result in health problems for patients and the need for them to return to the hospital.
Healthcare organizations can use patient surveys to measure the success of care transitions. One survey about the quality of care transitions already exists. The survey was created with input from patients but with no input from caregivers and healthcare providers. In addition, the survey doesn’t ask about topics that patients may find important, such as caregiver involvement and the time after care transitions.
In this study, the research team created and tested a new survey. To create the survey, the team asked for input from patients, caregivers, and healthcare providers. The team tested whether the survey was
- Valid, or able to correctly capture what it intends to measure
- Reliable, or able to get consistent answers
What were the results?
The new survey about the quality of care transitions was valid and reliable. The survey asked patients if they
- Understood what health care they needed during and after their hospital stay
- Had received a hospital discharge plan tailored to their needs
- Were confident they could manage their care after leaving the hospital
- Had a transition plan that accounted for transportation needs, payment for care, and other situations that might arise
The survey was as reliable as the existing survey. Both surveys were related to patient-reported health status at three months after discharge.
Who was in the study?
The study included 493 patients discharged from a hospital in Massachusetts. Of these, 92 percent were white, 3 percent were black, 1 percent were Asian or Pacific Islander, 1 percent were American Indian or Alaska Native, and 3 percent were more than one race. The average age was 59, and 64 percent were men. Patients had been in the hospital for an average of about two days.
What did the research team do?
To develop the survey, the research team interviewed 48 patients, caregivers, and healthcare providers. The team asked what care transition topics are important. Then the team created survey questions about these topics and asked 11 patients to read and answer them. The team changed or removed questions that weren’t clear.
Next, 493 patients completed the survey. The research team looked at which questions best measured each care transition topic. The team also tested whether the survey was reliable and valid.
A group of patients, healthcare providers, and a hospital president helped design the study and review results.
What were the limits of the study?
All patients in the study were from one hospital. Most patients were white and were receiving care for heart disease. Results may differ for patients from other backgrounds or with other health problems.
Future research can test how well the survey works with patients of other backgrounds and with other health problems.
How can people use the results?
Hospitals can consider using the new survey that reflects patient and caregiver input to understand patients’ experiences with their care transitions.
Professional Abstract
Objective
To develop and validate a new measure of care transition quality and compare it to the existing Care Transitions Measure (CTM-15)
Study Design
| Design Elements | Description |
|---|---|
| Design | Measuring development and testing |
| Data Sources and Data Sets | Survey responses from 493 patients discharged from UMass Memorial Medical Center |
| Analytic Approach | Factor analysis, item response theory modeling, measurement of internal consistency reliability and construct validity |
| Outcomes | Validated measure of care transition quality |
This study created a new measure of care transition quality that addresses gaps in content validity in the CTM-15, such as involvement of caregivers and post-discharge needs, by incorporating views from patients, caregivers, and clinicians in developing the new measure. The research team developed new questions to address methodological gaps related to acquiescence bias in response format. Acquiescence bias is the tendency of a respondent to agree with a statement when in doubt.
The research team conducted 48 interviews with patients, caregivers, and clinicians to identify concepts related to patient experiences with care transitions from hospital to home. After using these concepts to create questions, the team interviewed 11 patients to see how patients understood and responded to the questions. The team eliminated some questions that patients said were redundant and modified language of other questions to ease comprehension.
The research team then administered the new measure of care transition quality to patients one month after their discharge from the hospital. The team conducted factor analysis on survey responses to identify key aspects of care transition quality and applied item response theory to identify the best questions to measure each aspect. The team also evaluated the validity and reliability of the measure.
The study included survey responses from 493 patients discharged from UMass Memorial Medical Center. Of these, 92% were white, 3% were black, 1% were Asian or Pacific Islander, 1% were American Indian or Alaska Native, and 3% were more than one race. The average age was 59, and 64% were male. The average length of hospital stay was 2.1 days.
An advisory board of patients, caregivers, clinicians, and a hospital president helped design the study and review results.
Results
Results showed that the new measure of care transition quality was reliable and valid. The 12-item measure included questions about four aspects of care transition quality:
- Understanding of care required during and after the hospital stay
- Having a tailored discharge plan
- Confidence in care management after discharge
- Having a care plan
The measure showed comparable reliability with the CTM-15. Both measures were statistically significantly associated with patient ratings of quality of hospital care and patient-reported health status at three months after discharge but not with indicators of health status based on the patient’s condition as reported in medical records.
Limitations
All patients in the study were from one hospital; most patients were white and were receiving care for cardiac disease. Results may differ for patients from other backgrounds or with other health problems.
Conclusions and Relevance
The new measure of care transition quality, which reflects input from patients, caregivers, and providers, addressed content and methodological gaps in the CTM-15 and showed similar reliability and validity.
Future Research Needs
Future research can test the survey’s performance with a more diverse group of patients who are receiving care for other types of health problems.
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers noted that both the new Care Transitions Measure (CTM-NEW) scale and the widely used existing Care Transitions Measure (CTM-15) had poor predictive validity for rehospitalizations and emergency department (ED) visits in this study, although the CTM-15 has demonstrated better predictive validity in prior studies. The reviewers asked what role the CTM-NEW would then serve given its apparent lack of superiority over the CTM-15. The researchers acknowledged that the low predictive validity rates in this study for both measures could be the result of having a relatively homogenous sample and lower than expected rehospitalization rates. They commented on this issue in the report’s discussion section. The researchers agreed that the CTM-NEW would need further testing in a larger, more diverse environment to get better measures of psychometric performance.
- The reviewers noted that the study analyses imply that the CTM-NEW is a patient-level measure rather than a scale to make between-hospital comparisons. The researchers acknowledged that using patients in a single hospital is a major limitation of this study. They noted that it will be necessary to include patients from different health centers in the future in order to test how the scale can be used in the context of hospital comparisons.
- The reviewers noted that despite three general categories—trauma, general surgery, and acute cardiac event—included in tracking rehospitalization and ED visits, the sample was heavily weighted toward cardiac events. The researchers explained that this was due to the fact that trauma and general surgery patients were often discharged to short-term rehabilitation facilities, making them ineligible for participation.
Conflict of Interest Disclosures
Project Information
Key Dates
Study Registration Information
^Milena Anatchkova, PhD was the original principal investigator for this project.