Results Summary
What was the project about?
Patients with high healthcare needs include those with multiple health problems, like diabetes and heart disease. Current methods to identify patients with high healthcare needs aren’t always accurate. These methods may miss some patients such as those with mental illness or social risks like unstable housing. Better methods could help doctors make sure these patients are getting the care they need.
In this study, the research team developed a method to predict which patients may have high healthcare needs.
What did the research team do?
First, the research team created a new system for identifying patients who have high healthcare needs. The team organized patients into 10 groups based on different patient traits, health problems, and risks, such as chronic pain, mental illness, and unemployment. The team then tested the system to see if it could identify patients with high healthcare needs. To do so, the team used data from 428,024 patients receiving care at New York City health centers. They also used data about the neighborhoods where patients lived. All patients had Medicare insurance.
Using the 10 groups, the research team created a statistical method to predict if a patient would have high healthcare needs. The team checked how different groups and other patient data, like age and gender, improved how accurate the method was. They tested the method with data from 1,074,389 patients receiving care from New York City health centers from 2013 to 2016. The team confirmed the results using data from health systems in Florida.
Patients, caregivers, doctors, and health system administrators helped design the study.
What were the results?
The new grouping system identified 99 percent of patients with high healthcare needs.
The statistical method predicted which patients were likely to have high healthcare needs. The method was most accurate when it had patient data on all 10 groups plus age and gender.
What were the limits of the project?
The research team used data from patients with Medicare insurance in two states. The results may differ for patients with other types of insurance or from other locations.
Future studies could test the method using data from patients with other types of insurance or in other regions.
How can people use the results?
Researchers and doctors can use the methods to help make sure patients with high healthcare needs get the care they need.
| How this project fits under PCORI’s Research Priorities The PCORnet® Study reported in this results summary was conducted using PCORnet®, the National Patient-Centered Clinical Research Network. PCORnet® is intended to improve the nation’s capacity to conduct health research, particularly comparative effectiveness research (CER), efficiently by creating a large, highly representative network for conducting clinical outcomes research. PCORnet® has been developed with funding from the Patient-Centered Outcomes Research Institute® (PCORI®). |
Professional Abstract
Background
Identifying which patients have high healthcare needs and utilization is important for providing patients with personalized care and reducing avoidable hospitalizations. But existing methods do not fully capture the range and variation in patient characteristics that may be associated with higher needs and healthcare use. For example, most methods do not identify patients with overlapping health problems or complex health and social needs. A better classification system, or taxonomy, and statistical models can help clinicians identify and predict which patients may have future high healthcare needs.
Objective
To develop and validate a statistical model based on a taxonomy of patients with high healthcare needs and utilization to predict future healthcare needs and utilization
Study Design
| Design Element | Description |
|---|---|
| Design | Empirical analysis |
| Population |
|
| Analytic Approach | Prediction model development using logistic regression |
| Outcomes | Predicting risk of greater healthcare needs and utilization; model performance using the area under the receiver operating characteristic curve (AUC), Brier score, and calibration plots |
Methods
First, using data from prior research, investigators developed a taxonomy of 10 overlapping categories to classify patients with high healthcare needs and utilization. Researchers looked at whether the categories could be identified using data on clinical diagnoses and healthcare utilization from claims and electronic health records (EHRs) for 428,024 patients with Medicare insurance from the INSIGHT PCORnet® Clinical Research Network in New York City. Researchers also added data from the American Community Survey on the socioeconomic status of patients’ neighborhoods, including income, education, employment, and housing quality.
Then researchers used the 10 taxonomy categories as predictor variables in statistical models to determine a patient’s risk for increased future healthcare needs and utilization. To develop and test different prediction models, they varied the combination of taxonomy categories and other types of variables, like demographics and social conditions. Researchers then tested the prediction models using longitudinal data from 2013 to 2016 for 1,074,389 patients with Medicare from INSIGHT. They assessed the models’ accuracy and performance using logistic regression and machine learning approaches. They then validated the models using data from the OneFlorida PCORnet Clinical Research Network.
Patients, caregivers, clinicians, and health system administrators helped design the study.
Results
Using claims and EHR data, researchers found that the 10 taxonomy categories identified 99% of all patients with high healthcare needs and utilization.
Among the different statistical prediction models tested, the logistic regression-based prediction model with all 10 taxonomy categories and demographics performed the best, with good discrimination (AUC 0.71–0.77) and accuracy (Brier score 0.19–0.21). The model correctly predicted patients with high healthcare needs and utilization in each year. Adding more predictive variables to the model did not improve accuracy.
Limitations
Researchers used data from Medicare claims in New York City and Florida to develop the taxonomy and predictive models. Results may differ with data from patients in other locations or with other types of insurance.
Conclusions and Relevance
The statistical model helped predict which patients had high healthcare needs and utilization. Health systems could use the model to help personalize care for these patients.
Future Research Needs
Future research could test the statistical model’s performance using other clinical data such as lab test results or data for patients who live in other regions or have other types of insurance.
| How this project fits under PCORI’s Research Priorities The PCORnet® Study reported in this results summary was conducted using PCORnet®, the National Patient-Centered Clinical Research Network. PCORnet® is intended to improve the nation’s capacity to conduct health research, particularly comparative effectiveness research (CER), efficiently by creating a large, highly representative network for conducting clinical outcomes research. PCORnet® has been developed with funding from the Patient-Centered Outcomes Research Institute® (PCORI®). |
COVID-19-Related Study
Summary
This study received additional funding in 2020 to quickly initiate new research related to COVID-19. The additional research is in progress. PCORI will post the research findings on this page once the results are final.
Because COVID-19 is new, patients and healthcare providers lack evidence to inform care decisions and predict disease course for patients in the hospital with severe COVID-19. One decision is whether a patient will need to go to the intensive care unit. Another is whether they will need to be intubated. Intubation is the process of inserting a tube to open the airway and help patients breathe.
With this enhancement, the research team will develop statistical models using data from health systems in New York. The models will predict
- Whether and when patients need intensive care
- What will happen during and after intubation
- The risk of death
Results can support alignment of care decisions with patient goals.
Enhancement Award Amount: $500,000
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers pointed out that there is a distinction between patients with high health needs and patients with high health costs, because research has demonstrated that patients from underrepresented minority groups often face discrimination and less access to health care, so high-needs patients from these communities may not have high healthcare costs. They recommended that the researchers instead use a proxy of number of chronic conditions to identify high-needs patients rather than costs. The researchers agreed with this consideration and analyzed the data based on this premise to determine whether their results remained consistent. They compared the number of chronic conditions by race and socioeconomic status for each level of predicted cost-based patient risk, and found no biases based on race or socioeconomic status in their results. However, they acknowledged that the databases they used to collect health needs may themselves have biased information because the data may be incomplete for patients from disadvantaged groups with less access to care.
- The reviewers noted the poor performance of the machine-learning models compared to logistic regression, in predicting healthcare costs. The researchers agreed that the machine-learning models did not perform well and agreed with reviewers that this was probably due to them using only 10 predictor variables in calculating those models via machine learning.
- The reviewers asked the researchers to discuss their proposed approach to predicting high-cost patients in comparison to other prediction methods that the researchers mentioned in the background section of the report. The researchers pointed out that they could not readily compare their methods to some of the other prediction methods because of different definitions for what percent of annual health spending constituted high cost, different populations of patients, and different goals for developing the taxonomy predicting high-cost patients.