Results Summary
What was the project about?
When choosing a treatment, doctors often look at research results that show how well the treatment worked in large groups of people. But many factors can affect how well a treatment works for an individual patient. These factors may include the patient’s sex, age, other health problems, or how they responded to treatments in the past. Some patient data sources, such as electronic health records, have this information. But existing statistical methods may not use these data well. For example, existing methods may not be able to take advantage of data that include measurements of a patient’s health from more than one point in time.
For this project, the research team developed new methods to analyze data that includes measurements of a patient’s health from different points in time. To develop the new methods, the team used a Bayesian approach. Bayesian approaches include findings from previous studies in the analysis, which can make results more accurate.
What did the research team do?
The research team used data on a patient’s own health and on how well a treatment worked for others to create the new methods. To help researchers use the new methods, the team created a computer program. Finally, the team used three studies to adapt the methods to predict health changes and responses to treatment for patients having one of three different health problems.
Patients, doctors, and a health plan administrative leader provided input to build and refine the methods. A group of researchers helped to create the computer program.
What were the results?
The methods predicted aspects of a patient’s health and changes in health over time. In the first study, the new methods helped researchers find out whether a child’s pneumonia was caused by a virus or bacteria. In the second study, the team used the new methods to see if active monitoring or surgery would work better for a patient with prostate cancer. In the third study, the new methods helped to predict a patient’s mental health symptoms over time.
What were the limits of the project?
The research team tested the new methods with three health problems; the methods might work differently for other health problems.
Future research could test the methods with other examples. Studies could also look at whether using the methods for treatment decisions can help improve patients’ health.
How can people use the results?
Researchers can use the methods to help predict changes in a patient’s health and how well a treatment will work.
Professional Abstract
Background
Researchers can predict a patient’s health trajectory and response to treatments, but to do so, they need improved statistical methods to leverage information in electronic health records. Better predictions of patient health outcomes can help clinicians improve clinical diagnoses and treatments. But gaps in existing statistical methods make it difficult to analyze health data to predict changes in individual patient health outcomes. This study developed Bayesian hierarchical models for analyzing diverse types of health data in electronic health records to answer research questions about patient health outcomes. Bayesian models incorporate data from previous trials or studies in the estimation of treatment effects.
Objective
To use a Bayesian analytic approach to develop and implement new methods and software for predicting individual patient health status, changes in health status over time, and response to treatment
Study Design
Design Element | Description |
---|---|
Design | Theoretical development, empirical analysis, software development |
Data Sources and Data Sets | Data from 3 clinical studies, including (1) the PERCH Study on childhood pneumonia, (2) the Brady Urological Institute active surveillance study on prostate cancer, and (3) the Janssen schizophrenia trial and National Network of Depression Centers project |
Analytic Approach | Bayesian hierarchical modeling |
Outcomes |
Model outputs, including (1) predictions of individual health status, (2) predictions of individual health trajectory, (3) estimates of treatment effects, and (4) measurement of heterogeneity of treatment New software package called OSLER inHealth |
Methods and Results
The research team developed a Bayesian hierarchical statistical model. The model had four components:
- Estimation of the effects of current treatment and patient characteristics such as age and clinical history on health status
- Use of multiple clinical measures to infer health status
- Analysis of the effect of health measurements at one time point on subsequent treatment decisions
- Specification of a model with two levels in which factors affecting the estimation of treatment effects could vary over time for the same patient and could also vary across many patients with similar characteristics
The research team then developed a software package called OSLER inHealth to help other researchers implement the Bayesian hierarchical model as well as existing methods to make individual predictions on health status and treatment effects.
Finally, the research team used three case studies to test and adapt the statistical model and software to predict a patient’s health status and changes in health over time. The team applied the methods to estimate
- Whether a child’s pneumonia was caused by a virus or bacteria
- The efficacy and safety of active surveillance compared with surgery for a patient with prostate cancer
- A patient’s level and trajectory of depression, anxiety, and mania symptoms in the past, which could be used to predict future symptoms
Patients, clinicians, and a health plan administrative leader helped build and refine the statistical models for each case study. The research team also worked with a steering committee of researchers and statisticians to develop the software.
Limitations
The research team implemented the methods in three case studies; the performance of the methods may differ when applied to other health conditions.
Conclusions and Relevance
Bayesian hierarchical models can combine diverse sources of prior knowledge and data about an individual patient and a reference population to predict the patient’s health status and the likely effects of different treatment options. Such predictions can help doctors and patients monitor disease progression and make better clinical decisions.
Future Research Needs
Future research could systematically evaluate the new methods in larger and more diverse patient populations. Studies could also evaluate whether using the new methods in treatment decisions can improve health outcomes.
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- Most of the research presented in this final report had already been published in peer-reviewed journals. The reviewers’ comments primarily related to the presentation of the research, acknowledging that it was difficult to apply the appropriate amount of technical information when summarizing mostly published research. The researchers made several changes in describing their research to improve the flow of the report and provide enough information for readers to understand the project.
- The reviewers asked the researchers to clarify that their project was not intended to compare to existing approaches the impact of the new statistical models on diagnostic accuracy and treatment decisions. The researchers edited the text to clarify that this project was meant to be a proof-of-concept study that developed the new approaches. However, they noted that a larger study would be necessary for any comparisons to existing models.