PCORI funded the Pilot Projects to explore how to conduct and use patient-centered outcomes research in ways that can better serve patients and the healthcare community. Learn more.

Background

For patients with multiple sclerosis (MS), treatment decisions can be complicated. Not all treatments work for every patient, and there is little information available for doctors and patients about how to choose among them. Symptoms can change over time, too, and treatment can be expensive. This makes it hard for patients to

• Understand how their disease might change over time
• Make decisions with their doctor about which treatment to choose
• Project Purpose

The research team created an app called MS BioScreen that lets patients and doctors store information about a patient’s condition. In this study, patients used the app to compare their health information with information from other people with MS. The researchers then tested the app to see whether patients and doctors liked it and could use it.

Methods

Researchers built the app after holding meetings with medical experts and patients to get input on what the app should do and how people might use it. The app can compare a patient’s clinical information against a database of more than 2,500 patients whose health was monitored yearly for periods of 5 to 15 years.

The research team tested the app with 10 doctors and their patients with MS at the University of California San Francisco (UCSF) Multiple Sclerosis Center. The doctors asked their patients questions and entered the answers into the app on a tablet computer. The app compared each patient’s information with data from similar patients in the database. The app then predicted how the patient’s disease might change over time. The doctor and patient could look at the results together and discuss the best ways to manage the patient’s MS.

Findings

The MS Bioscreen app was able to find information about similar patients and predict how a patient’s disease might change in the future.

Doctors said the app made it easier to see each patient’s information in a single place. Doctors said having the information helped them teach patients about how their condition is likely to evolve and gave them more information to make decisions about treatment. When testing the app in real-life situations, both doctors and patients said that the app helped them talk about treatment decisions.

Limitations

The information in the app comes mostly from two large clinics that study MS. The information from those patients might not look the same as data for all patients with MS across the country. Also, some people may not have access to a tablet computer and so will not be able to use the app.

Conclusions

The MS BioScreen app allows patients to compare their own clinical information with information from many other patients with MS. This could help patients and doctors understand how the disease might change over time and how to better treat each individual patient.

Patients and doctors agree that there is a need for an app like this that can help inform treatment decisions.

Sharing the Results

The research team published an article about the MS BioScreen app. The MS BioScreen app, which now includes even more information, will be tested with more patients at the UCSF Multiple Sclerosis Center.

PCORI funded the Pilot Projects to explore how to conduct and use patient-centered outcomes research in ways that can better serve patients and the healthcare community. Learn more.

Background

Despite the high cost, treatments for multiple sclerosis (MS) are prescribed with very little data available to identify individuals who would be most likely to benefit, especially over the long-term, and thus those for whom the therapies will be most useful/cost-effective. The complexity of decision making for MS creates barriers that make it extremely difficult for patients to participate in decision making in a meaningful way.

Project Purpose

To develop a transparent digital medical tool (MS BioScreen) that quantifies outcomes and disease trajectories, in the context of relevant comparison groups, to facilitate the patient’s engagement in disease management and research data acquisition.

Study Design

Several prospective cohort studies were integrated into a common reference database and an individual patient’s disease trajectory was compared to others with similar disease characteristics.

Participants, Interventions, Settings, and Outcomes

The primary setting was the MS Research Center at the University of California, San Francisco (UCSF), a state-supported academic medical center. The research team held more than 70 formal meetings with stakeholders and interested parties and had innumerable individual conversations over five years regarding applications of the MS BioScreen beyond the specialized MS clinic.

The researchers built a framework to integrate clinical scores, patient-reported outcomes, imaging, and genetic data into a machine learning prediction algorithm. The prediction outcome was short- and middle-term worsening of the patient’s disease trajectory. This framework allowed the researchers to (1) build patient-specific predictions based on learning from the complete set of past observations, (2) estimate the relevance of these predictions, and (3) explore which data and methods were most meaningful to this task.

Data Sources

The primary dataset was the UCSF MS EPIC (Expression/genomics, Proteomics, Imaging, and Clinical) cohort of >500 patients. The cohort was expanded to included 2,500 patients followed for 5 to 15 years by integrating and standardizing data from MS centers at Brigham and Women’s Hospital (BWH) and Barcelona. 

The data sources consisted of standardized well-curated clinical, imaging, genetic, and other biomarker data prospectively collected primarily at UCSF and BWH. After integrating data from the sites, researchers were able to assess disability progression. At UCSF, at a median time of 16 years after disease onset, 10.3% (95% CI = 6.9% to 13.6%) of patients reached an Expanded Disability Status Scale (EDSS) ≥6 (cane dependence). At BWH, 22% of patients reached an EDSS of 6 by 15 years (32% by 20 years and 38% by 25 years). In contrast, in earlier natural history studies, up to 50% of patients reached an EDSS of 6 by 15–16 years.

The researchers also developed a pipeline to import clinical data from the UCSF electronic medical record (EMR). Using an algorithm combining ICD-9 codes, free text, and MS-specific drugs, researchers examined de-identified EMR data from 53,798 patients, encompassing more than 1.8 million clinical notes and successfully identified 4,142 MS patients (including 95.9% of the EPIC research cohort). There was good concordance between EMR and research values (ICC = 0.86 for EDSS, weighted Kappa = 0.66 for MS type). Therefore, the researchers were able to (1) identify MS patients from the large UCSF EMR system, and (2) show that some extracted demographic and clinical data are comparable to gold standard research quality data.

Data Analysis

A cornerstone of the MS BioScreen is the contextualization algorithm. The contextualization algorithm allows the user to contextualize any metric against a reference population that can be tailored in real-time by “filters.” This feature allows for personalization of the contextualization based on user-driven hypothesis or external information. The speed and reliability of the contextualization engine has been improved, thanks to extensive use of caching, which speeds the delivery of the personalized insights to the app, enabling greater use by the physician and the patient.

Researchers also built a framework to integrate clinical scores, patient-reported outcomes, imaging, and genetic data into a machine learning prediction algorithm. The prediction outcome is short- and midterm worsening of the patient’s disease trajectory. The advantage of this new approach compared to classic parametric models is the granularity at which the question can be answered: each prediction is based solely on a single patient’s data. Finally, the prediction probability of worsening for each patient at a given time can be interpreted as a much needed dynamic score that sums up the full array of static data relevant to a patient into an actionable tool for the clinician that describes the activity of the disease.

Findings

The MS Bioscreen is a data infrastructure platform designed to integrate and visualize a patient’s myriad clinical, imaging, and biomarker data of relevance to their long-term disease course, to contextualize these within the framework of a deeply phenotyped cohort, and to predict their clinical course. This enables precise assessment of clinical trajectory and tailoring of clinical care. In its first iteration, researchers incorporated these algorithms into a tablet application connected to the back-end data infrastructure, allowing instant access to individual patient information. In preliminary testing within the MS clinical center, both clinicians and patients expressed enthusiasm for the device, reporting that it promoted mutual engagement, understanding, and decision making in this real-life situation.

Limitations

Data from this study were derived from prospective cohort studies primarily at two sites in the United States and therefore findings may not be generalizable to the broader MS population. A second potential limitation is that some end users might not be able to access an iPad-based application or system.

Conclusions

From more than 70 formal meetings with stakeholders and interested parties and innumerable individual conversations over five years regarding applications of the MS BioScreen, the researchers conclude that:

• Patients have a clear need and enthusiasm for openly accessible ways to visualize and contextualize their own health trajectories and decisions.
• Among general and MS neurologists outside of large MS centers there is high demand for a “second opinion” tool that frames a patient’s course within the context of large, treated, modern cohorts. This framing allows for better honing of management decisions, even as the ultimate recommendations remain personalized.
• There is utmost concern from patients and providers that such a tool be free of commercial interests.