Developing Software to Predict Patient Responses to Knee Osteoarthritis Treatments and to Identify Patients for Possible Enrollment in Randomized Controlled Trials

This project has results

Results Summary and Professional Abstract

Results Summary

Download this abstract: In English (pdf)

What was the research about?

Comparative effectiveness research compares two or more treatments to see which one works better for certain patients. This research may include randomized controlled trials, or RCTs, in which researchers assign patients to one of the treatments by chance.

A patient may enroll in an RCT when, based on current knowledge of that patient’s traits, the treatments being tested have about the same chance of helping. If one treatment is known to have a better chance of helping a patient, then the patient would not enroll and would receive that treatment from the doctor.

Sometimes there isn’t enough research to show if one treatment has a better chance of helping than another. In this case, researchers may use computer programs. The programs estimate how well different treatments work in patients with certain traits. For example, a person’s age and pain level may affect how much a treatment helps.

These programs would be useful for patients with knee osteoarthritis. Not many RCTs have compared total knee replacement surgery with other treatments such as medicine or physical therapy.

In this study, the research team made a computer program for patients with knee osteoarthritis. It uses data from electronic health records. The program could help identify patients for whom

The treatments in the study have about the same chance of helping. These patients may wish to take part in an RCT.
A certain treatment may help more than another. These patients could choose that treatment.

The research team also made an online system based on the program for patients and doctors to use during a visit. Doctors can use the results from the system to talk with patients about treatment. If appropriate, they could talk about taking part in an RCT.

What were the results?

The program is useful to estimate how well treatments work in patients with certain traits.

For one year after treatment, the program estimates

How much pain a patient is likely to have
How well a patient is likely to function, such as how well they can walk

What did the research team do?

To make the program, the research team needed information about the long-term effects of total knee replacement and nonsurgical treatments. The team used information from 1,322 patients with knee osteoarthritis. Some patients had a total knee replacement. Other patients had nonsurgical treatments. The information came from four databases. The databases had survey information about patients’ pain and function one year after treatment.

The research team paired each total knee replacement patient in the databases with a patient who didn’t have surgery. The paired patients had similar traits, like similar ages, health problems, and pain before treatment. This pairing let the team compare results in similar patients who had different treatments. The team used information from the paired patients to make and test the computer program. They wanted to see how well the program could use information on patients’ traits to figure out if a certain treatment may help more than another. Then, the team made an online system for doctors and patients.

During the study, the team worked with a group of knee osteoarthritis researchers, patients, doctors, and patient advocates. The group gave input on research questions and study design.

What were the limits of the study?

Differences in the databases made it hard to pair patients who had surgery with similar patients who had nonsurgical treatment. This may have affected how well the computer program was able to estimate treatment results for patients.

The information on how well patients function after treatment came from a survey on overall physical function. A survey that asked specifically about patients’ knee function, separate from pain, may have helped the research team better predict knee function.

Future research could use newer ways to analyze information from patients. Researchers could also use information from more patients. Such research may help researchers make programs that better estimate treatment results.

How can people use the results?

The computer program could help doctors identify patients with knee osteoarthritis who, based on their traits, could take part in an RCT. The program may also help doctors identify patients who may get more, or less, benefit from a certain treatment.

Professional Abstract

Objectives

(1) To use nonrandomized data from patients with knee osteoarthritis to create models that predict patient-specific outcomes of different treatment options; (2) To use the models to develop software to help identify patients who may be eligible to enroll in randomized controlled trials (RCTs)

Study Design

Design Elements	Description
Design	Empirical analysis (nonrandomized study)
Data Sources and Data Sets	A consolidated database of 1,452 patient knees, half with total knee replacement and half with nonsurgical treatment, from 1,322 patients, combining data from 4 databases: Multicenter Osteoarthritis Study, Osteoarthritis Initiative, New England Baptist Hospital Orthopedic Registry, and Tufts Medical Center Orthopedic Surgery Registry
Analytic Approach	Multivariable linear regression, “greedy” matching computer algorithm, multiple imputation
Outcomes	Pain, based on observed or estimated Western Ontario and McMaster Universities Arthritis Index (WOMAC) score; functional status based on SF-12 Health Survey score

When conducting RCTs to compare treatments, researchers must recruit only patients with clinical equipoise, that is, patients for whom insufficient evidence exists to favor one treatment over another. When limited prior RCT evidence is available to identify patients with clinical equipoise, researchers can apply mathematical models to clinical registries, electronic health records (EHRs), and other non-RCT data to predict patient-specific outcomes of the treatments under study. If predicted outcomes are similar across treatments, called mathematical equipoise, random treatment assignment may be appropriate, and patients may wish to consider participating in an RCT.

For patients with knee osteoarthritis, the choice between total knee replacement and nonsurgical treatment is an important clinical question for which there are few RCTs. Nonsurgical treatment may include medication and/or physical therapy. In this study, the researchers developed Knee Osteoarthritis Mathematical Equipoise Tool (KOMET) software, with accompanying clinician and patient web-based interfaces, for use in EHR systems to

Help identify patients with mathematical equipoise who could consider enrolling in RCTs
Support decision making by providing patients with individualized, predicted outcomes for treatment options

To develop KOMET, the researchers used nonrandomized data from four databases to match total-knee-replacement knees to similar nonsurgical-treatment knees. The researchers then developed models to predict one-year outcomes for knee pain and functional status, modeling each outcome separately. Analysis consisted of three rounds of predictive modeling based on estimation and testing using data from various combinations of the four available databases. After identifying optimal prediction models for each outcome, the researchers programmed associated algorithms into the KOMET software.

During the study, knee osteoarthritis researchers, patients, clinicians, and patient advocates provided input on study questions, modeling issues, outcomes, and user interface development.

Results

The final model used for predicting WOMAC pain scores included main effects for baseline WOMAC knee pain, treatment type, and an interaction of the two. The predictors accounted for 32% of the variance in pain scores.

The final model used for predicting SF-12 functional status scores included main effects for age, gender, baseline SF-12 mental and physical component scores, body mass index, and treatment type. The predictors accounted for 34% of the variance in functional status scores.

Limitations

Differences among the various databases made it challenging to match comparable patients, which could have affected the models’ accuracy. The research team defined functional status using a measure of overall physical functioning; a survey that measured knee-specific functioning may have better captured meaningful knee function improvement. Many of the patient variables under consideration were burdensome to collect or difficult to capture in a consistent manner. Thus, the team chose to develop the KOMET software using algorithms from only the models that were based on databases containing a more limited set of predictor variables.

Conclusions and Relevance

The results of this study demonstrate the use of mathematical modeling for identifying potential enrollees in RCTs, or when RCT enrollment is not appropriate, for informing treatment decisions based on predicted outcomes.

Future Research Needs

Future research could test this approach with other medical conditions requiring important treatment decisions. Researchers could also develop approaches to lessen the bias inherent in nonrandomized data. In addition, using a more knee-specific functional scale, researchers could work to develop improved models to predict patient outcomes, applying newer statistical procedures and validation and using larger databases.

More on this Project

This project's final research report is expected to be available by April 2019.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.

Peer review identified the following strengths and limitations in the report:

The reviewers were impressed with the quality of the report and the interesting subject of this methods development study.
The reviewers questioned the 1:1 matching approach to create a dataset of matched patients with total knee replacement (TKR) and patients who chose medical therapy instead. The authors acknowledged that this matching approach could limit generalizability, as the medical therapy (non-TKR) patients were chosen to match the TKR patients, and not to provide a representative sample. But the authors noted that this approach reduces the effect of variables that influence choice of therapy and, independently, the outcome of therapy.

Conflict of Interest Disclosures

View the COI disclosure form

Project Details

Principal Investigator

Harry P. Selker, MD, MSPH

Project Status

Completed; PCORI Public and Professional Abstracts Posted

Project Title

A Method for Patient-Centered Enrollment in Comparative Effectiveness Trials: Mathematical Equipoise

Board Approval Date

December 2013

Project End Date

July 2018

Organization

Tufts Medical Center

Year Awarded

2013

State

Massachusetts

Year Completed

2018

Project Type

Research Project

Health Conditions

Muscular and Skeletal Disorders

Joint Replacement

Arthritis

Cardiovascular Diseases

Coronary or Ischemic Heart Disease

Funding Announcement

Improving Methods for Conducting Patient-Centered Outcomes Research

Project Budget

$1,123,588

Study Registration Information

HSRP20143597

Page Last Updated:

June 26, 2019