Results Summary

What was the research about?

Patients and their healthcare providers, such as doctors and nurses, can use survey scores to track the symptoms of illnesses like rheumatoid arthritis, or RA, over time. Tracking symptoms in this way can help them understand if a treatment is working well for a patient.

When researchers create and test these surveys, they want to be sure that patients’ survey scores match how severe patients feel their symptoms are. Researchers also want to know what changes in survey results show that symptoms have changed so much that patients might want to change treatment.

In this study, the research team had patients with RA and providers read stories that described what patients felt like with higher and lower scores of two symptoms:

  • Fatigue, or lack of energy
  • Pain interference, or how much pain interferes with their lives

Patients and providers decided whether each story showed a mild, moderate, or severe level of symptoms. They also gave their views about how large a change in scores would need to be to show that pain or fatigue was getting better or worse.

What were the results?

The research team found that patients varied in their opinions about which stories showed mild, moderate, or severe symptoms. Providers varied less. When choosing which stories show mild, moderate, or severe symptoms, patients chose higher scores compared with providers.

Patients and providers also agreed on the amount of change in scores that showed

  • Pain and fatigue were getting better
  • Fatigue was getting worse

But compared with providers, patients felt that a larger change in scores would be needed to show that pain was getting worse.

Who was in the study?

The study included 11 patients receiving treatment for RA and eight RA providers. Of the patients, six were white, three were black, one was Asian, and one was mixed race. The average age was 55. On average, patients had RA for 20 years. Of the providers, all were white, and the average age was 49.

What did the research team do?

First, the research team created a series of written stories about patients with RA and their symptoms. Each story represented a specific level of symptoms and included four or five descriptions of symptoms at that level.

The research team sorted the stories from least to most severe symptoms and then presented these stories to patients and providers. Patients and providers categorized which stories showed mild, moderate, or severe symptoms.

Next, patients and providers viewed a story that showed a patient with severe symptoms, followed by stories showing less severe symptoms. They then decided how much the symptoms would have to improve to show that a treatment for RA was working. They also viewed a story of a patient with mild symptoms, followed by other stories showing more severe symptoms. Patients and providers said how much symptoms would have to worsen to show that a patient needed a change in treatment.

Patients with RA and patient advocates helped design the study, analyze data, and interpret results.

What were the limits of the study?

The study included patients who’d had RA for a long time. Results might differ for patients who were diagnosed with RA more recently.

Future research could include patients who have more recent diagnoses of RA to understand how these patients think about symptom severity.

How can people use the results?

Researchers who work on surveys to track RA symptoms can take into account this study’s findings that patients and providers differed in the level of symptoms they thought were mild, moderate, or severe.

Final Research Report

View this project's final research report.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented, and the researchers made changes or provided responses. The comments and responses included the following:

  • Reviewers said the research described did not establish minimal important differences (MID) but provided only descriptive statistics about change scores associated with perceived change rather than statistical approaches noted in the literature to establish MID. The reviewers suggested that the authors focus on providing meaningful interpretations of change scores rather than MID, to avoid getting into a lengthy explanation of why they did not use some of the expected measures for calculating MID. The researchers explained that the methods and results related to MID were part of their funded-research plan.  Therefore, they could not be deleted from the report. However, the researchers did add a statement acknowledging that some observers might have technical objections to various approaches used in the study. They also stated that they would examine other methods for estimating MID beyond the methods used in the report.
  • Reviewers objected to the use of distribution-based methods for interpreting meaningful change, saying such methods indicate the amount of change but say nothing about its clinical relevance. The researchers explained that their intention in this methods-focused study was to evaluate different methods of estimating change, one of which was distribution-based methods, rather than determining the clinical relevance of that change.
  • Reviewers said it was not clear why the study did not ask patients about anxiety and depression separately but instead combined them into one domain, emotions. The researchers said that they could not change the way they had asked the question in their survey but agreed that, in retrospect, perhaps they should have separated anxiety and depression. They noted that they collected information on depression and anxiety in a different part of the study, aim 2.

Conflict of Interest Disclosures

Project Information

Clifton Bingham, MD
Johns Hopkins University
$742,198 *
Making PROMIS Meaningful to Patients and Providers in Clinical Practice

Key Dates

September 2014
December 2019

Study Registration Information

Final Research Report

View this project's final research report.

Journal Articles


Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: January 20, 2023