Skip to main content
Patient-Centered Outcomes Research Institute
Patient-Centered Outcomes Research Institute
  • Blog
  • Newsroom
  • Find It Fast
  • Help Center
  • Subscribe
  • Careers
  • Contact Us

PCORI

Patient-Centered Outcomes Research Institute

Search form

  • About Us
    Close mega-menu

    About Us

    • Our Programs
    • Governance
    • Financials and Reports
    • Procurement Opportunities
    • Our Staff
    • Our Vision & Mission
    • Contact Us

    Fact Sheets: Learn More About PCORI

    Download fact sheets about out work, the research we fund, and our programs and initiatives.

    Find It Fast

    Browse through an alphabetical list of frequently accessed and searched terms for information and resources.

    Subscribe to PCORI Email Alerts

    Sign up for weekly emails to stay current on the latest results of our funded projects, and more.

  • Research & Results
    Close mega-menu

    Research & Results

    • Explore Our Portfolio
    • Research Fundamentals
    • Research Results Highlights
    • Putting Evidence to Work
    • Peer Review
    • Evidence Synthesis
    • About Our Research

    Evidence Updates from PCORI-Funded Studies

    These updates capture highlights of findings from systematic reviews and our funded research studies.

    Journal Articles About Our Funded Research

    Browse through a collection of journal publications that provides insights into PCORI-funded work.

    Explore Our Portfolio of Funded Projects

    Find out about projects based on the health conditions they focus on, the state they are in, and if they have results.

  • Topics
    Close mega-menu

    Topics

    • Addressing Disparities
    • Arthritis
    • Asthma
    • Cancer
    • Cardiovascular Disease
    • Children's Health
    • Community Health Workers
    • COVID-19
    • Dementia and Cognitive Impairment
    • Diabetes
    • Kidney Disease
    • Medicaid
    • Men's Health
    • Mental and Behavioral Health
    • Minority Mental Health
    • Multiple Chronic Conditions
    • Multiple Sclerosis
    • Obesity
    • Older Adults' Health
    • Pain Care and Opioids
    • Rare Diseases
    • Rural Health
    • Shared Decision Making
    • Telehealth
    • Transitional Care
    • Veterans Health
    • Women's Health

    Featured Topic: Women's Health

    Learn more about the projects we support on conditions that specifically or more often affect women.

  • Engagement
    Close mega-menu

    Engagement

    • The Value of Engagement
    • Engagement in Health Research Literature Explorer
    • Influencing the Culture of Research
    • Engagement Awards
    • Engagement Resources
    • Engage with Us

    Engagement Tools and Resources for Research

    This searchable peer-to-peer repository includes resources that can inform future work in patient-centered outcomes research.

    Explore Engagement in Health Literature

    This tool enables searching for published articles about engagement in health research.

    Research Fundamentals: A New On-Demand Training

    It enables those new to health research or patient-centered research to learn more about the research process.

  • Funding Opportunities
    Close mega-menu

    Funding Opportunities

    • What & Who We Fund
    • What You Need to Know to Apply
    • Applicant Training
    • Merit Review
    • Awardee Resources
    • Help Center

    PCORI Funding Opportunities

    View and learn about the newly opened funding announcements and the upcoming PFAs in 2021.

    Tips for Submitting a Responsive LOI

    Find out what PCORI looks for in a letter of intent (LOI) along with other helpful tips.

    PCORI Awardee Resources

    These resources can help awardees in complying with the terms and conditions of their contract.

  • Meetings & Events
    Close mega-menu

    Meetings & Events

    • Upcoming
    • Past Events

    January 2021 Board of Governors Meeting

    The Board approved funding for a new research study relating to kidney health and a new funding allocation for PCORnet. Learn more

    Confronting COVID-19: A Webinar Series

    Learn more about the series and access recordings and summary reports of all six sessions.

    2020 PCORI Annual Meeting

    Watch recordings of all sessions, and view titles and descriptions of the posters presented at the virtual meeting.

You are here

  • Research & Results
  • Explore Our Portfolio
  • Having Patients and Providers Identif...

This project has results

Having Patients and Providers Identify Meaningful Changes in Rheumatoid Arthritis Symptoms

Sign Up for Updates to This Study  

Results Summary and Professional Abstract

Results Summary

Results Summary

Download Summary

What was the research about?

Patients and their healthcare providers, such as doctors and nurses, can use survey scores to track the symptoms of illnesses like rheumatoid arthritis, or RA, over time. Tracking symptoms in this way can help them understand if a treatment is working well for a patient.

When researchers create and test these surveys, they want to be sure that patients’ survey scores match how severe patients feel their symptoms are. Researchers also want to know what changes in survey results show that symptoms have changed so much that patients might want to change treatment.

In this study, the research team had patients with RA and providers read stories that described what patients felt like with higher and lower scores of two symptoms:

  • Fatigue, or lack of energy
  • Pain interference, or how much pain interferes with their lives

Patients and providers decided whether each story showed a mild, moderate, or severe level of symptoms. They also gave their views about how large a change in scores would need to be to show that pain or fatigue was getting better or worse.

What were the results?

The research team found that patients varied in their opinions about which stories showed mild, moderate, or severe symptoms. Providers varied less. When choosing which stories show mild, moderate, or severe symptoms, patients chose higher scores compared with providers.

Patients and providers also agreed on the amount of change in scores that showed

  • Pain and fatigue were getting better
  • Fatigue was getting worse

But compared with providers, patients felt that a larger change in scores would be needed to show that pain was getting worse.

Who was in the study?

The study included 11 patients receiving treatment for RA and eight RA providers. Of the patients, six were white, three were black, one was Asian, and one was mixed race. The average age was 55. On average, patients had RA for 20 years. Of the providers, all were white, and the average age was 49.

What did the research team do?

First, the research team created a series of written stories about patients with RA and their symptoms. Each story represented a specific level of symptoms and included four or five descriptions of symptoms at that level.

The research team sorted the stories from least to most severe symptoms and then presented these stories to patients and providers. Patients and providers categorized which stories showed mild, moderate, or severe symptoms.

Next, patients and providers viewed a story that showed a patient with severe symptoms, followed by stories showing less severe symptoms. They then decided how much the symptoms would have to improve to show that a treatment for RA was working. They also viewed a story of a patient with mild symptoms, followed by other stories showing more severe symptoms. Patients and providers said how much symptoms would have to worsen to show that a patient needed a change in treatment.

Patients with RA and patient advocates helped design the study, analyze data, and interpret results.

What were the limits of the study?

The study included patients who’d had RA for a long time. Results might differ for patients who were diagnosed with RA more recently.

Future research could include patients who have more recent diagnoses of RA to understand how these patients think about symptom severity.

How can people use the results?

Researchers who work on surveys to track RA symptoms can take into account this study’s findings that patients and providers differed in the level of symptoms they thought were mild, moderate, or severe.

Professional Abstract

Professional Abstract

Objective

To estimate thresholds for mild, moderate, and severe levels of rheumatoid arthritis (RA) symptoms as measured by the PROMIS® Pain Interference and Fatigue scores and to estimate changes in PROMIS scores that indicate clinically meaningful changes in symptoms based on patient and provider assessment

Study Design

Design Elements Description
Design Empirical analysis
Data Sources and Data Sets Patient and provider reactions to vignettes representing a range of PROMIS scores
Analytic Approach Bookmarking method to identify thresholds for PROMIS scores; evaluation of patient and provider choice of thresholds for dividing vignettes into categories of mild, moderate, and severe symptoms; evaluation of meaningful changes in PROMIS scores identified by patients and providers
Outcomes Thresholds for mild, moderate, and severe PROMIS Pain Interference and Fatigue scores; clinically meaningful changes in scores that indicate improvement or worsening of symptoms

Currently, researchers do not have a good understanding of how patients and providers with specific conditions, like RA, characterize their symptoms in terms of different levels of PROMIS scores. This empirical analysis sought to understand how patients and providers classify PROMIS Pain Interference and Fatigue scores and what patients and providers consider as meaningful changes in scores.

First, the research team created a series of patient vignettes describing the level of symptoms that a hypothetical patient would experience if they had a specific PROMIS score. PROMIS scores have a mean of 50 and a standard deviation of 10 in the general US population. For example, a vignette for a PROMIS fatigue score of 62 might describe a person as rarely having enough energy for strenuous exercise and often feeling too fatigued to plan for future activities. The team created one set of nine vignettes for PROMIS measures of pain interference and another for measures of fatigue.

The research team had patients and providers go through three scenarios. First, the team sorted the vignettes from least to most severe and presented the set of vignettes to the patients and providers. Patients and providers then indicated where they felt transitions between vignettes showed an increase in symptom severity (for example, from moderate to severe symptoms).

Second, patients and providers viewed another set of vignettes arranged from high to low levels of either pain or fatigue. Starting with a vignette with high symptom levels, patients and providers suggested which subsequent vignette would show that a treatment was working. Third, patients and providers viewed a vignette representing a low level of either pain or fatigue and suggested which subsequent vignette would indicate symptoms worsened so much that a change in treatment should be considered.

The research team recruited 11 patients receiving treatment for RA and eight rheumatology providers. Of the patients, six were white, three were black, one was Asian, and one was mixed race. The average age was 55. On average, patients had RA for 20 years. Providers included physicians, nurses, and psychologists. All providers were white, with an average age of 49.

Patients with RA and patient advocates helped design the study, analyze data, and interpret results.

Results

Patients varied widely in the thresholds they selected for mild, moderate, and severe symptoms; providers varied less. Patients also selected higher thresholds than providers for mild, moderate, and severe pain or fatigue.

When considering meaningful reduction in pain or fatigue, both patients and providers selected a vignette representing a 10-point change in PROMIS scores. For an increase in pain, patients selected a 10-point change, while providers suggested only a 5-point change. For an increase in fatigue, both patients and providers suggested a 5-point change.

Limitations

The study included patients who had had RA for a long time. Results might differ for patients diagnosed with RA more recently.

Conclusions and Relevance

The research team found that, compared with providers, patients differed in the level of symptoms they considered to be mild, moderate, and severe. Providers were more sensitive to changes in levels of pain interference.

Future Research Needs

Future research could include patients who have more recent diagnoses of RA to understand how these patients think about symptom severity.

Final Research Report

View this project's final research report.

Journal Articles

Related Articles

Arthritis Care and Research

What Does the Patient Global Health Assessment in Rheumatoid Arthritis Really Tell Us? Contribution of Specific Dimensions of Health-Related Quality of Life

Arthritis Care and Research

Identifying Minimal and Meaningful Change in PROMIS(®) for Rheumatoid Arthritis: Use of Multiple Methods and Perspectives

Journal of Patient-Reported Outcomes

PROMIS Fatigue short forms are reliable and valid in adults with rheumatoid arthritis

Journal of Rheumatology

Perceptions of Patients, Caregivers, and Healthcare Providers of Idiopathic Inflammatory Myopathies: An International OMERACT Study

Quality of Life Research

Combining online and in-person methods to evaluate the content validity of PROMIS fatigue short forms in rheumatoid arthritis

The Journal of Rheumatology

Content and Construct Validity, Reliability, and Responsiveness of the Rheumatoid Arthritis Flare Questionnaire: OMERACT 2016 Workshop Report

RMD Open

Identifying flares in rheumatoid arthritis: reliability and construct validation of the OMERACT RA Flare Core Domain Set

More on this Project  

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented, and the researchers made changes or provided responses. The comments and responses included the following:

  • Reviewers said the research described did not establish minimal important differences (MID) but provided only descriptive statistics about change scores associated with perceived change rather than statistical approaches noted in the literature to establish MID. The reviewers suggested that the authors focus on providing meaningful interpretations of change scores rather than MID, to avoid getting into a lengthy explanation of why they did not use some of the expected measures for calculating MID. The researchers explained that the methods and results related to MID were part of their funded-research plan.  Therefore, they could not be deleted from the report. However, the researchers did add a statement acknowledging that some observers might have technical objections to various approaches used in the study. They also stated that they would examine other methods for estimating MID beyond the methods used in the report.
  • Reviewers objected to the use of distribution-based methods for interpreting meaningful change, saying such methods indicate the amount of change but say nothing about its clinical relevance. The researchers explained that their intention in this methods-focused study was to evaluate different methods of estimating change, one of which was distribution-based methods, rather than determining the clinical relevance of that change.
  • Reviewers said it was not clear why the study did not ask patients about anxiety and depression separately but instead combined them into one domain, emotions. The researchers said that they could not change the way they had asked the question in their survey but agreed that, in retrospect, perhaps they should have separated anxiety and depression. They noted that they collected information on depression and anxiety in a different part of the study, aim 2.

Conflict of Interest Disclosures

View the COI disclosure form.

Project Details

Principal Investigator
Clifton Bingham, MD
Project Status
Completed; PCORI Public and Professional Abstracts, and Final Research Report Posted
Project Title
Making PROMIS Meaningful to Patients and Providers in Clinical Practice
Board Approval Date
September 2014
Project End Date
December 2019
Organization
Johns Hopkins University
Year Awarded
2014
State
Maryland
Year Completed
2019
Project Type
Research Project
Health Conditions  
Allergies and Immune Disorders
Immune Disorders
Muscular and Skeletal Disorders
Arthritis
Funding Announcement
Improving Methods for Conducting Patient-Centered Outcomes Research
Project Budget
$767,229
DOI - Digital Object Identifier
10.25302/04.2020.ME.140210818
Study Registration Information
HSRP20152294
Page Last Updated: 
June 11, 2020

About Us

  • Our Programs
  • Governance
  • Financials and Reports
  • Procurement Opportunities
  • Our Staff
  • Our Vision & Mission
  • Contact Us

Research & Results

  • Explore Our Portfolio
  • Research Fundamentals
  • Research Results Highlights
  • Putting Evidence to Work
  • Peer Review
  • Evidence Synthesis
  • About Our Research

Engagement

  • The Value of Engagement
  • Engagement in Health Research Literature Explorer
  • Influencing the Culture of Research
  • Engagement Awards
  • Engagement Resources
  • Engage with Us

Funding Opportunities

  • What & Who We Fund
  • What You Need to Know to Apply
  • Applicant Training
  • Merit Review
  • Awardee Resources
  • Help Center

Meetings & Events

January 21
Cycle 1 2021 Broad PFA Applicant Town Hall
February 2
PCORI 2021 and Beyond: Opportunities for Funding and Involvement in Patient-Centered Research
February 9
Board of Governors Meeting: February 9, 2021

PCORI

Footer contact address

Patient-Centered Outcomes
Research Institute

1828 L Street, NW, Suite 900
Washington, DC 20036
Phone: (202) 827-7700 | Fax: (202) 355-9558
[email protected]

Subscribe to Newsletter

Twitter Facebook LinkedIn Vimeo

© 2011-2021 Patient-Centered Outcomes Research Institute. All Rights Reserved.

Privacy Policy | Terms of Use | Trademark Usage Guidelines | Credits | Help Center