Skip to main content
Patient-Centered Outcomes Research Institute
Patient-Centered Outcomes Research Institute
  • Blog
  • Newsroom
  • Find It Fast
  • Help Center
  • Subscribe
  • Careers
  • Contact Us

PCORI

Patient-Centered Outcomes Research Institute

Search form

  • About Us
    Close mega-menu

    About Us

    • Our Programs
    • Governance
    • Financials and Reports
    • Procurement Opportunities
    • Our Staff
    • Our Vision & Mission
    • Contact Us

    Fact Sheets: Learn More About PCORI

    Download fact sheets about out work, the research we fund, and our programs and initiatives.

    Find It Fast

    Browse through an alphabetical list of frequently accessed and searched terms for information and resources.

    Subscribe to PCORI Email Alerts

    Sign up for weekly emails to stay current on the latest results of our funded projects, and more.

  • Research & Results
    Close mega-menu

    Research & Results

    • Explore Our Portfolio
    • Research Fundamentals
    • Research Results Highlights
    • Putting Evidence to Work
    • Peer Review
    • Evidence Synthesis
    • About Our Research

    Evidence Updates from PCORI-Funded Studies

    These updates capture highlights of findings from systematic reviews and our funded research studies.

    Journal Articles About Our Funded Research

    Browse through a collection of journal publications that provides insights into PCORI-funded work.

    Explore Our Portfolio of Funded Projects

    Find out about projects based on the health conditions they focus on, the state they are in, and if they have results.

  • Topics
    Close mega-menu

    Topics

    • Addressing Disparities
    • Arthritis
    • Asthma
    • Cancer
    • Cardiovascular Disease
    • Children's Health
    • Community Health Workers
    • COVID-19
    • Dementia and Cognitive Impairment
    • Diabetes
    • Kidney Disease
    • Medicaid
    • Men's Health
    • Mental and Behavioral Health
    • Minority Mental Health
    • Multiple Chronic Conditions
    • Multiple Sclerosis
    • Obesity
    • Older Adults' Health
    • Pain Care and Opioids
    • Rare Diseases
    • Rural Health
    • Shared Decision Making
    • Telehealth
    • Transitional Care
    • Veterans Health
    • Women's Health

    Featured Topic: Women's Health

    Learn more about the projects we support on conditions that specifically or more often affect women.

  • Engagement
    Close mega-menu

    Engagement

    • The Value of Engagement
    • Engagement in Health Research Literature Explorer
    • Influencing the Culture of Research
    • Engagement Awards
    • Engagement Resources
    • Engage with Us

    Engagement Tools and Resources for Research

    This searchable peer-to-peer repository includes resources that can inform future work in patient-centered outcomes research.

    Explore Engagement in Health Literature

    This tool enables searching for published articles about engagement in health research.

    Research Fundamentals: A New On-Demand Training

    It enables those new to health research or patient-centered research to learn more about the research process.

  • Funding Opportunities
    Close mega-menu

    Funding Opportunities

    • What & Who We Fund
    • What You Need to Know to Apply
    • Applicant Training
    • Merit Review
    • Awardee Resources
    • Help Center

    PCORI Funding Opportunities

    View and learn about the newly opened funding announcements and the upcoming PFAs in 2021.

    Tips for Submitting a Responsive LOI

    Find out what PCORI looks for in a letter of intent (LOI) along with other helpful tips.

    PCORI Awardee Resources

    These resources can help awardees in complying with the terms and conditions of their contract.

  • Meetings & Events
    Close mega-menu

    Meetings & Events

    • Upcoming
    • Past Events

    January 2021 Board of Governors Meeting

    The Board approved funding for a new research study relating to kidney health and a new funding allocation for PCORnet. Learn more

    Confronting COVID-19: A Webinar Series

    Learn more about the series and access recordings and summary reports of all six sessions.

    2020 PCORI Annual Meeting

    Watch recordings of all sessions, and view titles and descriptions of the posters presented at the virtual meeting.

You are here

  • Research & Results
  • Explore Our Portfolio
  • Creating Recommendations on Types of ...

This project has results

Creating Recommendations on Types of Data to Present in Patient-Reported Outcome Measure Development Studies

Sign Up for Updates to This Study  

Results Summary and Professional Abstract

Results Summary

Results Summary

Download Summary

What was the project about?

Patient-reported outcome measures, or PROMs, are surveys that ask patients how they feel and what activities they can do. All PROMs have measurement properties. These properties help researchers understand how well the PROM was designed. For example, one property is the ability to get consistent responses over time. Another is how accurately the PROM measures how patients feel about a health problem. But researchers may not consistently report these properties in studies about PROMs.

In this study, the research team created guidance about reporting measurement properties in studies on PROM.

What did the research team do?

First, the research team reviewed 1,658 studies on PROMs. From these, the team created a list of 127 recommendations for reporting measurement properties. Then the team recruited 47 people with expertise on PROMs. The experts had published

  • Guidelines for PROM measurement properties
  • Manuals on PROM measurement
  • Reviews of methods for creating PROMs
  • Reporting guidelines for clinical research

The research team sent three rounds of online surveys to the group of experts. The group rated the importance of each recommendation. They also suggested other reporting recommendations. The team looked at agreement among responses. If less than 70 percent of experts agreed with a recommendation, the team revised it and sent it out again for review.

What were the results?

The study resulted in a final set of 71 reporting recommendations that included

  • 35 for all studies about PROMs
  • 36 for specific measurement properties, such as accuracy or ability to get consistent responses

What were the limits of the project?

The experts who took part in this study may have specific views on measurement. The final set of recommendations may have differed if the study had included experts from other backgrounds.

Future research could develop reporting recommendations for other aspects of PROMs, such as when researchers test how well PROMs work across cultures.

How can people use the results?

Researchers can use the guidance to report useful information in studies on PROMs.

Professional Abstract

Professional Abstract

Background

Studies on the measurement properties of patient-reported outcome measures (PROMs) often lack information that could help a reader understand what methods were used, what the results are, and if the study provides evidence about why researchers should use a particular PROM. Guidelines on what to report and how to report this information do not currently exist.

Objective

To develop a set of reporting recommendations for studies examining PROM measurement properties.

Study Design

Design Element Description
Design Empirical analysis
Data Sources and Data Sets Published and unpublished literature on guidance or reporting for PROM measurement properties; surveys of panelists with experience in PROM development methodology
Analytic Approach Literature review of existing guidelines for reporting PROM measurement properties; Delphi study of panelists with experience in PROM development methodology to identify relevant guidelines
Outcomes Consensus-based reporting recommendations for primary studies of PROM measurement properties

Methods and Results

First, the research team reviewed the methods for reporting PROM measurement properties in 1,658 studies. From these studies, the team created 127 reporting recommendations for PROM measurement properties and grouped these recommendations into two sets. One set included common recommendation items relevant to any study of measurement properties, and the other set was specific to nine reliability- and validity-related measurement properties.

Second, the research team recruited a group of 47 individuals who they considered to be experts in PROM development due to having published one of the following:

  • Reporting guidelines for measurement property studies of PROMs
  • Manuals on psychometrics or measurement
  • Systematic reviews of methods or reporting of measurement studies of PROMs
  • Reporting guidelines for clinical research

The research team then conducted three Delphi rounds with these experts. In the first round, the team sent the group an online survey. Participants rated the importance of each reporting recommendation on a 5-point scale and suggested additional reporting recommendations. The team deemed recommendations to be important if 70% of participants agreed or strongly agreed that the items were important. In the next two rounds, the team presented revised versions of the items that had less than 70% participant agreement in the previous rounds.

The Delphi panel achieved consensus on a final set of 71 reporting recommendations separated into

  • 35 common recommendations relevant to all studies on measurement properties
  • 36 specific recommendations relevant to one of the nine measurement properties: reliability, measurement error, internal consistency, content validity, face validity, construct validity, criterion validity, responsiveness, and interpretability

Limitations

Individuals’ views on reporting specific measurement properties may have made them more likely to volunteer and respond, biasing the voting and the resultant final set of recommendations.

Conclusions and Relevance

The research team created consensus-based recommendations to be used in studies that report PROM measurement properties. These recommendations can be used to help report on future studies of measurement properties.

Future Research Needs

Future research could develop additional reporting recommendations for other aspects of PROM development, such as item response theory or cross-cultural validity and translations.

This project's final research report is expected to be available by August 2020.

More on this Project  

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:

  • The reviewers had questions about the literature review that was used to collect patient-reported outcome measures (PROMs). Specifically, reviewers expressed concern about whether the literature review was comprehensive and suggested that there could have been bias in the guidelines identified. The researchers said they reviewed literature from a wide variety of fields, including medicine, psychology, and sociology. They said they believed it was unlikely that their search was biased because it was so extensive. The researchers offered a supplementary file listing the thousands of citations found in their searches.
  • The reviewers said the literature search may have missed many important documents because of the terminology used or omitted in the searches. The reviewers questioned the value of the consensus generated in this project compared with existing guidelines that experts in psychometrics have developed. The researchers said they included many psychometricians among the experts who reviewed the collected PROMs. The researchers said they did use old terminology in their literature review but said these terms were important to use since they are still used frequently in the literature. The researchers added that their goal was not to develop a framework for the field of psychometrics but to organize reporting guidelines for patient outcomes.
  • The reviewers said that the report was not sufficiently detailed about how the researchers developed their list of items that need to be reported in studies of PROMs. The researchers added more details of their stepped process to identify items first by examining key resources providing reporting guidelines, then completing a literature review to determine whether there were other items that need to be reported for PROMs studies but were not listed in those key resources. The researchers also added tables with the final list of general reporting expectations for any study testing the properties of PROMs, and the final list of expectations specific to each of the nine PROMs measurement properties.

Conflict of Interest Disclosures

The COI disclosure form will be posted here soon.

Project Details

Principal Investigator
Joel Joseph Gagnier, PhD ND, MSc
Project Status
Completed; PCORI Public and Professional Abstracts Posted
Project Title
Development of Reporting Guidelines for Psychometric Research on Patient-Reported Outcome Measures
Board Approval Date
December 2016
Project End Date
November 2020
Organization
University of Michigan
Year Awarded
2016
State
Michigan
Year Completed
2020
Project Type
Research Project
Funding Announcement
Improving Methods for Conducting Patient-Centered Outcomes Research
Project Budget
$348,189
Study Registration Information
HSRP20181636
Page Last Updated: 
December 17, 2020

About Us

  • Our Programs
  • Governance
  • Financials and Reports
  • Procurement Opportunities
  • Our Staff
  • Our Vision & Mission
  • Contact Us

Research & Results

  • Explore Our Portfolio
  • Research Fundamentals
  • Research Results Highlights
  • Putting Evidence to Work
  • Peer Review
  • Evidence Synthesis
  • About Our Research

Engagement

  • The Value of Engagement
  • Engagement in Health Research Literature Explorer
  • Influencing the Culture of Research
  • Engagement Awards
  • Engagement Resources
  • Engage with Us

Funding Opportunities

  • What & Who We Fund
  • What You Need to Know to Apply
  • Applicant Training
  • Merit Review
  • Awardee Resources
  • Help Center

Meetings & Events

January 21
Cycle 1 2021 Broad PFA Applicant Town Hall
February 2
PCORI 2021 and Beyond: Opportunities for Funding and Involvement in Patient-Centered Research
February 9
Board of Governors Meeting: February 9, 2021

PCORI

Footer contact address

Patient-Centered Outcomes
Research Institute

1828 L Street, NW, Suite 900
Washington, DC 20036
Phone: (202) 827-7700 | Fax: (202) 355-9558
[email protected]

Subscribe to Newsletter

Twitter Facebook LinkedIn Vimeo

© 2011-2021 Patient-Centered Outcomes Research Institute. All Rights Reserved.

Privacy Policy | Terms of Use | Trademark Usage Guidelines | Credits | Help Center