Results Summary

What was the project about?

Patient-reported outcome measures, or PROMs, are surveys that ask patients how they feel and what activities they can do. All PROMs have measurement properties. These properties help researchers understand how well the PROM was designed. For example, one property is the ability to get consistent responses over time. Another is how accurately the PROM measures how patients feel about a health problem. But researchers may not consistently report these properties in studies about PROMs.

In this study, the research team created guidance about reporting measurement properties in studies on PROM.

What did the research team do?

First, the research team reviewed 1,658 studies on PROMs. From these, the team created a list of 127 recommendations for reporting measurement properties. Then the team recruited 47 people with expertise on PROMs. The experts had published

  • Guidelines for PROM measurement properties
  • Manuals on PROM measurement
  • Reviews of methods for creating PROMs
  • Reporting guidelines for clinical research

The research team sent three rounds of online surveys to the group of experts. The group rated the importance of each recommendation. They also suggested other reporting recommendations. The team looked at agreement among responses. If less than 70 percent of experts agreed with a recommendation, the team revised it and sent it out again for review.

What were the results?

The study resulted in a final set of 71 reporting recommendations that included

  • 35 for all studies about PROMs
  • 36 for specific measurement properties, such as accuracy or ability to get consistent responses

What were the limits of the project?

The experts who took part in this study may have specific views on measurement. The final set of recommendations may have differed if the study had included experts from other backgrounds.

Future research could develop reporting recommendations for other aspects of PROMs, such as when researchers test how well PROMs work across cultures.

How can people use the results?

Researchers can use the guidance to report useful information in studies on PROMs.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:

  • The reviewers had questions about the literature review that was used to collect patient-reported outcome measures (PROMs). Specifically, reviewers expressed concern about whether the literature review was comprehensive and suggested that there could have been bias in the guidelines identified. The researchers said they reviewed literature from a wide variety of fields, including medicine, psychology, and sociology. They said they believed it was unlikely that their search was biased because it was so extensive. The researchers offered a supplementary file listing the thousands of citations found in their searches.
  • The reviewers said the literature search may have missed many important documents because of the terminology used or omitted in the searches. The reviewers questioned the value of the consensus generated in this project compared with existing guidelines that experts in psychometrics have developed. The researchers said they included many psychometricians among the experts who reviewed the collected PROMs. The researchers said they did use old terminology in their literature review but said these terms were important to use since they are still used frequently in the literature. The researchers added that their goal was not to develop a framework for the field of psychometrics but to organize reporting guidelines for patient outcomes.
  • The reviewers said that the report was not sufficiently detailed about how the researchers developed their list of items that need to be reported in studies of PROMs. The researchers added more details of their stepped process to identify items first by examining key resources providing reporting guidelines, then completing a literature review to determine whether there were other items that need to be reported for PROMs studies but were not listed in those key resources. The researchers also added tables with the final list of general reporting expectations for any study testing the properties of PROMs, and the final list of expectations specific to each of the nine PROMs measurement properties.

Conflict of Interest Disclosures

Project Information

Joel Joseph Gagnier, PhD ND, MSc
University of Michigan
$348,189
10.25302/03.2021.ME.160635556
Development of Reporting Guidelines for Psychometric Research on Patient-Reported Outcome Measures

Key Dates

December 2016
November 2020
2016
2020

Study Registration Information

Tags

Has Results
Project Status
Award Type
Funding Opportunity Type
State

State

The state where the project originates, or where the primary institution or organization is located.

View Glossary
Last updated: December 27, 2021