Results Summary
What was the project about?
Patient-reported outcome measures, or PROMs, are surveys that ask patients how they feel and what activities they can do. All PROMs have measurement properties. These properties help researchers understand how well the PROM was designed. For example, one property is the ability to get consistent responses over time. Another is how accurately the PROM measures how patients feel about a health problem. But researchers may not consistently report these properties in studies about PROMs.
In this study, the research team created guidance about reporting measurement properties in studies on PROM.
What did the research team do?
First, the research team reviewed 1,658 studies on PROMs. From these, the team created a list of 127 recommendations for reporting measurement properties. Then the team recruited 47 people with expertise on PROMs. The experts had published
- Guidelines for PROM measurement properties
- Manuals on PROM measurement
- Reviews of methods for creating PROMs
- Reporting guidelines for clinical research
The research team sent three rounds of online surveys to the group of experts. The group rated the importance of each recommendation. They also suggested other reporting recommendations. The team looked at agreement among responses. If less than 70 percent of experts agreed with a recommendation, the team revised it and sent it out again for review.
What were the results?
The study resulted in a final set of 71 reporting recommendations that included
- 35 for all studies about PROMs
- 36 for specific measurement properties, such as accuracy or ability to get consistent responses
What were the limits of the project?
The experts who took part in this study may have specific views on measurement. The final set of recommendations may have differed if the study had included experts from other backgrounds.
Future research could develop reporting recommendations for other aspects of PROMs, such as when researchers test how well PROMs work across cultures.
How can people use the results?
Researchers can use the guidance to report useful information in studies on PROMs.
Professional Abstract
Background
Studies on the measurement properties of patient-reported outcome measures (PROMs) often lack information that could help a reader understand what methods were used, what the results are, and if the study provides evidence about why researchers should use a particular PROM. Guidelines on what to report and how to report this information do not currently exist.
Objective
To develop a set of reporting recommendations for studies examining PROM measurement properties.
Study Design
| Design Element | Description |
|---|---|
| Design | Empirical analysis |
| Data Sources and Data Sets | Published and unpublished literature on guidance or reporting for PROM measurement properties; surveys of panelists with experience in PROM development methodology |
| Analytic Approach | Literature review of existing guidelines for reporting PROM measurement properties; Delphi study of panelists with experience in PROM development methodology to identify relevant guidelines |
| Outcomes | Consensus-based reporting recommendations for primary studies of PROM measurement properties |
Methods and Results
First, the research team reviewed the methods for reporting PROM measurement properties in 1,658 studies. From these studies, the team created 127 reporting recommendations for PROM measurement properties and grouped these recommendations into two sets. One set included common recommendation items relevant to any study of measurement properties, and the other set was specific to nine reliability- and validity-related measurement properties.
Second, the research team recruited a group of 47 individuals who they considered to be experts in PROM development due to having published one of the following:
- Reporting guidelines for measurement property studies of PROMs
- Manuals on psychometrics or measurement
- Systematic reviews of methods or reporting of measurement studies of PROMs
- Reporting guidelines for clinical research
The research team then conducted three Delphi rounds with these experts. In the first round, the team sent the group an online survey. Participants rated the importance of each reporting recommendation on a 5-point scale and suggested additional reporting recommendations. The team deemed recommendations to be important if 70% of participants agreed or strongly agreed that the items were important. In the next two rounds, the team presented revised versions of the items that had less than 70% participant agreement in the previous rounds.
The Delphi panel achieved consensus on a final set of 71 reporting recommendations separated into
- 35 common recommendations relevant to all studies on measurement properties
- 36 specific recommendations relevant to one of the nine measurement properties: reliability, measurement error, internal consistency, content validity, face validity, construct validity, criterion validity, responsiveness, and interpretability
Limitations
Individuals’ views on reporting specific measurement properties may have made them more likely to volunteer and respond, biasing the voting and the resultant final set of recommendations.
Conclusions and Relevance
The research team created consensus-based recommendations to be used in studies that report PROM measurement properties. These recommendations can be used to help report on future studies of measurement properties.
Future Research Needs
Future research could develop additional reporting recommendations for other aspects of PROM development, such as item response theory or cross-cultural validity and translations.
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers had questions about the literature review that was used to collect patient-reported outcome measures (PROMs). Specifically, reviewers expressed concern about whether the literature review was comprehensive and suggested that there could have been bias in the guidelines identified. The researchers said they reviewed literature from a wide variety of fields, including medicine, psychology, and sociology. They said they believed it was unlikely that their search was biased because it was so extensive. The researchers offered a supplementary file listing the thousands of citations found in their searches.
- The reviewers said the literature search may have missed many important documents because of the terminology used or omitted in the searches. The reviewers questioned the value of the consensus generated in this project compared with existing guidelines that experts in psychometrics have developed. The researchers said they included many psychometricians among the experts who reviewed the collected PROMs. The researchers said they did use old terminology in their literature review but said these terms were important to use since they are still used frequently in the literature. The researchers added that their goal was not to develop a framework for the field of psychometrics but to organize reporting guidelines for patient outcomes.
- The reviewers said that the report was not sufficiently detailed about how the researchers developed their list of items that need to be reported in studies of PROMs. The researchers added more details of their stepped process to identify items first by examining key resources providing reporting guidelines, then completing a literature review to determine whether there were other items that need to be reported for PROMs studies but were not listed in those key resources. The researchers also added tables with the final list of general reporting expectations for any study testing the properties of PROMs, and the final list of expectations specific to each of the nine PROMs measurement properties.