Results Summary

What was the research about?

Organizations that fund cancer research need to decide which studies to fund. Value of information (VOI) is a way to help rank research studies. VOI estimates the value of research by looking at the impacts on health and on healthcare that could result from the research.

SWOG (formerly the Southwest Oncology Group) is a network of cancer researchers funded by the National Cancer Institute. SWOG leaders review and score new study proposals based on the scientific value of the studies and their potential impact. Based on a study’s score, SWOG’s leadership committee decides whether to send the study to the National Cancer Institute for funding review.

In this study, the research team wanted to learn if giving VOI data to the committee affected its scoring of proposals. The team also wanted to see if providing VOI data was helpful in deciding which studies to fund. The study had two parts. The research team created a process to quickly estimate VOI. Then, the team tested the process on nine study proposals that the SWOG committee reviewed.

What were the results?

The research team’s new process to estimate VOI was much quicker than the normal process, taking less than one week instead of several months.

The VOI data caused the committee to change scores for eight of the nine proposals. Most of the time, the committee gave worse scores to proposals after looking at the VOI data. The VOI data didn’t change whether the committee approved these proposals.

Most committee members felt the VOI data were easy to understand (67 percent). Most thought the information was either helpful (50 percent) or neutral (42 percent). A few people (8 percent) felt the VOI data made their review harder. Forty-two percent thought adding VOI data to the review process was a good idea, 41 percent felt neutral, and 17 percent thought it was a bad idea.

What did the research team do?

The research team randomly selected nine proposals the SWOG committee had reviewed from 2008 to 2013. The team used these proposals to create the new process to quickly estimate VOI. The research team improved its process based on feedback from the SWOG committee and other SWOG researchers.

The research team then tested the process using nine new proposals that the SWOG committee received between February 2015 and December 2016. The research team used the new process to estimate VOI for each proposal. The team sent the resulting VOI data to the SWOG committee. The team also sent committee members information about VOI to help them understand the data. Committee members scored each proposal before and after looking at the VOI data.

The research team asked committee members to fill out a survey at the start of the study and again at the end. The survey asked how the committee decides which studies to fund. It also asked how the committee members felt about using VOI data.

What were the limits of the study?

The research team tested the process with a small number of cancer studies within one research network, SWOG. Results may be different for other research networks. The quicker process may not work as well as the longer process to estimate VOI. Members of the committee changed during the study. The changes made it hard to see if committee members’ opinions about VOI changed over time.

Future research could test the new process on more studies. Researchers could see if the process works for research networks other than SWOG.

How can people use the results?

Organizations that fund research may consider using the quicker process to estimate VOI to help choose which research studies to fund.

Final Research Report

View this project's final research report.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also confirms that the research has followed PCORI’s Methodology Standards. During peer review, experts who were not members of the research team read a draft report of the research. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. Reviewers do not have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve how the research team analyzed its results or reported its conclusions. Learn more about PCORI’s peer-review process here.

In response to peer review, Ramsey made changes including

  • Revising the Background section to make more evident what methodological gaps the study addressed, and to provide more justification for the importance of a value of information (VOI) approach to informing research prioritization
  • Revising the section on patient and stakeholder participation to describe participation in each of the phases separately
  • Adding to the Methods section a description of changes made to the original study protocol, in particular the reduction in study scope due to difficulties completing Aim 1

Conflict of Interest Disclosures

Project Information

Scott D. Ramsey, PhD, MD
Fred Hutchinson Cancer Research Center
$908,738
10.25302/12.2018.ME.13035889
A Structured Approach to Prioritizing Cancer Research Using Stakeholders and Value of Information

Key Dates

September 2013
September 2017
2013
2018

Study Registration Information

Tags

Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: January 20, 2023