Methods Supplement to Support Innovative Research on AI and Large Language Models in Patient-Centered CER (Invitation Only)
- All Funding Opportunities
- What & Who We Fund
- What You Need to Know to Apply
- Merit Review
|Applications for this supplemental funding opportunity are now under review.
Funding approval notifications will be delivered by end-February 2024.
PCORI’s mission is to produce knowledge that allows individuals to make informed healthcare decisions and that improves healthcare delivery and outcomes. Through its portfolio of methodological research, PCORI seeks to fund projects that address important methodological gaps in patient-centered comparative clinical effectiveness research (CER).
Advances in artificial intelligence (AI) and natural language processing are occurring at a rapid pace. Large language models (LLMs), a general-purpose type of language AI, have advanced quickly with the release and rapid adoption of ChatGPT, GPT-4 and others. However, these advances require further methodological development and rigorous evaluation before being widely adopted in clinical research and clinical care.
Given significant new opportunities to leverage LLMs for patient-centered CER, as well as the need for rigorous study of these approaches, PCORI seeks investigator-initiated requests to build on existing, PCORI-funded methods projects to improve the understanding of emerging innovations in LLMs.
Only PCORI-funded principal investigators/institutions with active contracts previously awarded under Improving Methods for Conducting Patient-Centered Outcomes Research (Methods) PCORI Funding Announcements are eligible to submit a request in response to this opportunity.
The purpose of this supplemental funding opportunity is to allow current Methods awardees to incorporate or address recent breakthroughs in LLMs into their ongoing Methods projects. This opportunity provides a mechanism for current Methods awardees to supplement or amend their aims to respond to fast-moving developments in LLMs in ways that will increase the relevance and impact of their PCORI-funded work.
Proposals must be related to the use or implications of recent developments in LLMs that will advance the funded methodological research project underway. Areas of interest include but are not limited to:
- Developing new LLMs
- Calibrating existing LLMs to new data
- Developing approaches to improve multimodal data streams (e.g., extracting and incorporating information from images, audio/video, patient reported/generated data from sensor/wearable/personal devices)
- Developing experimental methods, tests or other approaches to assess LLMs’ validity and/or safety
- Considering or modeling potential sources of bias
- Proposing or developing approaches to improve research study management by enhancing study efficiency (e.g., reducing information loss such as missing data, loss to follow-up)
- Proposing or developing approaches to improve patient-centeredness and diverse participation in research, including patient engagement, and approaches to return results to study participants
- Considering governance implications for patient consent or strategies to improve informed consent
- Considering the ethical implications of these developments for clinical research
PCORI’s 2021 Emerging Technologies and Therapeutics Report on Ai in Clinical Care addressed a number of perspectives from patients and CER stakeholders about the use of AI technologies in health care. During the interviews conducted for the report, stakeholders expressed concerns regarding the applications of AI in health: perpetuation and amplification of bias, an unequal distribution of benefits and harms, violation of individual privacy, and the threat of malicious attacks on data and devices. Though AI developments have accelerated since the report was published, these considerations are highly relevant to methodological research and should be considered by investigators when proposing work under this supplement.
- Awarded contracts must:
- Have been funded under Improving Methods for Conducting Patient-Centered Outcomes Research PFAs issued in Cycle 1 2019 or later.
- Have the Draft Final Research Report (DFRR) due on or after June 30, 2024.
- Be deemed by PCORI staff to be in good standing. Metrics for assessment include:
- On-time completion of project milestones to-date.
- Cumulative enrollment progress (if the project includes prospective enrollment of participants and is anticipated to have started enrollment by the supplement application receipt date).
- Have not received a cumulative extension of greater than 18 months to the original executed contract’s research period of performance end date (i.e., the Final Progress Report due date).
- Requested supplemental funding must:
- Capitalize on fast-moving developments in LLMs.
- Support and extend the original project aims of the Methods award.
- Awardees that received a prior award under the PCORI COVID-19 enhancement mechanism are eligible to apply.
Supplements for all projects should seek to leverage existing project personnel expertise and infrastructure when possible. At least one core member of the research team must have demonstrated experience with methodological facets of AI/LLM development and evaluation. Additional personnel may be necessary to ensure there is appropriate expertise on the research team. Requests for supplement funding should provide evidence that existing project stakeholders support the proposed supplement. Proposed supplements must adhere to the PCORI Methodology Standards. A detailed report of all work completed as part of the supplement will be included in the original award’s DFRR.
Following submission of the supplement request, PCORI will complete a review of each request. You may be contacted by your assigned Program Officer (PO) for additional information as part of PCORI’s review.
Determinations will be made using the review criteria below:
- Is the proposed supplement responsive to the funding opportunity? Specifically, does the submission enhance or modify existing project aims to incorporate advancements in LLMs?
- Does the proposed supplement demonstrate usefulness and/or impact in alignment with the Methods PFA by demonstrating significant contributions to innovative methods that enhance a) the infrastructure that facilitates patient-centered CER to drive lasting improvements in health and/or b) transformation of both the research enterprise and care delivery?
- Does the proposed supplement have methodological rigor?
- Does the supplement demonstrate that there is appropriate expertise on the research team (proposed or current) to carry out the supplement?
- Does the engagement approach reflect appropriate patient and other stakeholder participation, fully incorporating any new aims and activities?
- Is the requested budget amount commensurate with the scope of the proposed supplement, including appropriate resources, timelines and awareness of dependencies to complete the outlined scope of work within 12 months?
- Is the currently funded award in good standing, and is there no anticipated negative impact of the supplement on success of the original award?
|December 12, 2023, 5 p.m. (ET)
|Funding Approval Notification
|Earliest Start Date
Funds and Project Period
To Apply for This Opportunity:
- Read the full opportunity above.
- Contact your assigned Program Officer (PO) to discuss whether your proposed supplement fits within the scope of this opportunity and/or send any questions to [email protected].
- Provide the information requested under Supplemental Funding Resources below. The Administrative Official (AO) for the current award should email the required, completed templates as separate PDF documents to [email protected], copying your assigned PO, Program Associate and Contract Administrator.
Methods Supplemental Funding Resources
*This section was updated on October 31, 2023.
- Methods Supplemental Funding Research Plan Template (required for all)
- Methods Supplemental Funding Budget Summary Template (required for all)
- Methods Supplemental Funding Budget Justification Template (required for all)
- Methods Supplemental Funding Key Personnel Template (required only for requests adding additional Key Personnel)
- Methods Supplemental Funding Letters of Support Table (required only for requests adding additional sites)