Pragmatic Clinical Studies to Evaluate Patient-Centered Outcomes -- Cycle 2 2017
Funding Opportunities
PCORI seeks to fund pragmatic clinical trials, large simple trials, or large-scale observational studies that compare two or more alternatives for addressing prevention, diagnosis, treatment, or management of a disease or symptom; improving healthcare-system-level approaches to managing care; communicating or disseminating research results to patients, caregivers, or clinicians; or eliminating health or healthcare disparities.
Proposed studies must address critical clinical choices that patients, their caregivers, clinicians, or delivery systems face. They must involve broadly representative patient populations and be large enough to provide precise estimates of hypothesized effectiveness differences and to support evaluation of potential differences in treatment effectiveness in patient subgroups.
For this solicitation, applicants are not required to demonstrate that patients and other stakeholders are already engaged as research team members at the time an application is submitted. However, applicants should outline how patients and other stakeholders will participate as partners in various phases of the proposed research, once awarded. Applicants should describe their plan to form a Study Advisory Committee (SAC),[A] or other appropriate engagement body, to ensure that a broad spectrum of patients and other stakeholders advise and assist the research team with refining the study questions, outcomes, and protocols. These patients and other stakeholders must include national or regional organizations that represent—at a minimum—patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders. PCORI may recommend additional representation in collaboration with the applicant, including individual patients with lived experience and other relevant stakeholders, such as scientific and methodological experts.
Note that this funding program does not support applications in conducting cost-effectiveness analyses, systematic reviews (with or without meta-analyses), or developing or conducting efficacy evaluation of shared decision making. PCORI will not cover costs for interventions that are being compared in the proposed study. (See Appendix 3: Administrative Actions in the Application Guidelines for details.)
This PFA is a collaborative effort of PCORI’s Clinical Effectiveness and Decision Science and Healthcare Delivery and Disparities Research programs.
Table of Contents
Key Dates
Funds and Project Period
$90 million
$10 million
5 years
I.Special Area of Emphasis Notice
Special Area of Emphasis Topics
For this cycle, the Patient-Centered Outcomes Research Institute (PCORI) is indicating a Special Area of Emphasis for the following topic:
Anxiety Disorders in Children, Adolescents, and/or Young Adults
Compare the effectiveness of one or more digital applications of cognitive behavioral therapy (CBT) to an appropriate active control (e.g., face-to-face CBT) for treatment of mild-to-moderate anxiety in children, adolescents, and young adults (through age 25). Digital applications may include computer-based therapy (e.g., delivered via the Internet or downloadable software) and CBT smartphone applications that are appropriate for treating anxiety. PCORI has a preference for digital interventions available in the public domain to support accessibility of effective treatments.
PCORI is particularly interested in:
- Comparisons of patient-important treatment outcomes across CBT interventions with differing levels and types of support (e.g., parent, therapist, peer, and school counselor involvement).
- Studies in pediatric and primary-care clinics, community mental health centers, schools, and community social service organizations.
- Broad patient populations (e.g., comorbidities and range of severity) and a spectrum of developmental stages represented by patients ages 5 through 25 years.
- Outcomes to include child and family functioning, symptom response, progression, and acceptability of and adherence to recommended care.
- About 2–3 years of follow-up from baseline/randomization, with a minimum of 12 months. Applicants should make clear the length of the active intervention and whether maintenance or booster sessions are included.
Applicants should justify any proposed interventions with efficacy data applicable to the population. Studies must capture primary and comorbid diagnoses (anxiety or otherwise), co-treatments, and measures of anxiety severity for potential heterogeneity of treatment response analyses.
PCORI has set aside up to $25 million to fund up to 3-4 applications in response to this topic. Studies are subject to the direct-cost limitations of the Pragmatic Clinical Studies PCORI Funding Announcement (PFA). Applicants for this special emphasis topic who are requesting budgets that exceed the specified limitations should provide strong and clear rationales justifying the needs for higher budget limitations in the Letters of Intent (LOIs) for PCORI’s review and approval.
II. Introduction
Summary of Program
PCORI is launching this funding initiative to expand its support of patient-centered comparative clinical effectiveness research (CER). The program goals are to (1) address important evidence gaps that are meaningful and central to the critical decision-making context, for which the lack of sufficient evidence is a barrier to clinical or healthcare system decision making; and (2) to gain insight into Heterogeneity of Treatment Effect (HTE) through the conduct of trials with sufficiently large sample sizes. PCORI seeks to fund large pragmatic clinical trials, large simple trials, or large-scale comparative observational studies that involve representative patient populations; have strong endorsement and study participation by relevant patient organizations, professional organizations, or payer or purchaser organizations; take place within typical clinical care and community settings; and have a sample large enough to enable precise estimates of effect sizes and to support evaluation of potential differences in treatment effectiveness in patient subgroups. Funded studies will compare the relative effectiveness[1] of two or more alternatives for improving patient-centered outcomes. We will consider proposed studies of comparative efficacy[2] as nonresponsive.
Background
Although traditional randomized controlled trials (RCTs) are widely accepted for assessing the efficacy of medical interventions, RCTs are expensive and time consuming. Furthermore, findings from these trials may have limited generalizability for evaluating the comparative clinical effectiveness of interventions already in use because of the following well-documented factors: (1) the comparisons in the trial often fail to reflect the choices patients and clinicians face; (2) the chosen study population tends to be homogeneous, highly motivated, and relatively free of many comorbid conditions; (3) research tends to take place in specialized research settings; (4) research protocols are often rigid and not representative of typical clinical practice; and (5) the trial may use a placebo, rather than an active comparator, as the comparison.
To meet these concerns, applicants should design trials so that they address practical comparative questions faced by patients and clinicians—to include broader and more diverse populations—and can be conducted in real-world clinical and diverse health-system settings. Such trials are often referred to as “pragmatic clinical trials” because they are intended to provide information that healthcare providers can adopt directly. They tend to be conducted in routine clinical care settings and, in many cases, they must be relatively large because expected differences in comparative clinical effectiveness might be small yet important. The large size of these trials will also permit the evaluation of effectiveness in different patient subgroups. In some cases, these trials may be much simpler than traditional RCTs, and such trials would be considered large simple trials.
The protocols for these trials are typically less complex and less intrusive to routine clinical practice than are efficacy studies. In considering pragmatic approaches to their study design, conduct, and analysis, applicants should refer to the multiple elements of the pragmatic-explanatory continuum and explicitly consider the tradeoffs of each element on the continuum. Absolute pragmatism is not the ideal, particularly for standardizing interventions chosen for comparison and the outcomes measures of interest, both of which define the assessment of specific research questions and the underlying causal inferences. Although interventions require some degree of flexibility in their use, they must be defined sufficiently to be replicable in their dissemination and implementation in U.S. health care. For more extensive discussion on pragmatic versus traditional explanatory trials, see Patsopoulos,[3] Thorpe et al.,[4] and Loudon et al.[5]
For those trials targeting populations at risk for experiencing disparities (e.g., racial or ethnic minorities and low-income groups), it might be necessary to tailor interventions that take place in real-world settings to address the population's specific needs. These trials may require complex, multicomponent, multi-level interventions (e.g., targeting the patient, provider, and system), as evidenced by the literature on disparities. It might be necessary to gather more than a minimal level of outcome data to assess the impact of the intervention adequately.
Examples of Successful Pragmatic Clinical Trials
- Choudhry and colleagues[6] enrolled 5,855 patients to test whether the elimination of out-of-pocket expenses for medications prescribed after a myocardial infarction would increase the percentage of patients adhering to medication regimens and improve clinical outcomes.
- In the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT), 33,357 participants 55 years or older with hypertension—and at least one other coronary heart disease risk factor—drawn from 623 North American centers, were randomized to chlorthalidone, amlodipine, or lisinopril.[7]
- A randomized, real-world, open-label comparative clinical effectiveness trial enrolled patients diagnosed as depressed by primary-care practitioners. Patients were randomly assigned to a serotonin reuptake inhibitor or one of two tricyclic antidepressants and followed (passively) for two years to evaluate depression symptoms, health-related quality of life, healthcare utilization patterns, and costs.[8]
Features of Patient-Centered Outcomes Research
PCORI funds patient-centered outcomes research (PCOR), which helps patients and their caregivers communicate and make informed healthcare decisions, allowing their voices to be heard in assessing the value of healthcare options. This research:
- Assesses the benefits and harms of preventive, diagnostic, therapeutic, palliative, and health information communication or dissemination strategies, or health-delivery-system features to inform decision making, highlighting the choices that matter to people
- Is inclusive of an individual’s preferences, autonomy, and needs, focusing on outcomes that people notice and care about (including survival, functioning, symptoms, and health-related quality of life)[9]
- Incorporates a wide variety of settings and diversity of participants to address individual differences and barriers to implementation and dissemination
- Directly compares clinical and delivery-system interventions that are currently available or used in the settings in which people access health care
- Obtains stakeholder perspectives to address the burdens to individuals, care access, care quality, and technology and personnel requirements
Research Characteristics and Objectives
PCORI seeks to support new research that addresses critical clinical and health-related questions faced by patients, their caregivers, and their providers. PCORI seeks to fund investigator-initiated research displaying the following characteristics:
- The research studies the benefits and harms of different interventions and strategies that are currently delivered in typical clinical and community settings.
- The research compares at least two alternative clinical approaches. Because PCORI’s mission is to develop evidence to inform difficult decisions, we strongly prefer applications that propose to compare well-defined interventions that are already being used in the condition and the population of interest.
- The research examines such interventions as specific drugs, devices, procedures, assistive technologies, behavioral change, communication or dissemination, or complementary treatments. Studies may also address complex interventions occurring at, or pertaining to, care delivery systems. Please note that “usual care” is not an appropriate comparator for CER studies submitted to PCORI for funding consideration. “Usual care” is too often ill defined; difficult to quantify; and subject to considerable geographic and temporal variations, limiting interpretability, applicability, and reproducibility. If the applicant proposes “usual care” as a rational and important comparator in the proposed study, then it must be described in detail, coherent as a clinical alternative, and justified properly as a legitimate comparator (e.g., “usual care” is guidelines-based). The applicant must also include an explanation of how the care given in the “usual care” group will be measured in each patient, to the extent possible, and how appropriate inferences will be drawn from its inclusion.
- The research compares health outcomes that are meaningful to the study’s patient population (e.g., morbidity, mortality, symptoms, functional status, quality of life, and absenteeism from work or school). Such outcomes should be measured using validated methods. In select instances, surrogate physiological measurements may be sufficiently linked to final health outcomes to be of interest, but they might not be the sole study outcomes.
This solicitation has two objectives. First and most important, PCORI seeks to commit adequate funding to address critical clinical and health-related questions faced by patients, their caregivers, and their clinicians. Second, when randomized studies are proposed, PCORI is interested in testing novel and efficient methodological approaches, such as Bayesian adaptive designs. PCORI has particular interest in funding studies that:
- Use validated measures of patient-reported outcomes (PROs).
- Can be completed quickly because the primary outcomes focus on symptoms or other patient-reported measures.
- Examine interventions and outcomes that cut across specific diagnoses (e.g., studies with primary outcomes focused on symptoms, such as pain).
- Employ strategies to enhance study efficiency, such as Bayesian adaptive designs in which trial characteristics (e.g., sample size, randomization proportions, treatment arms, and eligibility criteria) evolve during the trial in response to interim trial data. (See PCORI’s Standards for the Design, Conduct, and Evaluation of Adaptive Randomized Clinical Trials.)
Such studies will help determine not only how to employ these approaches within real-world settings, but also how to integrate such approaches within a dynamic and rapidly learning environment.
Topic Selection
PCORI’s multi-stakeholder advisory panels have identified research topics and research questions (see Appendix: Research Topics of Interest to PCORI). During PCORI’s award selection process, PCORI Board of Governors (Board) members on the Selection Committee, merit reviewers, and program staff pay attention to applications addressing PCORI-identified topics and research questions. Although other prioritized lists of CER questions are also of interest (e.g., the Institute of Medicine [IOM] Priorities for CER and the Agency for Healthcare Research and Quality [AHRQ] Future Research Needs Projects), PCORI will give first consideration to applications that directly address one or more of the PCORI-identified topics (see Appendix: Research Topics of Interest to PCORI). For this Pragmatic Clinical Studies funding cycle, PCORI is indicating a Special Area of Emphasis for one topic from the topic list: anxiety in children. PCORI has set aside up to $25 million to fund up to 3–4 applications in response to this special emphasis topic. The full amount of this set-aside may be reduced but not eliminated by application review and award decisions that are pending.
PCORI is open to receiving and reviewing LOIs for studies on investigator-initiated CER questions. In such cases, applicants must explain why PCORI should consider the proposed research question to be important. Regardless of the research questions, applicants are expected to adhere to PCORI Methodology Standard RQ1, which states that “gap analysis and systematic reviews should be used to support the need for a proposed study.”
Since August 2015, PCORI has made funding decisions on applications submitted for four Pragmatic Studies Funding Cycles. PCORI may entertain additional studies within a given research topic if the proposed study complements the funded (or to-be-funded) studies. Applicants should therefore be aware that the application topic could be a factor in our decision to invite a full application. (Note: PCORI does not provide information about pending awards.)
PCORI expects that project budgets and duration will vary substantially depending on the topic and approach selected, the recruitment or primary data-collection needs, the length of follow-up, and the analytic complexity. PCORI seeks efficient studies, such as those taking advantage of large populations already under observation, as well as the supportive involvement of delivery systems or health plans to enhance recruitment and data collection. A prolonged recruitment period is not an acceptable rationale for longer studies, except in the case of a rare disease. Funding requests to develop or build on initial collaboration between researchers and patient/stakeholder groups are also not appropriate.
In-kind contributions to a proposed study are welcome, as are opportunities for co-funding between PCORI and another research sponsor. Each of these factors is taken as further evidence of the research question’s importance.
III. Guidance for Preparing Applications
Specific Requirements
The proposed study should strive to meet the following requirements:
- Focus on a comparative clinical effectiveness question that is important to patients and other decision makers.
- Address an evidence gap in deciding among available options; this gap should have been substantiated by an existing (recent or updated), rigorously conducted systematic review or emphasized by an official professional society’s clinical practice guideline.
- Demonstrate consultation with patients and other stakeholders or their representative groups, or reference previously documented decisional dilemmas to determine if the study is answering a critical question—one that, if adequately answered, would substantially improve decision making.
- Propose a sample size that is sufficiently large to allow for precise estimation of hypothesized effect sizes or for clear demonstration of non-inferiority. The sample size must also support testing of a priori hypotheses related to potential differences in effectiveness among relevant patient subgroups (HTE).
- Examine diverse populations receiving care in real-world settings.
- For studies aiming to reduce or eliminate health or healthcare disparities, specify one or more of the Addressing Disparities Program target populations (i.e., racial or ethnic minorities; low-income groups; residents of rural areas; individuals with special healthcare needs [including individuals with disabilities]; individuals with low health literacy or numeracy or limited English proficiency; and lesbian, gay, bisexual, and transgender [LGBTQ] persons) that will be the focus of the study. Studies should test the ability of interventions to improve outcomes (including patient-centered, clinical, and structural outcomes) and reduce disparities for at-risk populations.
- Have strong interest from and support of host delivery systems and clinical care settings.
- Specify broad and simple eligibility criteria that will allow for wide generalization of results while attending appropriately to ethical concerns of excess risk in some patient subgroups.
- Compare interventions that are known to be efficacious, effective, or commonly used and that can be implemented in real-world settings.
- Feature near-term outcomes and PROs as primary outcomes, when appropriate.
- Plan to collect patient-centered outcome data efficiently and periodically during follow-up.
- Provide preliminary evidence of the potential for efficient recruitment, high participation rates, and appropriate oversight by local or centralized Institutional Review Boards (IRBs), including plans for streamlining or waiving individual informed consent in cases of low-risk interventions (if applicable). PCORI believes that the intensity of oversight and the complexity of informed consent procedures should be closely related to the degree of risk from study participation. Applicants must address this issue and present evidence that the study will not encounter significant recruitment or participation barriers. The relevant IRBs make the final determination of the adequacy of informed-consent procedures and participant protections.
- Adhere to all applicable PCORI Methodology Standards. The full application will require the applicant to identify the standards appropriate to the proposed study and to describe how the study team plans to address each standard.
- In the case of RCTs, also adhere to current best practices (standardized inclusion or exclusion criteria; proper randomization; techniques to minimize potential for missing data; and appropriate safety monitoring, including establishing a Data and Safety Monitoring Board [DSMB] or indicating why such a board is unnecessary).
To carry out studies that allow for adoption of the findings in a real-world setting, and to maximize the efficient use of resources, take care to prevent these trials from becoming more complex and onerous than necessary. We encourage the applicant to be creative and consider the following innovative strategies, as appropriate and feasible:
- Consult with patients and other stakeholders on their decisional dilemma and evidence needs, or reference previously documented decisional dilemmas in preparation for submitting LOIs and full applications.
- Carefully describe the pertinent evidence gaps and why the project questions represent decisional dilemmas for patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders. Similarly, applicants should document why project outcomes are especially relevant and meaningful endpoints for patients and their families.
- Minimize disruption to participants’ daily routines (e.g., minimize participant visits intended for study-assessment purposes and capture PROs during office visits, electronically, or via phone).
- Design the study so that you can conduct it using routine clinic or office operations.
- Use efficient methods to obtain participant consent while still meeting ethical and legal requirements.
- Capitalize on the existing electronic health records (EHRs) and other computerized information to identify and recruit eligible patients, monitor study conduct and patient safety, and collect study outcomes information. PCORI specifically encourages applications that use the National Patient-Centered Clinical Research Network (PCORnet) infrastructure.
- If data standardization and interoperability across study sites have not already been accomplished, develop methods that will enhance the standardization of data that are accessed from different EHR systems.
Nonresponsiveness
Applications will be considered nonresponsive to this PFA if the proposed research:
- Tests efficacy (or comparative efficacy) of interventions that are novel or with limited evidence of efficacy.
- Involves studies conducted within tightly controlled research environments instead of in clinical settings reflective of real-world healthcare delivery.
- Conducts a formal cost-effectiveness analysis.
- Directly compares the costs of care between two or more alternative approaches to providing care.
- Conducts studies of the natural history of disease, instrument development, pharmacodynamics, and fundamental science or biological mechanisms.
- Evaluates validity or efficacy of (rather than the comparative clinical effectiveness of) new or existing decision-support tools. This includes the development and efficacy evaluation of decision-support or shared-decision tools or systems for patients, clinicians, or both.
- Develops clinical prediction or prognostication tools.
- Establishes efficacy for a new clinical strategy.
- Pilots studies intended to inform larger efforts.
- Compares patient characteristics rather than clinical strategy options.
- Compares interventions for which the primary focus or the sole intervention is examining the role of compensated or volunteer community health workers, including patient navigators.
Proposals may report use of any or all health services, but may not employ direct measurements of care costs. For further information, please reference our cost-effectiveness analysis FAQs.
PCORI does have an interest, however, in studies addressing questions about conditions leading to high costs to the individual or to society. This is included in our review criterion on the potential for research to fill a critical gap in knowledge or practice. As a result, PCORI is interested in studies that:
- Examine the effect of costs on patients, such as patients’ out-of-pocket costs, hardship or lost opportunity, or costs as a determinant of or barrier to access to care.
- Address cost-related issues, such as the resources needed to replicate or disseminate a successful intervention.
- Evaluate interventions to reduce health-system waste or increase health-system efficiency.
Addressing this issue specifically, our PFAs say that “applications that include studies of these issues without using a formal cost-effectiveness analysis or directly measuring and comparing costs of care alternatives will be considered responsive and will be reviewed.”
Leveraging Existing Resources
PCORI encourages investigators to propose studies that leverage existing resources, such as adding PCOR to an existing large clinical trial or analyzing existing large databases that contain valuable and relevant information that can be used to answer important CER questions. PCORI is interested in studies that leverage existing research networks or consortia that would facilitate the conduct of large, multi-site studies called for in this PFA.
Applicants proposing use of an existing research network infrastructure (e.g., PCORnet); research consortia; or related data resources (e.g., electronic medical record [EMR] data from healthcare delivery systems or administrative claims data from public or commercial insurers) should address the following in the Research Plan (as appropriate), with sufficient specificity:
- Identify and justify all participating research network entities (e.g., health plans, consortia projects and members, etc.) or, in the case of PCORnet, identify the names of the participating Clinical Data Research Networks (CDRNs), Patient-Powered Research Networks (PPRNs), and PCORnet Collaborative Research Groups) that will be collaborating on the project. Also identify the affiliated study performance sites.
- Demonstrate that the data source can comprehensively capture the study variables needed to assess the interventions, covariates, and outcomes.
- Describe how you will manage data across study sites within the research network or the proposed research consortia, and whether you will use any dedicated data-coordinating functions or facilities.
- As applicable, demonstrate familiarity with the existing network governance policies or data-use restrictions. Provide a study management structure that identifies roles, responsibilities, and decision-making authority across the proposed research consortia.
- As applicable, provide a timeline for establishing data-use agreements.
- As applicable, describe the network infrastructure resource(s) used to conduct the study (i.e., core research-support facilities, streamlined IRBs, contracting, engagement and consenting processes, standardized data resources training, etc.). Indicate the percentage of sites that have previously used centralized versus localized IRBs.
- As applicable, provide documentation supporting the involvement of network leadership throughout the study (e.g., detailed Letters of Support, budgets, and Budget Justifications that cover the costs of the network’s efforts). You can obtain Letters of Support from PCORnet from the Coordinating Center by submitting a request via the PCORnet Front Door.
Preliminary Data and Use of Accepted Measures
PCORI encourages investigators to design their research using valid patient-centered outcome measures. Include preliminary data that support using the proposed measures in the study population. We also encourage investigators to consider those measures described in the Patient-Reported Outcomes Measurement Information System (PROMIS).
Studies in Rare Diseases
PCORI is interested in investigating strategies addressing care for patients with rare diseases. These conditions are defined as “life-threatening” or “chronically debilitating.” They are of such low prevalence (affecting fewer than 200,000 in the United States [i.e., less than 1 in 1,500 persons]) that special efforts—such as combining data across large populations—might be needed to address them.
Methodological Considerations
Regardless of study design, applications must adhere to all relevant PCORI Methodology Standards. These include 48 individual standards that fall into 12 categories. The first five categories are cross-cutting and relevant to most PCOR studies. Researchers should refer to all of these standards when planning and conducting their research projects. These cross-cutting categories are:
- Standards for Formulating Research Questions
- Standards Associated with Patient-Centeredness
- Standards on Data Integrity and Rigorous Analyses
- Standards for Preventing and Handling Missing Data
- Standards for Heterogeneity of Treatment Effect (HTE)
In addition to these five sets of standards, the first standard of “Standards for Causal Inference Methods” -(CI-1)- is cross-cutting and applicable to all PCOR studies.
The seven other standards categories will be applicable to particular study designs and methods. Applicants should use the standards in each of these categories as guidance when they are relevant to a study. These categories are:
- Standards for Data Registries
- Standards for Data Networks as Research-Facilitating Infrastructures
- Standards for Causal Inference Methods
- Standards for Adaptive and Bayesian Trial Designs
- Standards for Studies of Medical Tests
- Standards for Systematic Reviews
- Standards for Research Designs Using Clusters
Most of these standards are minimal. The PCORI Methodology Standards reflect practices that applicants should follow in all cases, and all deviations need to be explained and justified. Applicants should address additional best practices—including relevant guidelines for conducting clinical trials developed by other organizations—in the application for PCORI funding. To help reviewers quickly identify adherence to a particular standard, applicants must cite each relevant PCORI Methodology Standard within the Methodology Standards Checklist, following the instruction in the checklist itself and in the Application Guidelines. Program staff use the checklist to evaluate applications.
Applicants should specifically discuss their capacity to measure such factors as differential adherence to chosen treatments (or participation in intervention programs) that could create or explain apparent differences in the effectiveness of the alternative interventions being compared in clinical populations.
Clinical Trial Design Guidance and Consultation
PCORI realizes that some applicants might not have extensive experience conducting large, real-world, comparative, pragmatic, and patient-centered trial designs, or in nontraditional designs (such as adaptive designs). Applicants selected for funding may receive PCORI financial support for external expert statistical and trial design consultation done in collaboration with the applicants. This consultation may include experience and expertise in techniques such as trial design simulation and adaptive designs to enhance the scientific rigor and efficiency of large, PCORI-funded pragmatic trials.
Patient and Stakeholder Engagement
PCORI encourages all applicants to outline how patients and other stakeholders will participate as partners in various phases of the proposed research. Before completing this section of the Research Strategy, we encourage to review PCORI’s Engagement Rubric, which can be found in the PCORI Funding Opportunities. Applicants should also review the PCORI Methodology Standards Associated with Patient-Centeredness, and PCORI’s Sample Engagement Plans. The rubric and Sample Engagement Plans are not intended to be comprehensive or prescriptive; instead, they provide a variety of examples to incorporate engagement, where relevant, into the research process.
PCORI expects applicants to consult with patients and other stakeholders on their decisional dilemma and evidence needs, or to reference previously documented decisional dilemmas in preparation for the submission of LOIs and applications. To describe the decisional dilemma, state the specific clinical decision(s) or treatment choice(s) the decision makers confront and explain how the findings from the proposed research will inform those decisions. State why this decision—such as choosing a specific medication, surgical approach, or care delivery strategy to treat a condition or manage a specific population—is important to patients. Document the uncertainty patients and other stakeholders face when making this decision. Identify the patients and other stakeholders you consulted when determining that the proposed study addresses their evidentiary needs for decision making, and indicate your commitment to continue engaging them actively in the study. Similarly, applicants should document how the project outcomes are especially relevant and meaningful endpoints for patients and other stakeholders.
For this PFA, applicants are not required to demonstrate that patients and other stakeholders are already engaged as research team members at the time an application is submitted. However, the Engagement Plan should outline how patients and other stakeholders will participate as partners in various phases of the proposed research, once awarded. Applicants should describe their plan to form a Study Advisory Committee (SAC), or other appropriate engagement body, to ensure that a broad spectrum of patients and other stakeholders advise and assist the research team with refining the study questions, outcomes, and protocols. These patients and other stakeholders must include national or regional organizations that represent—at a minimum—patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders. Additional representation may be recommended in collaboration with PCORI, including individual patients with lived experience and other relevant stakeholders, such as scientific and methodological experts. The SAC or other appropriate engagement body should meet in person at least two times per year, and the budget should account for these engagement costs.
PCORI understands that engagement structures and approaches vary widely. Other engagement approaches, such as forming stakeholder groups, panels, task forces, working groups and other bodies, or involving individual patient and other stakeholder partners in various ways, are also permissible to employ—either in addition to or instead of—the formation of the SAC. For clarification in your application materials and for merit review purposes, please indicate which body or structure is filling the SAC requirements, including the requirements for in-person meetings at least two times per year and appropriate budgeting.
Populations Studied
PCORI seeks to fund research that includes diverse populations with respect to age, gender, race, ethnicity, geography, and clinical status. PCORI recognizes that some proposed studies might represent important PCOR opportunities, even in the absence of a broadly diverse study population. However, the burden is on the applicant in such cases to justify the study’s importance in the absence of diversity.
Alternatively, PCORI is particularly interested in including previously understudied populations for whom effectiveness information is needed, such as hard-to-reach populations or patients with multiple conditions. Thus, comparisons should examine the impact of strategies in various subpopulations, with attention to the possibility that the strategy’s effects might differ across subgroups. Populations of interest include those that are less frequently studied. PCORI has developed the following list of populations of interest to guide our research and engagement efforts:
- Racial and ethnic minority groups
- Low-income groups
- Women
- Children (age 0–17 years)
- Older adults (age 65 years and older)
- Residents of rural areas
- Individuals with special healthcare needs, including individuals with disabilities
- Individuals with multiple chronic diseases
- Individuals with rare diseases
- Individuals whose genetic makeup affects their medical outcomes
- Individuals with low health literacy, numeracy, or limited English proficiency
- LGBTQ persons
- Veterans and members of the Armed Forces and their families
Project Budget and Duration
Applicants may request up to $10 million in total direct costs for a research project period not to exceed five years (not including peer review). At the time of contract execution, PCORI sets aside all of the funds associated with an awarded project to be made available throughout the contract’s period of performance. The maximum budget includes all research and peer-review-related costs. (Please refer to the Application Guidelines for further details.) In general, PCORI will not cover costs for interventions that are being compared in the proposed study. (Appendix 2: Allowable and Unallowable Costs in the Application Guidelines for details.) Applicants should submit a realistic budget and timeline reflecting the proposed study’s scope and requirements. In some rare circumstances, the estimated budget may exceed $10 million total direct costs, depending on the nature of the research question, the design and analytical requirements of the proposed study, the expected size of the patient enrollment, or the complexity and frequency of the outcomes assessment. PCORI expects these to be selective cases, which include topics that are of greatest interest to us. Applicants who intend to propose such studies must provide evidence of prior approval by PCORI scientific staff to exceed the budgetary limit and succinct justifications in their LOI, documenting the budget requirements with respect to the scope of the proposed research and the data-collection and analysis efforts. Please note that this justification counts toward the LOI three-page limit. PCORI will deny any request for a project period longer than five years. Note that although subcontractor indirect costs are included in the prime applicant’s direct-cost budget, subcontractor indirect costs are not factored when determining adherence to the PFA’s direct-cost limit.
A contract is the funding mechanism for this program. A milestones and deliverables schedule, as well as specified recruitment targets, should be linked directly to and included in the proposed budget that will be subject to negotiation at the time of award. Some of the other activities that will be considered during negotiations include:
- Developing a study protocol and procedure manual for the intervention
- Assigning roles and responsibilities to study team members for project implementation
- Forming a SAC or other appropriate engagement body
- Providing a detailed task-based budget with level of effort for project staff, specified by task
- Obtaining clearances from all institutional and community partners, including IRB approvals
- Establishing a DSMB or providing a clear description of why one is unnecessary
- Executing all subcontractor agreements
- Agreeing on eligible patient populations for study recruitment
- Identifying barriers to patient recruitment in the study and addressing these barriers effectively
- Structuring a feasibility phase to demonstrate the potential for successful recruitment
Total project funding is contingent upon successful programmatic and budget performance (e.g., meeting recruitment targets). Awardees must provide corroborating evidence to receive continuous funding support. Specifically, after 12 months of study performance, but no later than 18 months, PCORI will use information from the awardee to conduct a formal programmatic assessment of the study's progress and specified recruitment targets to determine the study’s viability and sustainability. Only studies that are deemed satisfactory in this assessment will receive continuous funding support.
Refer to the Application Guidelines for a list of additional project milestones specific to this PFA.
Collaboration
PCORI is particularly interested in applications involving community and commercial organizations that can help researchers design, implement, disseminate, and sustain effective interventions. We encourage applications that include novel collaborations with accreditation organizations, credentialing bodies, educational enterprises, patient advocacy groups, industry, professional societies, and subspecialty societies.
Protection of Human Subjects
This component (up to five pages) is included in the Research Plan Template. Describe the protection of human subjects involved in your proposed research. PCORI follows the Federal Policy for the Protection of Human Subjects (45 CFR part 46), including the Common Rule. For more detailed information, please see Section 5, titled “Human Subjects Research Policy” in the Supplemental Grant Application Instructions for All Competing Applications and Progress Reports, which is issued by the U.S. Department of Health and Human Services. PCORI does not require that applicants comply with sections of this policy referring to requirements for federal-wide assurance or that refer to standards for including women, minorities, and children. Awardees must also comply with appropriate state, local, and institutional regulations and guidelines pertaining to the use of human subjects in research.
PCORI requires awardees to ensure that there is a Data and Safety Monitoring Plan, which may include the need to appoint a DSMB, as provided in the PCORI Policy on Data and Safety Monitoring Plans for PCORI-Funded Research.
PCORI merit reviewers will examine plans for protection of human subjects in all applications and may provide comments regarding the plans (see How To Evaluate Human Subjects Protections). Reviewers’ comments on human subject research are not reflected in the overall application score, but PCORI staff might use them during potential funding negotiations. Final determinations about the adequacy of human subject protections rest with the IRB or international equivalent that has jurisdiction for the study.
The Awardee Institution, whether domestic or foreign, bears ultimate responsibility for safeguarding the rights and welfare of human subjects in PCORI-supported activities.
Required Education of Key Personnel on the Protection of Human Research Participants
PCORI requires that all applicants adhere to the National Institutes of Health (NIH) policy on education in the protection of human research participants in the conduct of research. This applies to all individuals listed as key personnel in the application. The policy and FAQs are available on the NIH website.
Data Management and Data-Sharing Plan
PCORI encourages openness in research and making research data available for purposes of replication and reproducibility. Although not required to be submitted as a component of the research application, if an award is made, the awardee must develop and maintain a plan addressing data management and data sharing of research project data. This must be done in a manner that is appropriate for the research project and the types of research project data, and in a manner consistent with applicable privacy, confidentiality, and other legal requirements.
Recruitment
Applications should include information about the size and representativeness of the potential recruitment pool of patients and the means by which this size estimate was determined (e.g., EMRs, claims records, clinic logs, or other administrative systems). Likewise, applications should provide evidence-based estimates of how many participants are ultimately expected in the study, based on expected recruitment applying the study’s inclusion and exclusion criteria; anticipated acceptance (or refusal) rates; and other factors, such as loss to follow-up. Such estimates must be discussed in the application, specified in the milestones, reviewed by Merit Review Officers (MROs) and PCORI staff, and monitored by PCORI in the funded research.
Peer Review and Release of Research Findings
PCORI has a legislative mandate to ensure the scientific integrity of the primary research it supports and to make study findings widely available and useful to patients, clinicians, and the general public within a specific time frame. Accordingly, the Board adopted the Process for Peer Review of Primary Research and Public Release of Research Findings.
In summary, Awardee Institutions must submit to PCORI for peer review a draft final research report that provides the methodological details, describes the main study results, and interprets the findings in clinical or other decisional contexts. Subject matter experts (SMEs); individuals with expertise in research methodology or biostatistics; and patients, caregivers, and other healthcare stakeholders will review the draft final research report. After Awardee Institutions have responded to reviewers’ comments to PCORI’s satisfaction, the report will be accepted and considered final. PCORI will then prepare a 500-word abstract summarizing the study results for patients and the general public, which the Awardee Institution will review and approve.
PCORI will post the following materials on its website no later than 90 days after the draft final research report is accepted: (1) a 500-word abstract for medical professionals; (2) a standardized summary of the study results for patients and the general public; (3) a link to the study record on ClinicalTrials.gov (as applicable); and (4) ancillary information, including conflict-of-interest disclosures. The final research report, along with anonymized reviewer comments, will be made publicly available on the PCORI website no later than 12 months after its acceptance, except by prior mutual agreement with the Awardee Institution.
IV. How To Submit an Application
Letter of Intent (LOI)
Applicants should download the Cycle 2 2017 Pragmatic Clinical Studies LOI Template from the PCORI Funding Opportunities. They must complete the document and convert it to a PDF with a three-page limit, excluding references. PCORI suggests including all references as in-text citations using American Medical Association citation style, but we do accept other citation styles. Do not upload additional documents as part of your LOI, such as Letters of Endorsement or Support, because they are not requested at this stage. Their inclusion will result in LOI rejection without review. Please visit the PCORI Funding Opportunities for additional applicant resources, including FAQs and required templates.
Please answer all of the questions in the LOI Template. This includes the question on brief justification for the proposed cost of the study. Providing the answer, “costs not to exceed $10 million” is not sufficient. Upload your document to PCORI Online. The deadline for LOI submission is July 25, 2017, by 5 p.m. (ET).
LOI Review
PCORI evaluates LOIs based on the following criteria:
- Whether the topics are related to those on PCORI’s own list (see Appendix: Research Topics of Interest to PCORI), versus the IOM/AHRQ lists, versus other topics initiated by investigators
- Importance to current clinical decision making, as evidenced by critical gaps identified by clinical guidelines developers or recent relevant systematic reviews
- A size or scope sufficient to have a significant impact on patient outcomes or healthcare practices
- Clarity and credibility of applicants’ responses to the LOI questions, as well as their justification of the need for a large pragmatic study—including the rationale for the estimated sample size—citing published estimates, including effect sizes, standard deviations, and the need for rigorous comparative analysis of important subgroups
- Prior relevant experience
- Programmatic fit and balance, considering whether the application significantly overlaps with concurrent applications or previously funded studies or, conversely, whether the application fills a gap in PCORI’s portfolio, considering such characteristics as disease category, topics, population, and methodologies
- Adherence to the administrative and formatting requirements listed in the Application Guidelines, especially the three-page limit for the LOI
LOIs are reviewed qualitatively; they are not scored. Only applicants whose LOIs are deemed most responsive to this PFA will be invited to submit a full application. At least two PCORI staff review LOIs, and they are not scored during review. Notification of denial or approval to submit a full application will occur no later than August 22, 2017. Please refer to the Application Guidelines in the PCORI Funding Opportunities for due dates and information on how to submit your LOI in PCORI Online.
All applicants, including those resubmitting from previous Pragmatic Studies PFA cycles, are required to submit an LOI for PCORI staff to review. This allows PCORI to determine whether proposed revisions and changes made to specific aims or methodological approaches from the original applications align with our evolving strategic priorities.
If you are invited to submit an application, do not make significant changes to your proposed project without consulting a program officer. For example, you should not revise your major aims and study design. Any significant changes are grounds for removal from the review process.
Note: A Principal Investigator (PI) can only submit one LOI per PFA. However, an individual listed as a PI on one LOI can be listed as and serve in another non-PI role (e.g., co-investigator or consultant) on other LOIs within the same PFA during the same cycle. A PI may submit multiple LOIs to different program PFAs in a cycle, but the PI must ensure that the research topics and projects are not similar. If a PI submits an LOI to multiple program PFAs, LOIs that exhibit scientific overlap or that appear to be duplicate submissions will be disqualified. PCORI will contact the PI and provide him or her with an opportunity to choose the PFA to which he or she would like to apply. This applies to single and dual-PI submissions.
Submission Dates
LOIs and applications must be submitted in accordance with the published dates and times listed in the Overview section of this document and in the PCORI Funding Opportunities.
PCORI Online System
To submit an application, you must register in PCORI Online and submit an LOI and an application for each cycle to which you are applying.
V. Applicant Resources
- Pragmatic Clinical Studies Application Guidelines (Cycle 2 2017)
VI. Merit Review
PCORI’s merit review process is designed to support the following goals:
- Identify applications that have the strongest potential to help patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders make informed decisions to improve patient outcomes.
- Implement a transparent, fair, objective, and consistent process to identify these applications.
- Elicit high-quality feedback that reflects a diversity of perspectives to ensure that the PCORI-funded research reflects the interests and views of patients and other stakeholders and those who care for them, and that it meets the criteria for scientific rigor.
- Fund projects that fill important evidence gaps and have strong implementation potential.
- Regularly evaluate and continually improve the merit review process and policies in support of PCORI's mission.
PCORI merit review is a multiphase process that includes PFA development; staff evaluation of LOIs; the review panel’s preliminary review of full applications; an in-person panel discussion of a subset of full applications (identified by PCORI’s Research Priority Area Program staff and based on the preliminary review and program priorities); the Selection Committee’s recommendation of applications for funding; and, finally, Board award approval.
Preliminary Review
PCORI conducts rigorous merit review of the full applications it receives. Note that PCORI may eliminate applications from the review process for administrative or scientific reasons (e.g., non-responsiveness). An application may be administratively withdrawn if it is incomplete; submitted past the stated due date and time; or does not meet the formatting criteria outlined in the Application Guidelines, in the PCORI templates, and in PCORI Online. An application can be scientifically withdrawn if it is not responsive to the guidelines described in this PFA, describes research that is not comparative, includes a cost-effectiveness analysis, or otherwise does not meet PCORI programmatic requirements.
PCORI MROs recruit each panel based on the number of and topic areas represented by invited LOIs. MROs recruit the panel chair, scientist reviewers who are SMEs, patient representatives, and representatives of other stakeholder groups. All panel members receive training during the review cycle to ensure that they understand the programmatic and organizational goals of review.
We designed the table below to help applicants understand how the PCORI merit review criteria align with criteria from other funding organizations with which applicants might be familiar (e.g., NIH). Though PCORI’s criteria do map to most NIH criteria, there are areas where we ask for different information (i.e., PCORI does not include a criterion that tracks to NIH’s innovation criterion, but does include criteria evaluating patient-centeredness and engagement) reflecting PCORI’s unique approach.
Crosswalk of PCORI Merit Review Criteria with NIH Criteria | |
---|---|
SIGNIFICANCE |
|
APPROACH |
|
PCORI-Only Merit Review Criteria | |
PATIENT-CENTEREDNESS/ENGAGEMENT |
|
Application Review Criteria
Criterion 1. Potential for the study to fill critical gaps in evidence
The application should address the following questions:
- Does the application convincingly describe the clinical burden?
- Does the application identify a critical gap in current knowledge as noted in systematic reviews, guideline development efforts, or previous research prioritizations?
- Does the application identify a critical gap in current knowledge, evidenced by inconsistency in clinical practice and decision making?
- Would research findings from the study have the potential to fill these evidence gaps?
Criterion 2. Potential for the study findings to be adopted into clinical practice and improve delivery of care
The application should describe how evidence generated from this study could be adopted into clinical practice and delivery of care by others. The application should address the following questions:
- Does the application identify who will make the decision (i.e., the decision maker) or use (i.e., the end-user) the study findings (not the intervention) this study produces, such as local and national stakeholders?
- Does the application identify potential end-users of study findings—such as local and national stakeholders—and describe strategies to engage these end-users?
- Does the application provide information that supports a demand for this kind of a study from end-users?
- Would this study’s research findings have the potential to inform decision making for key stakeholders? If so, provide an example. How likely is it that positive findings could be reproduced by others, resulting in improvements in practice and patient outcomes? Identify the potential barriers that could hinder adoption of the intervention by others.
- Does the application describe a plan for how study findings will be disseminated beyond publication in peer-review journals and national conferences?
Criterion 3. Scientific merit (Research design, analysis, and outcomes)
The application should show sufficient technical merit in the research design to ensure that the study goals will be met. The application should also address the following questions:
- Does the application describe a clear conceptual framework anchored in background literature which informs the design, key variables, and relationship between interventions and outcomes being tested?
- Does the Research Plan describe rigorous methods that demonstrate adherence to the PCORI Methodology Standards?
- Is the overall study design justified?
- Are the patient population and study setting appropriate for the proposed research question?
- Does the application provide justification that the outcome measures are validated and appropriate for the population?
- Are each of the comparators (e.g., active intervention arm and comparator arm) described clearly and well-justified? If “usual care” is one of the arms, is it adequately justified and will it be sufficiently measured?
- Are the sample sizes and power estimates appropriate? Is the study design (e.g., cluster randomized design, randomized controlled trial, or observational study) accounted for and is the anticipated effect size adequately justified?
- Is the study plan feasible? Is the project timeline realistic, including specific scientific and engagement milestones? Is the strategy for recruiting participants feasible? Are assumptions about participant attrition realistic, and are plans to address patient or site attrition adequate?
Criterion 4. Investigator(s) and environment
This criterion should assess the appropriateness (e.g., qualifications and experience) of the investigator(s)/team and the environment’s capacity (e.g., resources, facilities, and equipment) to support the proposed project. It should not be an assessment of the institution’s quality.
The application should also address the following questions:
- How well-qualified are the PIs, collaborators, and other researchers to conduct the proposed activities? Is there evidence of sufficient clinical or statistical expertise (if applicable)?
- Does the investigator or co-investigator have demonstrated experience conducting projects of a similar size, scope, and complexity?
- If the project is collaborative or dual-PI, do the investigators have complementary and integrated expertise? Are the leadership, governance, and organizational structures appropriate for the project?
- (Dual-PI Option Only) Does the Leadership Plan adequately describe and justify PI roles and areas of responsibility?
- Is the level of effort for each team member appropriate for successfully conducting the proposed work?
- Does the application describe adequate availability of and access to facilities and resources (including patient populations, samples, and collaborative arrangements) to carry out the proposed research?
- Is the institutional support appropriate for the proposed research?
Criterion 5. Patient-centeredness
The application should demonstrate that the study focuses on improving patient-centered outcomes and employs a patient-centered research design (i.e., a design informed or endorsed by patients). (Note: The study can be patient-centered even if the end-user is not the patient, as long as patients will benefit from the information.)
The application should also address the following questions:
- Does the application include a thorough description about which outcomes (both benefits and harms) are important to patients, and are those outcomes included in the study plan?
- Does the application provide information that indicates that closing the evidence gap is important to patients and other stakeholders?
- Are the interventions being compared in the study available to patients now, and are they the best options for comparison (including whether they would be chosen by patients and their healthcare providers for managing the condition being studied)?
Criterion 6. Patient and stakeholder engagement
The application should demonstrate the engagement of relevant patients and other stakeholders (e.g., patients, caregivers, clinicians, policy makers, hospitals and health systems, payers [insurance], purchasers [business], industry, researchers, and training institutions) in the conduct of the study. Quality of engagement should be evaluated based on scope, form, and frequency of patient and stakeholder involvement throughout the research process.
The application should also address the following questions:
- Does the application provide a well-justified description of how the research team incorporates stakeholder involvement? Does the study include the right individuals (e.g., researchers, patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders) to ensure that the projects will be carried out successfully?
- Does the application show evidence of active engagement among scientists, patients, and other stakeholders throughout the research process (e.g., formulating questions, identifying outcomes, monitoring study, disseminating, and implementing)? Is the frequency and level of patient and stakeholder involvement sufficient to support the study goals?
- Is the proposed Engagement Plan appropriate and tailored to the study?
- Are the roles and the decision-making authority of all study partners described clearly?
- Are the organizational structure and resources appropriate to engage patients and stakeholders throughout the project?
In-Person Review
During preliminary review, PCORI evaluates and scores all administratively and scientifically compliant applications based on PCORI’s merit review criteria, including evaluating adherence to the PCORI Methodology Standards. After PCORI completes the preliminary review, PCORI program staff members evaluate panel scores and critiques to identify a subset of applications for merit reviewers to discuss at the in-person review meeting. Not all submitted applications move forward to in-person review.
During the in-person review, merit reviewers meet to discuss applications and to clarify the merits of the proposed research. They also identify areas for improvement. Each application is re-scored based on the content of discussion. The Panel Chair and PCORI MRO lead the in-person panel meeting and ensure that all applications receive a fair and thorough review according to the standards outlined in the PFA.
In-Person Applicant Presentation
Based on the results of the merit review and PCORI’s programmatic priorities, PCORI invites a selective subset of applicants whose proposed studies are deemed to be highly meritorious or aligned with PCORI’s strategic priorities to the second phase of project presentation and follow-up discussions with PCORI on study methodological and execution issues. We expect applicants to address concerns and critiques identified in the merit review in this presentation. We will notify the selected applicants of the logistics for this presentation (including travel arrangements) in separate communications.
Post-Panel Review
After the in-person meeting, PCORI program staff evaluate final merit review panel scores and comments, identify duplication or synergy among funded projects, and consider the fit of applications within the programmatic vision. Program staff members then recommend projects to a Selection Committee, which includes members of the Board. The Selection Committee considers recommendations and works with staff to identify a slate of applications for possible funding based on merit review scores, programmatic balance and fit, and PCORI’s strategic priorities. This slate then goes to the Board for consideration and approval.
In addition, PCORI evaluates applicant risk before issuing an award. Factors considered include financial stability, quality of management systems, audit findings, and past performance on PCORI awards (e.g., compliance with PCORI reporting requirements, conformance to PCORI terms and conditions on previous awards, and timely achievement of milestones). Based on the risk assessment, PCORI may impose special terms and conditions on awardees or withhold contract issuance until such business risks are mitigated. PCORI will not award new contracts to current awardees with overdue reports (progress, interim, final, etc.) until the awardees have submitted the overdue reports.
Summary Statements and Funding Recommendations
Applicants receive summary statements approximately two weeks before funding decisions are announced. If an application progresses to in-person discussion, the applicant will receive a summary statement that includes:
- In-person panel discussion notes
- Final average overall score
- Preliminary reviewer critiques
- Application quartile, which provides information for applicants to understand how they did relative to other discussed applications
Summary statements for applications that do not progress to in-person discussion include only the preliminary reviewer critiques.
PCORI makes funding recommendations by identifying meritorious applications that fit the programmatic needs and that satisfactorily address the merit review criteria while adhering to the PCORI Methodology Standards. PCORI also considers the funds allotted for the current PFA when deciding which applications to recommend to the Board for approval. Applicants to this current cycle’s PFA will receive summary statements and notification of the funding status of their application no later than May 2018.
VII. Appendix: Research Topics of Interest to PCORI
PCORI Topics
Please note that these topics are not listed in rank order. Please also note that proposing one of these topics as your research question does not obviate the requirement that your research question has to be sufficiently justified by a rigorous gap analysis as described in an existing systematic review, clinical guideline, or a de novo systematic review.
- Compare the effectiveness of different remote delivery approaches to evidence-based, non-pharmacological treatments for mild to moderate depression and anxiety conditions.
- PCORI is particularly interested in research that compares the effectiveness of different levels of intensity of monitoring, guidance and/or feedback, and of different types of professionals (e.g., technician, clinician, mental health clinician) providing such monitoring, guidance and/or feedback for Internet-based non-pharmacological treatment.
- Specific subgroups of interest include patients with differing severities of the target condition(s), as well as patients at risk of reduced access to care (e.g., rural populations, low-income individuals, and racial and ethnic minorities).
- Special Area of Emphasis topic focused on digital health interventions for treating anxiety in children, adolescents, and/or young adults:
- Compare the effectiveness of one or more digital applications of CBT to an appropriate active control (e.g., face-to-face CBT) for treatment of mild-to-moderate anxiety in children, adolescents, and/or young adults (through age 25 years). Digital applications may include computer-based therapy (e.g., delivered via the Internet, downloadable software) and/or smartphone applications of CBT that are appropriate for treatment of anxiety. PCORI has a preference for digital interventions available in the public domain to support accessibility of effective treatments.
- Compare the benefits and harms of pharmacologic, psychological, or combination treatments for treating different types of insomnia (sleep onset vs. sleep maintenance insomnia) on sleep and patient-centered outcomes including next-day function, mood, and quality of life.
- The proposed study should have treatment duration > 6 months.
- Proposed comparators must be adequately operationalized and the proposed operationalization must align with available evidence of efficacy. Proposed comparisons of interventions must address actual clinical choices faced by patients, caregivers, and clinicians in specific practice settings.
- Community-acquired pneumonia
- What are the comparative clinical effectiveness and safety of alternative Food and Drug Administration (FDA)-approved antibiotic regimens in the empiric outpatient treatment of adults with community-acquired pneumonia?
- Alternative regimens may include:
- Across-class comparisons of FDA-approved antibiotic regimens that are recommended for use or are commonly used in community-acquired pneumonia.
- Varying duration of treatment, such as three or five days, or discontinuation at resolution of symptoms of infection.
- The proposed studies should be of sufficient sample size to ensure the resulting evidentiary information is readily generalizable to the broader population. In addition, applications should address effectiveness in distinct subpopulations (e.g., patients with chronic conditions, immunosuppression, and the elderly).
- Alternative regimens may include:
- What are the comparative clinical effectiveness and safety of alternative Food and Drug Administration (FDA)-approved antibiotic regimens in the empiric outpatient treatment of adults with community-acquired pneumonia?
- Studies of patients with non-muscle invasive bladder cancer (NMIBC) who failed first-line treatments
- Compare the effectiveness of treatments (including various intravesical agents or bladder-preserving alternatives to cystectomy) in patients with intermediate- or high-risk NMIBC who have failed first-line induction intravesical therapy with BCG or other agents.
- Proposed studies should be large scale (preferably an RCT) and should include patient-centered outcomes such as mortality, progression to MIBC, side effects, and quality of life.
- Compare the effectiveness and safety of Screening, Brief Intervention, and Referral to Treatment (SBIRT) for adolescent alcohol abuse in different settings (school based versus primary-care based), using different delivery modes (in-person, remote, and computer based) or providers (physician, mental health specialist, nurse, staff worker, or peer).
- Surgical options for hip fracture in the elderly
- Compare the effectiveness of different surgical treatments in elderly patients with hip fractures in terms of functionality and other patient-centered outcomes.
- Multicomponent interventions to reduce initiation of tobacco use and promote cessation of tobacco use among high-risk populations with known disparities
- Compare the effectiveness of clinical interventions to reduce initiation of tobacco use and promote tobacco cessation among populations with known tobacco disparities, including high-risk and vulnerable populations.
- Integration of mental and behavioral health services into the primary care of persons at risk for disparities in health care and outcomes
- Compare the effectiveness of care models that integrate mental and behavioral health care—including substance abuse treatment—into the primary care provided by community health centers and other relevant settings, with the goal of reducing disparities in care (i.e., access to mental and behavioral health services and the diagnosis and treatment of mental and behavioral health conditions) and improving health outcomes among underserved populations, including racial and ethnic minorities, low-income individuals, and rural populations.
- Treatment strategies for symptomatic osteoarthritis (OA), including joint replacement
- Compare methods for deciding when to undergo OA surgery; use outcomes such as patient satisfaction, functional status, clinical status, and quality of life.
- Compare the effectiveness of strategies for engaging early stage OA patients to adopt behaviors that can prevent OA progression and disability.
- Compare different nonsurgical therapies (e.g., pharmacotherapy, injections, physical therapy or exercise, weight loss alone and in combination with other therapies, and complementary medicine alternatives) to prevent OA progression and disability. The studies should seek to identify heterogeneity of treatment response among important subgroups of patients.
- Improving outcomes in mothers and babies at risk for disparities by comparing evidence-based models of perinatal care
- Compare the effectiveness of multicomponent systems interventions—such as evidence-based models of perinatal care—aimed at improving outcomes like pre-term birth and low birth weight for mothers and babies at risk for health disparities.
- Clinical interventions to reduce nontraumatic lower extremity amputations in racial or ethnic minorities and low-income populations with diabetes: Does expert, protocol-driven, team-based care reduce the risk of nontraumatic lower extremity amputations compared with existing and established guideline-based care for racial or ethnic minorities and low-income populations with diabetes?
- Team-based care could include, for example, acute-care teams with patient teams; staff working as a team (triage staff, etc.); use of standing orders, EMRs, or hardcopy templates and reminders; knowledge and use of community resources; EMR registry functionality; empanelment; group visits; and novel outpatient-based interventions with rapid response from the outpatient setting.
- Compare the effectiveness of diverse models of comprehensive support services (e.g., incorporation of wrap-around services, alternative providers, and technology) for infants and their families or caregivers after discharge from the neonatal intensive care unit.
- PCORI is particularly interested in studies that would help bridge the gap between acute and post-acute care.
- Proposed research should plan, justify, and adequately power the study to address prespecified patient subgroups (e.g., urban, rural, and socioeconomically challenged).
- Compare the effectiveness of alternative delivery models (e.g., primary care, schools, and mobile vans) versus the dentist’s office in preventing dental caries in children in medically underserved areas.
- Compare the benefits and risks of different models of integrating pharmacists into the care transitions team in order to reduce adverse drug events, improve patient-centered outcomes, and lower preventable emergency department visits and re-hospitalizations post-hospital discharge among patients with multiple chronic comorbidities.
- Compare the effectiveness of evidence-based screening and primary prevention approaches, including different modes and settings (e.g., universal screening versus targeting at-risk individuals; virtual versus face-to-face screening; and within primary-care setting versus school based) at minimizing suicidality among adolescents.
- Institute of Medicine 100 Initial Topics for Comparative Effectiveness Research
- AHRQ Future Needs Projects
The intent of the SAC described in the PFA is to ensure that a broad spectrum of patients and other stakeholders advise and assist the research team with refining the study questions, outcomes, and protocols. These patients and other stakeholders must include national or regional organizations that represent—at a minimum—patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders. PCORI may recommend additional representation in collaboration with the applicant, including individual patients with lived experience and other relevant stakeholders, such as scientific and methodological experts. However, PCORI understands that engagement structures and approaches vary widely. Other engagement approaches, such as forming stakeholder groups, panels, task forces, working groups, and other bodies, or involving individual patient and other stakeholder partners in various ways are also permissible to employ—either in addition to or instead of—the formation of the SAC. The SAC provision is not meant to require that a separate governance or advisory entity be established beyond the study governance and advisory structure the awardee has planned, if an applicant already has an approach for including the relevant and required patient and other stakeholder partners. For clarification in your application materials and for merit review purposes, please indicate which body or structure is filling the SAC requirements, including the requirements for in-person meetings at least two times per year and appropriate budgeting.[A]
[1] “Effectiveness” is the extent to which an intervention does more good than harm across a broad mix of patients when provided under the usual circumstances of healthcare practice (modified from http://www.europarl.europa.eu/RegData/etudes/workshop/join/2013/518741/IPOL-ENVI_AT%282013%29518741_EN.pdf).
[2] “Efficacy” is the extent to which an intervention does more good than harm in ideal patients under ideal circumstances (modified from http://www.europarl.europa.eu/RegData/etudes/workshop/join/2013/518741/IPOL-ENVI_AT%282013%29518741_EN.pdf).
[3] Patsopoulos, N.A. A pragmatic view on pragmatic trials. Dialogues Clin Neurosci. 2011; 13:217–24; http://ncbi.nlm.nih.gov/pmc/articles/PMC3181997/.
[4] Thorpe, K.E., et al. A pragmatic-explanatory continuum indicator summary (PRECIS): A tool to help trial designers. CMAJ. 2009; 180:E47–E57; http://cmaj.ca/content/180/10/E47.full.pdf/.
[5] Loudon, K., et al. The PRECIS-2 tool: Designing trials that are fit for purpose. Research Methods & Reporting. 2015; 350:h2147; http://www.bmj.com/content/350/bmj.h2147. (Note that this article describes an updated process to assess how closely the proposed study design elements [e.g., delivery of intervention and population of interest] mirror those encountered in “usual care,” a proxy for pragmatic. The term “usual care,” as used in this article, differs from how PCORI interprets and uses the term in the context of CER funding announcements—a control comparator.)
[6] Choudhry, N.K., et al. Full coverage for preventive medications after myocardial infarction. N Engl J Med. 2011; 365(22): 2088–97; http://www.nejm.org/doi/full/10.1056/NEJMsa1107913/.
[7] ALLHAT Collaborative Research Group. Major outcomes in high-risk hypertensive patients randomized to angiotensin-converting enzyme inhibitor or calcium channel blocker vs. diuretic. JAMA. 2002; 288:2981–97; http://www.ncbi.nlm.nih.gov/pubmed/12479763/.
[8] Simon, G.E., et al. Initial antidepressant choice in primary care: effectiveness and cost of fluoxetine vs. tricyclic antidepressants. JAMA. 1996; 275(24): 1897–902; http://www.ncbi.nlm.nih.gov/pubmed/8648870/.
[9] Investigators are encouraged to consider core outcome sets, such as those developed by the Core Outcomes Measures in Effectiveness Trials Initiative (COMET) to facilitate cross-study analysis. See: http://www.comet-initiative.org/.