I. Introduction

Background

To improve our nation’s capacity to conduct clinical research more efficiently and to answer important questions that patients and clinicians face, the Patient-Centered Outcomes Research Institute (PCORI) provided $105 million in 2014 to begin building the infrastructure for the National Patient-Centered Clinical Research Network (PCORnet). This large clinical research Network represents people, patients, clinicians, systems, and health plans across the country and supports research that will improve health care and health outcomes.

Since 2013 PCORI has awarded more than $417.5 million to create the infrastructure of a national, patient-centered research network. The network currently includes 13 clinical data research networks (CDRNs) and 20 patient-powered research networks (PPRNs), along with a Coordinating Center. The following are central to the rationale for and the sustainability of this network: pre-existing, standardized, curated, and research-ready clinical data on large numbers of persons with specific clinical conditions and illnesses; actively engaged patients who join in governing the research uses of this data; distributed (rather than centralized) data that maximizes the security and local control of all data; a readiness among network members to collaborate and a willingness to share data and effort in pursuit of worthy research aims; the capacity to link data across data sources at the individual patient level to create complete, longitudinal data; and the ability, ultimately, to attract research funding from a variety of funding sources, including federal agencies, industry sponsors, and not-for-profit foundations.  

Having made the infrastructure investment, PCORI now intends to test the network’s readiness. Specifically this PFA aims to test the readiness of the PPRNs within PCORnet to develop and lead competitive research proposals that will depend on capacity in three critical areas: (1) the capacity to conceive and propose patient-driven comparative clinical effectiveness research (CER) questions and studies; (2) the capacity to accomplish data linkages between data held by PPRNs and data held by other sources that will enhance the proposed project’s quality and strengthen PCORnet over the long term; and (3) the capacity to attract participation and support from non-PCORI research funders.

Developing External Partnerships

PCORnet’s value will ultimately depend on its ability to meet the research needs of a range of funders external to PCORnet and PCORI. The goal is a network that attracts a diverse set of public and private funders of research, finds ways to work with other networks and registries, and actively encourages collaboration with researchers not presently affiliated with PCORnet. Respondents to this PFA must collaborate with stakeholders external to PCORnet—industry sponsors or other funding organizations—to secure partner funding (which may include direct financial contributions and in-kind support) before submitting the full application. At a minimum, Letters of Intent (LOIs) must specify the organization(s) that have been or will be targeted as potential collaborators and co-funders. Among comparably meritorious applicants, PCORI will prioritize its funding for those that have obtained greater contributions and support from partner funding. 

Advancing Data Integration

In Phase II of PCORnet, PCORI encouraged networks to increase their access to more complete, accurate, and timely patient and organization data. Greater data completeness may enhance the capture of relevant outcomes; in other cases, it may add details of specific procedures or treatments, disease severity, or the presence of comorbid illnesses. Data completeness can be enhanced in many ways, all of which require linkage of PPRN data with that from other sources. Examples of rich data sources include claims data from commercial insurers, singly or as already aggregated in several large consortia; Medicare or Medicaid data; electronic health record data (from PCORnet CDRNs or from other healthcare delivery settings); comprehensive and timely mortality data; and detailed clinical and self-reported data contained in disease-specific registries, such as those built by specialty clinician organizations. Linkages may also provide a sufficient sample size for CER studies for smaller PPRNs and those focused on rarer diseases. This solicitation, therefore, requires that applicant PPRNs describe the linkage(s) that they are including in the proposed study, how linkages will serve study needs, how linkages will be accomplished (e.g., using de-identified or identifiable data linkages), and approaches to Institutional Review Board (IRB) oversight of those linkages.

Funds Available

PCORI has allotted up to $21 million in total costs under this PFA to fund multiple (up to five) high-quality and impactful studies to answer important patient- and stakeholder-prioritized CER questions that remain unanswered due to insufficient or inconclusive evidence. The proposed budget for studies under this initiative may be up to $5 million in direct costs as appropriate. The maximum project period is three years. However, PCORI will consider exceptions to the research project period of three years (not to exceed four years) with justification during the application phase.

PCORI will not cover costs for usual patient care or for study interventions that constitute the procedures, treatments, interventions, or other standard care (patient care) under study. Provision of the interventional treatment costs by an external collaborating organization would be considered a highly appropriate contribution and, if these costs are substantial, would satisfy requirements for an external partner. Host healthcare delivery systems, third-party payers, product manufacturers, intervention developers, other interested parties, or any combination of these sources could cover these costs, and by doing so would become partnered collaborators in the study. Covering only the routine (“usual”) care costs and excluding the costs of one or both comparators would not be responsive or a contribution. In the case of multiple participating organizations (more than one PPRN, collaborating CDRNs, or external collaborators), PCORI encourages the use of a single IRB of record for the research study. The Awardee Institution is responsible for the study, including oversight and dispersion of awarded funds to all subcontracts, including institutions from the CDRNs, other PPRNs, Health Plan Research Networks (HPRNs), and the Coordinating Center. The Application Guidelines document contains additional details for developing the Research Plan.

Features of Patient-Centered Outcomes Research (PCOR)

PCOR is comparative effectiveness research that helps people and their caregivers communicate and make better-informed, personalized healthcare decisions, allowing their voices to be heard in assessing the value of healthcare options. This research:

• Assesses the benefits and harms of preventive, diagnostic, therapeutic, and palliative care to inform decision making, highlighting the choices that matter to people
• Focuses on outcomes that people notice and care about (including survival, functioning, symptoms, and health-related quality of life) and is inclusive of an individual’s preferences, autonomy, and needs 
• Incorporates a wide variety of settings and diversity of participants to address individual differences and barriers to implementation and dissemination
• Directly compares clinical interventions that are or could soon become available in the clinical setting
• Obtains stakeholder perspectives to address the burdens to individuals, availability of services, and requirements for technology and personnel

Characteristics and Objectives of Comparative Clinical Effectiveness Research (CER)

PCORI seeks to support new research that addresses critical clinical and health-related questions faced by patients, their caregivers, and their providers. PCORI seeks to fund stakeholder-driven, investigator-initiated CER displaying the following characteristics:

• The research measures and compares the benefits and harms of different interventions and strategies that are currently delivered in typical clinical and community settings for screening, diagnosis, treatment or management of illnesses.
• The study should demonstrate a clear potential to change practice or to reduce practice variation for the condition(s) under study.
• The research compares at least two alternative clinical approaches. Because PCORI’s mission is to develop evidence to inform difficult decisions, we strongly prefer applications that propose to compare well-defined interventions that are already being used in the condition and the population of interest. In the case of newly approved treatments and technologies, acceptable studies may compare the new technology to the currently used technology for the same condition. 
• The research examines such interventions as specific drugs, devices, procedures, assistive technologies, behavioral change, communication or dissemination, or complementary treatments and may compare strategies for screening, diagnosing, treating, or managing illnesses. Studies may also address complex interventions occurring at, or pertaining to, care delivery systems. Please note that “usual care” is not a sufficiently described comparator for CER studies submitted to PCORI. “Usual care” is too often ill defined; difficult to quantify; and subject to considerable geographic and temporal variations, limiting interpretability, applicability, and reproducibility. If the applicant proposes “usual care” as a rational and important comparator in the proposed study, then the actual care expected in the usual care arm must be described in detail in the proposal, must be coherent as a fully acceptable clinical alternative, and justified properly as the best legitimate comparator (e.g., “usual care” could be current guidelines-based care or current “optimal management”). The applicant must also include an explanation of how the care given in the “usual care” group will be measured in each patient, to the extent possible, during the study and the nature of inferences that will be appropriately drawn from such a comparative study. 
• A major component of patient-centered research is that the research compares health outcomes that include those meaningful to the study’s patient population (e.g., morbidity, mortality, symptoms, functional status, quality of life, and absenteeism from work or school). Such outcomes should be measured using validated methods. In select instances, surrogate physiological measurements may be sufficiently linked to final health outcomes to be of interest, but they might not be the sole study outcomes. Outcomes should be supported by careful involvement or consultation in study planning by affected patients – NOT necessarily by study participants.

Back to top

II. Guidance for Preparing Applications

Specific Requirements

The proposed study should meet the following requirements:

• Focus on a comparative clinical effectiveness question that is important to patients and other decision makers and that has been formulated in consultation with patients and other stakeholders.
• Compare strategies for prevention, screening and diagnosis, treatment, or population management.
• The applicant must receive endorsement from relevant patient organizations, clinician organizations, payer or purchaser consortia, and life sciences industry representatives that the study addresses a critical question—one that, if adequately answered, would substantially improve decision making.
• Address an evidence gap in deciding among available options. Optimally, this gap should have been substantiated by an existing (recent or updated), rigorously conducted systematic review or emphasized by an official professional society’s clinical practice guideline.
• Demonstrate consultation with patients and other stakeholders or their representative groups, or reference previously documented decisional dilemmas to determine if the study is answering a critical question—one that, if adequately answered, would substantially improve decision making.
• Propose a sample size that is sufficiently large to allow for precise estimation of hypothesized effect sizes or for clear demonstration of non-inferiority. The sample size must also support testing of a priori hypotheses related to potential differences in effectiveness among relevant patient subgroups (HTE). Carefully explain in your application your choice for the proposed sample size, and the consequences of that sample size.
• Examine diverse populations receiving care in real-world settings. Specify broad and simple eligibility criteria that allow for wide generalization of results while attending appropriately to ethical concerns of excess risk in some patient subgroups. Highly selected patient populations are not of interest. 
• For studies aiming to reduce or eliminate health or healthcare disparities, specify one or more of the Addressing Disparities Program target populations (i.e., racial or ethnic minorities; low-income groups; residents of rural areas; individuals with special healthcare needs [including individuals with disabilities]; individuals with low health literacy or numeracy, or limited English proficiency; and lesbian, gay, bisexual, transgender, and questioning [LGBTQ] persons) that will be the focus of the study. Studies should test the ability of interventions to improve outcomes (including patient-centered, clinical, and structural outcomes) and reduce disparities for at-risk populations.
• Demonstrate strong interest from and support of host delivery systems and clinical care settings when appropriate (i.e., when the intervention is delivered or data is collected within the system).
• Compare interventions that are known to be efficacious, effective, or that are already in common use and that can be implemented in real-world settings.
• Feature near-term outcomes, including patient-reported outcomes (PROs) as primary outcomes, when appropriate.
• Plan to collect patient-centered outcome data efficiently and periodically during follow-up as appropriate.
• Provide preliminary evidence of the potential for efficient recruitment, high participation rates, and appropriate oversight by local or centralized IRBs, including plans for streamlining or waiving individual informed consent in cases of low-risk interventions (if applicable). PCORI believes that the intensity of oversight and the complexity of informed consent procedures should be closely related to the degree of risk from study participation. Applicants must address this issue and present evidence that the study will not encounter significant barriers to approval or to recruitment and participation plans. The relevant IRBs make the final determination regarding the adequacy of informed-consent procedures and participant protections.
• Adhere to all applicable PCORI Methodology Standards. The full application will require the applicant to identify the standards appropriate to the proposed study and to describe how the study team plans to address each standard.
• In the case of randomized control trials (RCTs), the study should also adhere to current best practices (standardized inclusion or exclusion criteria; proper randomization; techniques to minimize potential for missing data; and appropriate safety monitoring, including establishing a Data and Safety Monitoring Board [DSMB] or indicating why such a board is unnecessary).

To carry out studies that allow for adoption of the findings in a real-world setting and to maximize the efficient use of resources, take care to prevent these trials from becoming more complex and onerous than necessary. We encourage the applicant to be creative and consider the following innovative strategies, as appropriate and feasible:

• Consult with patients and other stakeholders on their decisional dilemma and evidence needs, or reference previously documented decisional dilemmas in preparation for submitting LOIs and full applications.
• Carefully describe the pertinent evidence gaps and why the project questions represent decisional dilemmas for patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders. Similarly, applicants should document why project outcomes are especially relevant and meaningful endpoints for patients and their families. 
• Minimize disruption to participants’ daily routines (e.g., minimize participant visits intended for study assessment purposes and capture PROs during office visits, electronically, or via phone).
• Design the study so that you can conduct it using routine clinic or office operations. Minimize disruption to clinical care when conducting portions of the study within clinical settings.
• Use efficient methods to obtain participant consent (e.g., electronic consent via a website or email) while still meeting ethical and legal requirements.
• Capitalize on existing electronic health records (EHRs) and other computerized information to identify and recruit eligible patients, monitor study conduct and patient safety, and collect study outcomes information.
• Take advantage of the data standardization and interoperability of PCORnet’s Common Data Model (CDM) to the full extent possible. Point out any areas where the study could provide opportunities to expand or enhance standardization of electronic data across study sites.
• Identify and engage with major patient and other stakeholder organizations that would help disseminate and implement study findings.

Non-responsiveness

Applications will be considered nonresponsive to this PFA if the proposed research:

• Tests efficacy (or comparative efficacy) of interventions that are novel or have limited evidence of efficacy. Applications should discuss current evidence on the efficacy and effectiveness of proposed interventions. New combinations of interventions known to be effective singly are acceptable. 
• Involves studies conducted within tightly controlled research environments instead of in clinical settings reflective of real-world healthcare delivery.
• Conducts a formal cost-effectiveness analysis.
• Directly compares the costs of care between two or more alternative approaches to providing care.
• Conducts studies of the natural history of disease, instrument development, pharmacodynamics, and/or fundamental science or biological mechanisms.
• Evaluates validity or efficacy of (rather than the comparative clinical effectiveness of) new or existing decision-support tools. This includes the development and efficacy evaluation of decision-support or shared-decision tools or systems for patients, clinicians, or both.
• Simply develops clinical prediction or prognostication tools without evaluating the impact of their use on clinical outcomes.
• Is a pilot study intended to inform larger efforts. This PFA is not soliciting pilot studies.
• Is a descriptive epidemiologic study. This PFA is not soliciting descriptive epidemiologic studies. 
• Compares interventions for which the primary focus or the sole intervention is examining the role of compensated or volunteer community health workers, including patient navigators.
• Has not received endorsement from relevant patient organizations, clinician organizations, payer or purchaser consortia, and life sciences industry representatives as potentially answering a critical question.

Proposals may report use of any health services, but may not employ direct measurements of care costs. For further information, please reference our cost-effectiveness analysis FAQs.

PCORI does have an abiding interest, however, in studies addressing questions about conditions leading to high costs to the individual or to society. This is included in our review criterion on the potential for research to fill a critical gap in knowledge or practice. As a result, PCORI is interested in studies that:

• Examine the effect of costs on patients, such as patients’ out-of-pocket costs, hardship or lost opportunity, or costs as a determinant of or barrier to access to care.
• Address cost-related issues, such as the resources needed to replicate or disseminate a successful intervention.
• Evaluate interventions to reduce health-system waste or increase health-system efficiency.

Leveraging Existing Resources

PCORI encourages applicant PPRNs to propose studies that leverage existing data resources within the PPRN and additional data resources such as those of CDRNs, other PPRNs (if appropriate), or data sources external to PCORnet. These may include local or national registries of patients with the disease(s) under study. Registries built and managed by charitable foundations, disease advocacy organizations, physician specialty organizations, the life sciences industry, or the federal government might be of interest, as would Medicare or Medicaid data and vital statistics data of the Social Security Administration or the National Death Index. All linked sources are of interest to the extent that they can be shown to expand the study population’s size or diversity, or to enhance the capture of needed data on the study population. Examples include claims data from insurers or from the Centers for Medicare and Medicaid that captures outcomes, healthcare utilization, or vital status; electronic medical record data that can add information on disease status and severity, comorbidity, laboratory, pathology, or other clinical information; and registry data that may add additional details of treatments or patient-reported data on outcomes, symptoms, or functional status. A key feature is that data must be linkable at the level of the individual patient or person. You must explain how you will accomplish this and provide clear evidence of the data partners’ willingness and capacity to effect such linkages. Both direct linkages using identifiable information and anonymous linkages may be feasible, but you must state this distinction and explain how the linkages will meet patient consent requirements for the relevant overseeing IRB(s).

Applicants proposing use of data from CDRNs, other PPRNs (if appropriate), or data sources external to PCORnet should address the following in the Research Plan (as appropriate), with sufficient specificity:

• Identify and justify all participating research network entities (e.g., health plans, consortia projects, disease registries, etc.). For PCORnet, identify the names of participating CDRNs, PPRNs, Health Plans Research Networks (HPRNs), their affiliated study performance sites, and PCORnet Collaborative Research Groups (CRGs) that will be collaborating on the project. Also, describe how you will minimize risk to the proposed study throughout the performance period after PCORnet Phase II infrastructure funding (to participating CDRNs, PPRNs, HPRNs, and/or CRGs) ends.
• Demonstrate that the proposed data source(s) can comprehensively capture the study variables needed to assess the interventions, covariates, and outcomes.
• Describe how you will link and manage data across proposed study sites, research consortia, and/or other collaborating organizations and whether you will use any dedicated data-coordinating functions or facilities. When feasible and relevant, the PCORnet Coordinating Center should meet data coordination needs.
• Provide a study management structure that identifies roles, responsibilities, and decision-making authority across the proposed research consortia.
• As applicable, provide a timeline for establishing data-use agreements and linkages.
• As applicable, describe any PCORnet infrastructure resource(s) used to conduct the study (i.e., Coordinating Center, streamlined IRBs, contracting, engagement and consenting processes, standardized data resources training, etc.).
• Indicate the experience of participating sites in the use of centralized versus localized IRBs.
• As applicable, you must document the involvement of partnered networks or entities in the study with detailed Letters of Support, clear budgets, and budget justifications that cover the costs of each network or entity’s described contributions.
• The proposed project should use the Common Data Model (CDM) to the full extent possible.   
• As applicable, describe the use of resources from the People-Centered Research Foundation (PCRF) as available (e.g., Coordinating Center support).

NOTE: Do not include in the LOI/application budget any funding for work that is already covered by funding from PCORI or other funders for an existing research network, research consortium, or other related data resource (e.g., Phase II infrastructure funding). The budget should include any funding needed to support project work initially covered by another source (such as Phase II infrastructure or other project support) that will decrease or end during the lifetime of the PaCR project.

Preliminary Data and Use of Accepted Measures

PCORI encourages applicants to design their research using valid patient-centered outcome measures. These may include both patient-reported outcomes and outcomes obtained from electronic data sources. Include preliminary data or references that support the validity of the proposed measures in similar study populations. We also encourage investigators applicants to consider those measures described in the Patient-Reported Outcomes Measurement Information System (PROMIS) when applicable.

Studies in Rare Diseases

PCORI has an ongoing interest in investigating strategies addressing care for patients with rare diseases. Rare diseases are defined as “life-threatening” or “chronically debilitating” conditions of such low prevalence (affecting fewer than 200,000 in the United States [i.e., less than 1 in 1,500 persons]) that special efforts—such as combining data across large populations—might be needed to study them. Therefore, this announcement is well suited to support studies of patient populations with rare diseases.

Methodological Considerations

Regardless of study design, applications must adhere to all relevant PCORI Methodology Standards. These include 48 individual standards that fall into 12 categories. The first five categories are cross-cutting and relevant to most PCOR studies. Researchers should refer to all of these standards when planning and conducting their research projects. These cross-cutting categories are:

1. Standards for Formulating Research Questions
2. Standards Associated with Patient-Centeredness
3. Standards on Data Integrity and Rigorous Analyses
4. Standards for Preventing and Handling Missing Data
5. Standards for Heterogeneity of Treatment Effect (HTE)

In addition to these five sets of standards, the first standard of “Standards for Causal Inference Methods”—(CI-1)—is cross-cutting and applicable to all PCOR studies.

The seven other standards categories will be applicable to particular study designs and methods. Applicants should use the standards in each of these categories as guidance when they are relevant to a study. These seven categories are:

1. Standards for Data Registries
2. Standards for Data Networks as Research-Facilitating Structures
3. Standards for Causal Inference Methods
4. Standards for Adaptive and Bayesian Trial Designs
5. Standards for Studies of Medical Tests 
6. Standards for Systematic Reviews
7. Standards for Research Designs Using Clusters

Most of these standards are minimal. The PCORI Methodology Standards reflect practices that applicants should follow in all cases, and you must explain and justify all deviations. Applicants should address additional best practices—including relevant guidelines for conducting clinical trials developed by other organizations—in the application for PCORI funding. To help reviewers quickly identify adherence to a particular standard, applicants must cite each relevant PCORI Methodology Standard within the PCORI Methodology Standards Checklist, following the instruction in the checklist itself and in the Application Guidelines. Program staff use the checklist to evaluate applications.

Applicants should specifically discuss their capacity to measure such factors as differential adherence to chosen treatments (or participation in intervention programs) that could create or explain apparent differences in the effectiveness of the alternative interventions being compared in clinical populations.

Patient and Stakeholder Engagement

PCORI encourages all applicants to outline how patients, including members of the PPRN, non-PCORI funding partner(s), and other stakeholders (caregivers, clinicians, delivery systems, health plans, and others) have participated in the planning of the application and will participate as partners in various phases of the proposed research. Before completing this section of the Research Strategy, we encourage you to review PCORI’s Engagement Rubric, which can be found in the PCORI Funding Opportunities. Applicants should also review the PCORI Methodology Standards Associated with Patient-Centeredness and PCORI’s Sample Engagement Plans. The rubric and Sample Engagement Plans are not comprehensive or prescriptive; instead, they provide a variety of examples to incorporate engagement, where relevant, into the research process. PCORnet’s Engagement Committee, the PCORnet Commons, and the PPRN Coordinating Center may all offer support and information of value in planning your proposal. 

PCORI expects applicants to consult with patients and other stakeholders to identify and clarify the decisional dilemma and evidence needs that you will address in the proposed study.  Alternatively, if decisional dilemmas have previously been identified, these may be referenced in preparation of the LOI and application. To describe the decisional dilemma, state the specific clinical decision(s) or screening and diagnostic or treatment choice(s) the decision makers face, and explain how the findings from the proposed research will inform those choices. State why this decision—such as choosing a specific medication, surgical approach, or care delivery strategy to treat a condition or manage a specific population—is important to patients in terms of potential benefits and risks, out-of-pocket costs, and preferences. Document the uncertainty patients and other stakeholders face when making this decision. Identify the patients and other stakeholders you consulted in planning the study, or reference the source of information that helped you to identify the decisional dilemma. Applicants should document how the project outcomes chosen are especially relevant and meaningful for patients and other stakeholders.

Populations Studied

PCORI seeks to fund research that includes populations diverse in age, gender, race, ethnicity, geography, and clinical status. PCORI recognizes that some proposed studies might represent important PCOR opportunities, even in the absence of a broadly diverse study population. However, the burden is on the applicant in such cases to justify the study’s importance in the absence of diversity. You should present and discuss preliminary data showing aspects of the diversity of the PPRN membership or of the planned study population. 

PCORI has an ongoing interest in including previously understudied populations for whom effectiveness information is needed, such as hard-to-reach populations or patients with multiple conditions. PCORI is also particularly interested in studying the possibly different impact of strategies in various subpopulations, with attention to the possibility that the strategy’s effects might differ across subgroups. PCORI has developed the following list of populations of interest to guide our research and engagement efforts:

• Racial and ethnic minority groups
• Low-income groups
• Women
• Children (age 0–17 years)
• Older adults (age 65 years and older)
• Residents of rural areas
• Individuals with special healthcare needs, including individuals with disabilities
• Individuals with multiple chronic diseases
• Individuals with rare diseases
• Individuals whose genetic makeup affects their medical outcomes
• Individuals with low health literacy, numeracy, or limited English proficiency
• LGBTQ persons
• Veterans and members of the Armed Forces and their families

Project Budget and Duration

Applicants may request up to $5 million in total direct costs for a research project period not to exceed three years (not including peer review). At the time of contract execution, PCORI sets aside all of the funds associated with an awarded project to be made available throughout the contract’s period of performance. The maximum budget includes all research-related costs as well as costs associated with PCORI’s peer-review process. (Please refer to the Application Guidelines for further details.) In general, PCORI will not cover the costs for interventions that are being compared in the proposed study. (See Appendix 2: Allowable and Unallowable Costs in the Application Guidelines for details.) Applicants should submit a realistic budget and timeline reflecting the proposed study’s scope and requirements. PCORI will consider exceptions to the research project period of three years (not to exceed four years) with justification when submitting the application. PCORI will not consider exceptions to the budget limit of $5 million in direct costs. However, PCORI will respond to an applicant’s inquiry regarding budget-related issues during the period leading up to the submission deadline. Note that although subcontractor indirect costs are included in the prime applicant’s direct-cost budget, PCORI does not factor in subcontractor indirect costs when determining adherence to the PFA’s direct-cost limit.

A contract is the funding mechanism for this program. A milestones and deliverables schedule, as well as specified recruitment targets, should be linked directly to and included in the proposed budget that will be subject to negotiation at the time of award. Some of the other activities that PCORI will consider during negotiations include:

• Developing a study protocol and procedure manual for the intervention
• Assigning roles and responsibilities to study team members for project implementation
• Forming an appropriate engagement body
• Providing a detailed task-based budget with level of effort for project staff, specified by task
• Obtaining clearances from all institutional and community partners, including IRB approvals
• Establishing a DSMB or providing a clear description of why one is unnecessary
• Executing all subcontractor agreements
• Agreeing on eligible patient populations for study recruitment
• Identifying barriers to patient recruitment in the study and addressing these barriers effectively
• Structuring a feasibility phase to demonstrate the potential for successful recruitment

Total project funding is contingent upon successful programmatic and budget performance (e.g., meeting recruitment targets). Awardees must provide corroborating evidence to receive continuous funding support. Specifically, after 12 months of study performance, but no later than 18 months, PCORI will use information from the awardee to conduct a formal programmatic assessment of the study's progress and specified recruitment targets to determine its viability and sustainability. Only studies that PCORI deems satisfactory in this assessment will receive continuous funding support.

Refer to the Application Guidelines for a list of additional project milestones specific to this PFA.

Protection of Human Subjects

This component (up to five pages) is in the Research Plan Template. Describe the protection of human subjects involved in your proposed research. PCORI follows the Federal Policy for the Protection of Human Subjects (45 CFR part 46), including the Common Rule. For more detailed information, please see Section 5, titled “Human Subjects Research Policy” in the Supplemental Grant Application Instructions for All Competing Applications and Progress Reports, which is issued by the U.S. Department of Health and Human Services. PCORI does not require that applicants comply with sections of this policy referring to requirements for federal-wide assurance or that refer to standards for including women, minorities, and children. Awardees must also comply with appropriate state, local, and institutional regulations and guidelines pertaining to the use of human subjects in research.

PCORI requires awardees to ensure that there is a Data and Safety Monitoring Plan, which may include the need to appoint a DSMB, as provided in the PCORI Policy on Data and Safety Monitoring Plans for PCORI-Funded Research.

PCORI merit reviewers will examine plans for protection of human subjects in all applications and may provide comments regarding the plans (see How To Evaluate Human Subjects Protections). Reviewers’ comments on human subject research are not reflected in the overall application score, but PCORI staff might use them during funding negotiations. Final determinations about the adequacy of human subject protections rest with the IRB or international equivalent that has jurisdiction for the study.

The Awardee Institution, whether domestic or foreign, bears ultimate responsibility for safeguarding the rights and welfare of human subjects in PCORI-supported activities.

Required Education of Key Personnel on the Protection of Human Research Participants

PCORI requires that all applicants adhere to the National Institutes of Health (NIH) policy on education in the protection of human research participants in the conduct of research. This applies to all individuals listed as key personnel in the application. The policy and FAQs are available on the NIH website.

Data Management and Data-Sharing Plan

PCORI encourages openness in research and making research data available for replication and reproducibility. Although not required to be submitted as a component of the research application, if an award is made, the awardee must develop and maintain a plan addressing data management and data sharing of research project data. This must be done in a manner that is appropriate for the research project and the types of research project data, and in a manner consistent with applicable privacy, confidentiality and other legal requirements.

Recruitment

For studies that require recruitment of subjects, applications should include information about the size and representativeness of the potential recruitment pool of patients (across all contributing sites) and the means by which this size estimate was determined (e.g., electronic medical records, claims records, clinic logs, or other administrative systems). Likewise, applications should provide evidence-based estimates of how many participants are ultimately expected in the study, based on expected recruitment applying the study’s inclusion and exclusion criteria; anticipated acceptance (or refusal) rates; other factors, such as loss to follow-up; and any preliminary data. Such estimates must be discussed in the application, specified in the milestones, reviewed by Merit Review Officers (MROs) and PCORI staff, and monitored by PCORI in the funded research.

Peer Review and Release of Research Findings

PCORI has a legislative mandate to ensure the scientific integrity of the primary research it supports and to make study findings widely available and useful to patients, clinicians, and the general public within a specific timeframe. Accordingly, the PCORI Board of Governors (Board) adopted the Process for Peer Review of Primary Research and Public Release of Research Findings.

In summary, Awardee Institutions must submit to PCORI for peer review a draft final research report that provides the methodological details, describes the main study results, and interprets the findings in clinical or other decisional contexts. Subject matter experts (SMEs); individuals with expertise in research methodology or biostatistics; and patients, caregivers, and other healthcare stakeholders will review the draft final research report. After Awardee Institutions have responded to reviewers’ comments to PCORI’s satisfaction, the report will be accepted and considered final. PCORI will then prepare a 500-word abstract summarizing the study results for patients and the general public, which the Awardee Institution will review and approve.

PCORI will post the following materials on its website no later than 90 days after it accepts the draft final research report: (1) a 500-word abstract for medical professionals; (2) a standardized summary of the study results for patients and the general public; (3) a link to the study record on ClinicalTrials.gov (as applicable); and (4) ancillary information, including conflict-of-interest disclosures. The final research report, along with anonymized reviewer comments, will be made publicly available on the PCORI website no later than 12 months after its acceptance, except by prior mutual agreement with the Awardee Institution.

Back to top

III. How To Submit an Application

Letter of Intent (LOI)

Applicants should download the Cycle 2 2017 PaCR LOI Template from the PCORI Funding Opportunities. They must complete the document and convert it to a PDF with a three-page limit, excluding references. PCORI suggests including all references as in-text citations using American Medical Association citation style, but we do accept other citation styles. Do not upload additional documents as part of your LOI, such as Letters of Endorsement or Support, because they are not requested at this stage. Their inclusion will result in LOI rejection without review. Please visit the PCORI Funding Opportunities for additional applicant resources, including FAQs and required templates.

Please answer all of the questions in the LOI Template. This includes the question asking for a brief justification for the study’s proposed cost. Providing the answer “costs not to exceed $5 million” is not sufficient. Upload your document to PCORI Online. The deadline for LOI submission is September 6, 2017, by 5 p.m. (ET).

LOI Review

PCORI evaluates LOIs based on the following criteria:

• Importance of the research question to current clinical decision making, as evidenced by critical gaps identified by clinical guidelines developers or recent relevant systematic reviews
• Study scope sufficient to have a significant impact on patient outcomes or healthcare practices
• Clarity and credibility of applicants’ responses to the LOI questions—including the rationale for the estimated sample size (i.e., citing published estimates, including effect sizes, standard deviations, and the need for rigorous comparative analysis of important subgroups); proposed partnerships and the anticipated level of partner-funding supporting the project; plans for advancing data integration; and use and enhancement of existing PCORnet infrastructure resources
• Prior relevant experience of the study team, including the Principal Investigator (PI) and other key personnel
• Programmatic fit and balance, considering whether the application significantly overlaps with concurrent applications or previously funded studies or, conversely, whether the application fills a gap in PCORI’s portfolio, considering such characteristics as disease category, topics, priority population, and methodologies
• Adherence to the administrative and formatting requirements listed in the Application Guidelines, especially the three-page limit for the LOI

PCORI reviews LOIs qualitatively; we do not score them. PCORI will invite applicants whose LOIs are most responsive to this PFA and to the criteria above to submit a full application. At least two PCORI staff review each LOI. Notification of approval or denial to submit a full application will occur no later than September 26, 2017. Please refer to the Application Guidelines in the PCORI Funding Opportunities for due dates and information on how to submit your LOI in PCORI Online.

If you are invited to submit an application, do not make significant changes to your proposed project without consulting a program officer. For example, you should not revise your major aims and study design. Any significant changes are grounds for removal from the review process.

Because of the complex requirements for partnered funding and data linkage in this solicitation, PCORI staff are willing and available to discuss proposal ideas with potential applicants both before and after the LOI submission deadline. The appropriate staff person may be reached by emailing [email protected].

Note: A PI can only submit one LOI per PFA. However, an individual listed as a PI on one LOI can be listed as and serve in another non-PI role (e.g., co-investigator or consultant) on other LOIs submitted under this limited PFA that are not on similar research topics and projects. This applies to single- and dual-PI submissions. Similarly, each PPRN may be the lead PCORnet affiliation for one LOI. However, a PPRN can be listed and serve as a collaborating entity on additional LOIs.

Submission Dates

You must submit LOIs and applications in accordance with the published dates and times listed in the Overview section of this document and in the PCORI Funding Opportunities.

PCORI Online System

To submit an application, you must register in PCORI Online and submit an LOI and an application for each cycle to which you are applying.

Back to top

IV. Applicant Resources

• Partnerships to Conduct Research within PCORnet PFA (Cycle 2 2017)
• Partnerships to Conduct Research within PCORnet Application Guidelines (Cycle 2 2017)
• Partnerships to Conduct Research within PCORnet Application FAQs (Cycle 2 2017)

Back to top

V. Merit Review

PCORI’s merit review process is designed to support the following goals:

• Identify applications that have the strongest potential to help patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders make informed decisions to improve patient outcomes.
• Implement a transparent, fair, objective, and consistent process to identify these applications.
• Elicit high-quality feedback that reflects a diversity of perspectives to ensure that the PCORI-funded research reflects the interests and views of patients and other stakeholders and those who care for them, and that it meets the criteria for scientific rigor.
• Fund projects that fill important evidence gaps and have strong implementation potential.
• Regularly evaluate and continually improve the merit review process and policies in support of PCORI's mission.

PCORI merit review is a multiphase process that includes PFA development; staff evaluation of LOIs; the review panel’s preliminary review of full applications; an in-person panel discussion of a subset of full applications (identified by PCORI’s Research Priority Area Program staff and based on the preliminary review and program priorities); the Selection Committee’s recommendation of applications for funding; and, finally, Board award approval.

Preliminary Review

PCORI conducts rigorous merit review of the full applications it receives. Note that PCORI may eliminate applications from the review process for administrative or scientific reasons (e.g., non-responsiveness). PCORI may administratively withdraw an application if it is incomplete; submitted past the stated due date and time; or does not meet the formatting criteria outlined in the Application Guidelines, in the PCORI templates, and in PCORI Online. Applications that do not include partner funding (may include in-kind) from industry or organization collaborators will also be considered nonresponsive and administratively withdrawn. An application can be scientifically withdrawn if it is not responsive to the guidelines described in this PFA, describes research that is not comparative, includes a cost-effectiveness analysis, or otherwise does not meet PCORI programmatic requirements.

PCORI MROs recruit each panel based on the number of and topic areas represented by invited LOIs. MROs recruit the panel chair, scientist reviewers who are SMEs, patient representatives, and representatives of other stakeholder groups. All panel members receive training during the review cycle to ensure that they understand the programmatic and organizational goals of review.

We designed the table below to help applicants understand how the PCORI merit review criteria align with criteria from other funding organizations with which applicants might be familiar (e.g., NIH). Though PCORI’s criteria do map to most NIH criteria, there are areas where we ask for different information (i.e., PCORI does not include a criterion that tracks to NIH’s innovation criterion, but does include criteria evaluating patient-centeredness and engagement) reflecting PCORI’s unique approach.

Merit Review Criteria

Below are PCORI’s merit review criteria. PCORI’s merit review panels use these criteria during the preliminary and in-person review phases to evaluate and score all submitted applications and to ensure consistency and fairness in application evaluation.

Criterion 1. Potential for the study to fill critical gaps in evidence

The application should address the following questions:

• Does the application convincingly describe the clinical burden?
• Does the application identify a critical gap in current knowledge as noted in systematic reviews, guideline development efforts, or previous research prioritizations?
• Does the application identify a critical gap in current knowledge, evidenced by inconsistency in clinical practice and decision making?
• Would research findings from the study have the potential to fill these evidence gaps?

Criterion 2. Potential for the study findings to be adopted into clinical practice and improve delivery of care

The application should describe how evidence generated from this study could be adopted into clinical practice and delivery of care by others. The application should also address the following questions:

• Does the application identify who will make the decision (i.e., the decision maker) or use (i.e., the end-user) the study findings (not the intervention) this study produces, such as local and national stakeholders?
• Does the application identify potential end-users of study findings—such as local and national stakeholders—and describe strategies to engage these end-users?
• Does the application provide information that supports a demand for this kind of a study from end-users?
• Would this study’s research findings have the potential to inform decision making for key stakeholders? If so, provide an example. How likely is it that others could reproduce positive findings, resulting in improvements in practice and patient outcomes? Identify the potential barriers that could hinder others from adopting the intervention.
• Does the application describe a plan for how to disseminate study findings beyond publication in peer-review journals and at national conferences?

Criterion 3. Scientific merit (research design, analysis, data linkages, and outcomes)

The application should show sufficient technical merit in the research design to ensure that the study goals will be met. The application should also address the following questions:

• Does the application describe a clear conceptual framework anchored in background literature, which informs the design, key variables, and relationship between interventions and outcomes being tested?
• Does the Research Plan describe rigorous methods that demonstrate adherence to the PCORI Methodology Standards?
• Is the overall study design justified?
• Are the patient population and study setting appropriate for the proposed research question?
• Does the application provide justification that the outcome measures are validated and appropriate for the population?
• Are each of the comparators (e.g., active intervention arm and comparator arm) described clearly justified? If “usual care” is one of the arms, is it adequately justified and will it be sufficiently measured?
• Are the sample sizes and power estimates appropriate? Is the study design (e.g., cluster randomized design, RCT, or observational study) accounted for, and is the anticipated effect size adequately justified?
• Is the study plan feasible? Is the project timeline realistic, including specific scientific and engagement milestones? Is the strategy for recruiting participants feasible? Are assumptions about participant attrition realistic, and are plans to address patient or site attrition adequate? As applicable, does the application provide a justification for the research project period to extend beyond three years (not to exceed four years)?
• Does the application clearly describe data linkages between the required data sources (i.e., patient data, electronic health record data, and disease-specific registry data) to facilitate the conduct of the proposed study?
• Does the proposed project provide an opportunity to utilize and enhance aspects of the PCORnet infrastructure? Does the proposed project describe a clear plan for minimizing risk in the absence of PCORI Phase II infrastructure funds used to support the identified participants from within PCORnet? As applicable, does the application describe the use of resources from the People-Centered Research Foundation (PCRF) as available (e.g., Coordinating Center support)?

Criterion 4. Investigator(s) and environment

This criterion should assess the appropriateness (e.g., qualifications and experience) of the investigator(s)/team and the environment’s capacity (e.g., resources, facilities, and equipment) to support the proposed project. It should not be an assessment of the institution’s quality.

The application should address the following questions:

• How qualified are the PIs, collaborators, and other researchers to conduct the proposed activities? Is there evidence of sufficient clinical or statistical expertise (if applicable)?
• Does the investigator or co-investigator have demonstrated experience conducting projects of a similar size, scope, and complexity?
• If the project is collaborative or dual-PI, do the investigators have complementary and integrated expertise? Are the leadership, governance, and organizational structures appropriate for the project?
  • (Dual-PI Option Only) Does the Leadership Plan adequately describe and justify PI roles and areas of responsibility?
• Is the level of effort for each team member appropriate for successfully conducting the proposed work?
• Does the application describe adequate availability of and access to facilities and resources (including patient populations, samples, and collaborative arrangements) to carry out the proposed research?
• Is the institutional support appropriate for the proposed research?

Criterion 5. Patient-centeredness

The application should demonstrate that the study focuses on improving patient-centered outcomes and employs a patient-centered research design (i.e., a design informed or endorsed by patients). (Note: The study can be patient-centered even if the end-user is not the patient, as long as patients will benefit from the information.)

The application should also address the following questions:

• Does the application include a thorough description about which outcomes (both benefits and harms) are important to patients, and are those outcomes included in the study plan?
• Does the application provide information that indicates that closing the evidence gap is important to patients and other stakeholders?
• Are the interventions being compared in the study available to patients now, and are they the best options for comparison (including whether patients and their healthcare providers would choose them for managing the condition being studied)?

Criterion 6. Patient and stakeholder engagement

The application should demonstrate the engagement of relevant patients and other stakeholders (e.g., patients, caregivers, clinicians, policy makers, hospitals and health systems, payers [insurance], purchasers [business], industry, researchers, training institutions, and non-PCORI funding partner[s]) in the conduct of the study. Quality of engagement should be evaluated based on scope, form, and frequency of patient and stakeholder involvement throughout the research process.

The application should also address the following questions:

• Does the application provide a well-justified description of how the research team incorporates stakeholder involvement? Does the study include the right individuals (e.g., researchers, patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders) to ensure that the projects will be carried out successfully?
• Does the application show evidence of active engagement among scientists, patients, and other stakeholders throughout the research process (e.g., formulating questions, identifying outcomes, monitoring the study, disseminating, and implementing)? Is the frequency and level of patient and stakeholder involvement sufficient to support the study goals?
• Is the proposed Engagement Plan appropriate and tailored to the study?
• Are the roles and the decision-making authority of all study partners described clearly?
• Are the organizational structure and resources appropriate to engage patients and stakeholders throughout the project?

In-Person Review

During preliminary review, PCORI evaluates and scores all administratively and scientifically compliant applications based on PCORI’s merit review criteria, including evaluating adherence to the PCORI Methodology Standards. After PCORI completes the preliminary review, PCORI program staff members evaluate panel scores and critiques to identify a subset of applications for merit reviewers to discuss at the in-person review meeting. Not all submitted applications move forward to in-person review.

During the in-person review, merit reviewers meet to discuss applications and to clarify the merits of the proposed research. They also identify areas for improvement. Each application is re-scored based on the content of discussion. The Panel Chair and PCORI MRO lead the in-person panel meeting and ensure that all applications receive a fair and thorough review according to the standards outlined in the PFA.

Post-Panel Review

After the in-person meeting, PCORI program staff evaluate final merit review panel scores and comments, identify duplication or synergy among funded projects, and consider the fit of applications within the programmatic vision. Program staff members then recommend projects to a Selection Committee, which includes members of the Board. The Selection Committee considers recommendations and works with staff to identify a slate of applications for possible funding based on merit review scores, programmatic balance and fit, and PCORI’s strategic priorities. This slate then goes to the Board for consideration and approval.

In addition, PCORI evaluates applicant risk before issuing an award. Factors considered include financial stability, quality of management systems, audit findings, and past performance on PCORI awards (e.g., compliance with PCORI reporting requirements, conformance to PCORI terms and conditions on previous awards, and timely achievement of milestones). Based on the risk assessment, PCORI may impose special terms and conditions on awardees or withhold contract issuance until such business risks are mitigated. PCORI will not award new contracts to current awardees with overdue reports (progress, interim, final, etc.) until the awardees have submitted the overdue reports.

Summary Statements and Funding Recommendations

Applicants receive summary statements approximately two weeks before PCORI announces the funding decisions. If an application progresses to in-person discussion, the applicant will receive a summary statement that includes:

• In-person panel discussion notes
• Final average overall score
• Preliminary reviewer critiques

Summary statements for applications that do not progress to in-person discussion include only the preliminary reviewer critiques.

PCORI makes funding recommendations by identifying meritorious applications that fit the programmatic needs and satisfactorily address the merit review criteria while adhering to the PCORI Methodology Standards. PCORI also considers the funds allotted for the current PFA when deciding which applications to recommend to the Board for approval. Applicants to this current cycle’s PFA will receive summary statements and notification of the funding status of their application no later than May 2018.

Contract Execution and Activation

PCORI will issue a contract to the selected Awardee Institutions for the study once it conducts a thorough programmatic and administrative review. The awardees must accept PCORI’s contract terms and conditions, which will be based on PCORI’s research funding contract terms and conditions, with additional provisions appropriate for the use of the PCORnet infrastructure and the specific research project. Among the expected contractual terms is a fully agreed-upon study plan as evaluated by PCORI. The study will commence only after PCORI and the Awardee Institution execute the applicable contract and agree on the final research project plan.

Back to top