Broad PCORI Funding Announcements - Cycle 1 2017

I. Requirements for PCORI Research

This section includes language that is specific to PCORI’s requirements for applications for funding. Applicants should use this section as guidance when preparing their applications.

Research Priorities

To be considered responsive, applications must:

• Describe comparators. Regardless of the approach being studied, all proposed research projects must compare at least two alternatives. If the applicant proposes “usual care” as a rational and important comparator in the proposed study, then it must be described in detail, coherent as a clinical alternative, and properly justified as a legitimate comparator (e.g., “usual care” is guidelines-based). It must also be accompanied by an explanation of how the care given in the “usual care” group will be measured in each patient, and how appropriate inferences will be drawn from its inclusion. “Usual care” must be described as mentioned above to ensure that it accounts for geographic and temporal variations, and it has wide interpretability, applicability, and reproducibility.
• Describe research that compares two or more alternatives, each of which has established efficacy. PCORI expects the efficacy or effectiveness of each intervention to be known. If the efficacy or evidence base is insufficient, then data need to be provided to document that the intervention is used widely. The application must provide information about the efficacy of the interventions that will be compared; pilot data might be appropriate. Projects aiming to develop new interventions that lack evidence of efficacy or effectiveness will be considered out of scope.
• Describe research that studies the benefits and harms of interventions and strategies delivered in real-world settings. PCORI is interested in studies that provide practical information that can help patients and other stakeholders make informed decisions about their health care and health outcomes.
• Describe consultation with patients and other stakeholders about how the study is answering a critical question. Explain the pertinent evidence gaps and why the project questions represent decisional dilemmas for patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders. Describe why project outcomes are especially relevant and meaningful endpoints to patients and other stakeholders.

Leveraging Existing Resources

PCORI encourages investigators to propose studies that leverage existing resources, such as adding patient-centered outcomes research (PCOR) to an existing large clinical trial or analyzing existing large databases that contain valuable, relevant information that may be used to answer important CER questions.

Patient-Centered Outcome Measures

PCORI encourages investigators to design their research using validated outcome measures. Include preliminary data that support using the proposed measures in the study population. We encourage investigators to consider those measures described in the Patient-Reported Outcomes Measurement Information System (PROMIS).

Studies in Rare Diseases

PCORI is interested in the investigation of strategies addressing care for patients with rare diseases. These conditions are defined as “life-threatening” or “chronically debilitating.” They are of such low prevalence (affecting fewer than 200,000 in the United States [i.e., less than 1 in 1,500 persons]) that special efforts—such as combining data across large populations—might be needed to address them.

Studies of Cost-Effectiveness

PCORI will consider an application nonresponsive if the proposed research:

• Conducts a formal cost-effectiveness analysis of alternative approaches to providing care
• Directly compares the costs of care between two or more alternative approaches to providing care

Proposals that include studies of these issues may measure and report utilization of any or all health services, but may not employ direct measurements of care costs. For further information, please reference our cost-effectiveness analysis FAQs.

PCORI does have an interest, however, in studies addressing questions about conditions leading to high costs to the individual or to society. This is included in our review criterion on the condition’s impact on the health of individuals and populations. Thus, PCORI is interested in studies that:

• Examine the effect of costs on patients, such as patients’ out-of-pocket costs, hardship or lost opportunity, or costs as a determinant of or barrier to access to care
• Address cost-related issues, such as the resources needed to replicate or disseminate a successful intervention
• Evaluate interventions to reduce health system waste or increase health system efficiency

Addressing specifically the issue of conditions that lead to high costs, our PCORI Funding Announcements (PFAs) say that “proposals that include studies of these issues without utilizing a formal cost-effectiveness analysis or directly measuring and comparing costs of care alternatives will be considered responsive and will be reviewed.”

Categories of Nonresponsiveness

PCORI discourages proposals in the following categories, and will likely deem them nonresponsive:

• Instrument development, such as new surveys, scales, etc.
• Developing, testing and validating new decision aids and tools, or clinical prognostication tools
• Pilot studies intended to inform larger efforts
• Comparing patient characteristics rather than clinical strategy options
• Applies to Assessment of Prevention, Diagnosis, and Treatment Options (APDTO) and Improving Healthcare Systems (IHS) program applicants ONLY: Comparing interventions for which the primary focus is the role of community health workers or patient navigators

Consistent with PCORI's authorizing law, PCORI does not fund research whose findings will include:

• Coverage recommendations
• Payment or policy recommendations
• Creation of clinical practice guidelines or clinical pathways
• Establishment of efficacy for a new clinical strategy
• Pharmacodynamics
• Study of the natural history of disease
• Basic science or the study of biological mechanisms

Avoiding Redundancy

PCORI encourages potential applicants to review funded research at pcori.org. We intend to balance our funded portfolio to achieve synergy and avoid redundancy where possible.

Methodological Considerations

Regardless of study design, proposals must adhere to all relevant PCORI Methodology Standards. These include 47 individual standards that fall into 11 categories. The first five categories are cross-cutting and relevant to most PCOR studies. Researchers should refer to these standards when planning and conducting their research projects. These cross-cutting categories are:

1. Standards for Formulating Research Questions
2. Standards Associated with Patient-Centeredness
3. Standards on Data Integrity and Rigorous Analyses
4. Standards for Preventing and Handling Missing Data
5. Standards for Heterogeneity of Treatment Effect (HTE)

Six other standards categories will be applicable to certain study designs and methods. The standards in each of these categories should be used as guidance when they are relevant to a study. These categories are:

1. Standards for Data Registries
2. Standards for Data Networks as Research-facilitating Infrastructures
3. Standards for Causal Inference Methods
4. Standards for Adaptive and Bayesian Trial Designs
5. Standards for Studies of Diagnostic Tests 
6. Standards for Systematic Reviews

Most of these standards are minimal. The PCORI Methodology Standards reflect practices that should be followed in all cases, and all deviations need to be explained and well-justified. Additional best practices—including accepted guidelines for conducting clinical trials or observational studies—should be addressed, if applicable, in the PCORI funding application.

Applicants should discuss how the planned study design will measure and adjust for potential confounding factors that may obscure or artificially create differences attributable to the alternatives being compared. Examples include, but are not limited to, baseline differences in disease severity or other risk factors within the study population, as well as differences in participation, adherence, or follow-up that might affect outcomes independent of the interventions being compared.

Upcoming New and Revised Methodology Standards

In 2015 the Methodology Committee undertook a process to review the existing PCORI Methodology Standards, updating and adding new standards where indicated. Although these proposed revised and new standards are still under review and have not yet benefited from public comment, we encourage you to refer to the potential revisions. Applicants should continue to adhere to the current PCORI Methodology Standards.

Patient and Stakeholder Engagement

PCORI encourages all applicants to outline how patients and other stakeholders will participate as partners in various phases of the proposed research. Before completing this section of the Research Strategy, applicants are encouraged to review the Engagement Rubric, which can be found in the Templates, Checklists, and More section. Applicants should also review the PCORI Methodology Standards Associated with Patient-Centeredness and PCORI’s Sample Engagement Plans. The rubric and Sample Engagement Plans are not intended to be comprehensive or prescriptive; instead, they provide a variety of examples to incorporate engagement, where relevant, into the research process.

Applicants are expected to consult with patients and other stakeholders on their decisional dilemma and evidence needs, or to reference previously documented decisional dilemmas in preparation for the submission of Letters of Intent (LOIs) and applications. To describe the decisional dilemma, state the specific clinical decision(s) or treatment choice(s) confronted by the decision makers and explain how the findings from the proposed research will inform those decisions. State why this decision—such as choosing a specific medication, surgical approach, or care delivery strategy to treat a condition or manage a specific population—is important to patients. Document the uncertainty patients and other stakeholders face in making this decision. Identify the patients and other stakeholders you consulted in determining that the proposed study addresses their evidentiary needs for decision making, and indicate your commitment to continue engaging them actively in the conduct of the study. Similarly, applicants should document how the project outcomes are especially relevant and meaningful endpoints to patients and other stakeholders.

Populations Studied

PCORI seeks to fund research that includes diverse populations with respect to age, gender, race, ethnicity, geography, or clinical status, so that possible differences in outcomes may be examined in defined subpopulations, otherwise known as HTE. PCORI recognizes that some proposed studies might represent important PCOR opportunities, even in the absence of a broadly diverse study population. However, the burden is on the applicant to justify the study’s importance in the absence of diversity; to discuss which subgroups are most important; and to discuss how the subgroups will be analyzed, including whether or not the study will be powered to examine the question of effectiveness in subgroups. PCORI is particularly interested in including previously understudied populations for whom effectiveness information is especially needed, such as hard-to-reach populations or patients with multiple conditions. Thus, comparisons should examine the impact of the strategies in various subpopulations, with attention to the possibility that the strategy’s effects might differ across subpopulations. PCORI has developed the following list of populations of interest to guide our efforts in research and engagement. (Note that the Addressing Disparities Program requires that proposed research focus on at least one of the groups indicated by an asterisk below.)

• Racial and ethnic minority groups*
• Low-income groups*
• Women
• Children (age 0–17 years)
• Older adults (age 65 years and older)
• Residents of rural areas*
• Individuals with special healthcare needs, including individuals with disabilities*
• Individuals with multiple chronic diseases
• Individuals with rare diseases
• Individuals whose genetic makeup affects their medical outcomes 
• Patients with low health literacy, numeracy, or limited English proficiency*
• LGBT persons*
• Veterans and members of the Armed Forces and their families

Protection of Human Subjects

This component (up to five pages) is included in the Research Plan Template. Describe the protection of human subjects involved in your proposed research. PCORI follows the Federal Policy for the Protection of Human Subjects (45 CFR part 46), including the Common Rule. For more detailed information, please see Section 5, titled “Human Subjects Research Policy,” in the Supplemental Grant Application Instructions for All Competing Applications and Progress Reports, which is issued by the U.S. Department of Health and Human Services. PCORI does not require that applicants comply with sections of this policy that refer to requirements for federal-wide assurance or that refer to standards for including women, minorities, and children. Awardees must also comply with appropriate state, local, and institutional regulations and guidelines pertaining to the use of human subjects in research.

PCORI requires awardees to ensure that there is a Data and Safety Monitoring Plan, which may include the need to appoint a Data and Safety Monitoring Board, as provided in the PCORI Policy on Data and Safety Monitoring Plans for PCORI-Funded Research.

PCORI merit reviewers will examine plans for protection of human subjects in all applications and may provide comments regarding the plans (see How To Evaluate Human Subjects Protections). Reviewers’ comments on human subject research are not reflected in the overall application score, but PCORI staff might use them during potential funding negotiations. Final determinations about the adequacy of human subject protections rest with the Institutional Review Board or international equivalent that have jurisdiction for the study.

The Awardee Institution, whether domestic or foreign, bears ultimate responsibility for safeguarding the rights and welfare of human subjects in PCORI-supported activities.

Required Education of Key Personnel on the Protection of Human Subject Participants

PCORI requires that all applicants adhere to the National Institutes of Health (NIH) policy on education in the protection of human subject participants in the conduct of research. This applies to all individuals listed as key personnel in the application. The policy and FAQs are available on the NIH website.

Data Management and Data-Sharing Plan

PCORI encourages openness in research and making research data available for purposes of replication and reproducibility. Although not required to be submitted as a component of the research application, if an award is made, the awardee is required to develop and maintain a plan that addresses data management and data sharing of research project data in a manner that is appropriate for the nature of the research project and the types of research project data, and that is consistent with applicable privacy, confidentiality, and other legal requirements.

Recruitment

Proposals should include information about the size and representativeness of the potential pool of patients from which recruitment will occur, and describe the means by which this size estimate was determined. Likewise, proposals should provide evidence-based estimates of how many participants are expected in the study, based on expected recruitment; applying the study’s inclusion and exclusion criteria; anticipated acceptance (or refusal) rates; and other factors, such as failure to follow up. Such estimates must be discussed in the application, specified in the milestones, reviewed by merit reviewers and PCORI staff, and monitored by PCORI in the funded research.

Peer Review and Release of Research Findings

PCORI has a legislative mandate to ensure the scientific integrity of the primary research it supports and to make study findings widely available and useful to patients, clinicians, and the general public within a specific timeframe. Accordingly, the PCORI Board of Governors (Board) adopted the Process for Peer Review of Primary Research and Public Release of Research Findings.

In summary, Awardee Institutions are required to submit to PCORI for peer review a draft final research report that provides the methodological details, describes the main study results, and interprets the findings in clinical or other decisional contexts. Subject matter experts, individuals with expertise in research methodology or biostatistics, and patients, caregivers, and other healthcare stakeholders will review the draft final research report. After Awardee Institutions have responded to reviewers’ comments to PCORI’s satisfaction, the report will be accepted and considered final. PCORI will then prepare a 500-word abstract summarizing the study results for patients and the general public, which the Awardee Institution will review and approve.

PCORI will post the following materials on its website no later than 90 days after the draft final research report is accepted: (1) a 500-word abstract for medical professionals; (2) a standardized summary of the study results for patients and the general public; (3) a link to the study record on ClinicalTrials.gov (as applicable); and (4) ancillary information, including conflict of interest disclosures. The final research report, along with anonymized reviewer comments, will be made publicly available on the PCORI website no later than 12 months after its acceptance, except by prior mutual agreement with the Awardee Institution. For more information, please see Peer Review of Research Studies.

Back to Top

II. How to Submit an Application

Applying for funding from PCORI is a two-stage process. An LOI must be submitted and an applicant must be invited to submit an application.

Letter of Intent (LOI)

Applicants should download the LOI Template from the Templates, Checklists, and More section. They must complete the document and convert it to a PDF file. The LOI is limited to two pages, excluding references. PCORI suggests including all references as in-text citations using American Medical Association citation style, but other citation styles are accepted. Do not upload additional documents as part of your LOI, including Letters of Endorsement or Letters of Support, because they are not requested at this stage. Their inclusion will result in LOI rejection without review. Please visit the Templates, Checklists, and More section for additional applicant resources, including the PFA and required templates.

The LOI for the proposed study should contain the following information:

• Title of the proposed study that preferably captures the comparative nature of the study
• Specific aims (clearly stated)
• How the study will improve the quality and relevance of evidence available to help patients and stakeholders make informed health decisions
• Knowledge gap being addressed by research question(s)
• Concise description of study design, including:
  • Study population (description of participants and participating study sites)
  • Outcomes (identification and description of why they are important to patients)
  • Sample size
• Comparators (described and listed clearly, with demonstrated efficacy specified for each and details on how the strategies will be delivered in real-world settings)
• Patient and other stakeholder engagement (involvement through planning, conducting, and disseminating)

The LOI Template provides guidance on responding to each item. Please refer to the Application Guidelines for due dates and information on how to submit an LOI via PCORI Online. The deadline for LOI submission is February 14, 2017, by 5 p.m. (ET).

LOI Review

LOIs are evaluated based on the following:

• Importance and relevance of the topics to PCORI priorities, as evidenced by critical gaps identified by clinical guidelines developers and recent systematic reviews
• Clarity and credibility of responses to the LOI questions
• Programmatic fit and balance, considering whether the application overlaps with previously funded studies or concurrent applications to a significant degree or, conversely, whether the application fills a gap in the portfolio with certain characteristics, including disease category, topics, priority population, methodologies, and other variables

Only applicants whose LOIs are deemed most responsive to this PFA will be invited to submit a full application. A minimum of two PCORI staff review the LOIs, which are not scored during review. Notification of the request to submit a full application will occur no later than March 15, 2017.

Applicants are invited to submit an application based on the information provided in the LOI. Any changes to the following require PCORI's approval:

• Research question(s)
• Specific aims
• Study design
• Comparators
• Principal Investigator (PI)
• Institution

If you need to change any of this information or have questions, please email [email protected].

Note: A PI can only submit one LOI per PFA. However, an individual listed as a PI on one LOI may be listed as and serve in another non-PI role (e.g., co-investigator or consultant) on other LOIs within the same PFA, during the same cycle. A PI can submit multiple LOIs to different program PFAs in a cycle, but the PI must ensure that the research topics and projects are not similar. If a PI submits an LOI to multiple program PFAs, LOIs that exhibit scientific overlap or that appear to be duplicate submissions will be disqualified. PCORI will contact the PI and provide him or her with an opportunity to choose which PFA he or she would like to apply to. This applies to single and dual-PI submissions.

Project Budget and Duration

The maximum budget and research period of performance for this PFA can be found in the Overview section. At the time of contract execution, PCORI sets aside all of the funds associated with an awarded project to be made available throughout the contract’s period of performance. The maximum budget includes all research- and peer-review-related costs. Refer to the Application Guidelines for additional details. Appendix 2 within the guidelines provides a complete list of allowable and unallowable costs. This program does not consider exceptions to the budget or to period-of-performance limits. PCORI will not review requests exceeding the stated maximum budget or period of performance. Note that although subcontractor indirect costs are included in the prime applicant’s direct-cost budget, subcontractor indirect costs are not factored when determining adherence to the PFA’s direct-cost limit.

Submission Dates

LOIs and applications must be submitted in accordance with the published dates and times listed in the Key Information section of this document.

PCORI Online System

To submit an application, you must register with PCORI Online and submit an LOI and an application for each cycle to which you are applying.

Back to Top

III. Merit Review Criteria

PCORI’s merit review process is designed to support the following goals:

• Identify applications that have the strongest potential to help patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders make informed decisions to improve patient outcomes.
• Implement a transparent, fair, objective, and consistent process to identify these applications.
• Elicit high-quality feedback that reflects a diversity of perspectives to ensure that the PCORI-funded research reflects the interests and views of patients and other stakeholders and those who care for them, and that it meets the criteria for scientific rigor.
• Fund projects that fill important evidence gaps and have strong implementation potential.
• Regularly evaluate and continually improve the merit review process and policies in support of PCORI's mission.

PCORI merit review is a multiphase process that includes PFA development; staff evaluation of LOIs; the review panel’s preliminary review of full applications; an in-person panel discussion of a subset of full applications (identified by PCORI’s Research Priority Area Program staff and based on the preliminary review and program priorities); the Selection Committee’s recommendation of applications for funding; and, finally, Board award approval.

Preliminary Review

PCORI conducts rigorous merit review of the full applications it receives. Note that PCORI may eliminate applications from the review process for administrative or scientific reasons (e.g., nonresponsiveness). An application may be administratively withdrawn if it is incomplete; submitted past the stated due date and time; or does not meet the formatting criteria outlined in the Application Guidelines, in the PCORI templates, and in PCORI Online. An application can be scientifically withdrawn if it is not responsive to the guidelines described in this PFA, describes research that is not comparative, includes a cost-effectiveness analysis, or otherwise does not meet PCORI programmatic requirements.

PCORI Merit Review Officers (MROs) recruit each panel based on the number of and topic areas represented by invited LOIs. MROs recruit the panel chair, scientist reviewers who are subject matter experts, patient representatives, and representatives of other stakeholder groups. All panel members receive training during the review cycle to ensure that they understand the programmatic and organizational goals of review.

The table below is designed to help applicants understand how the PCORI merit review criteria align with criteria from other funding organizations with which applicants might be familiar (e.g., NIH). Though PCORI’s criteria do map to most NIH criteria, there are areas where we ask for different information (i.e., PCORI does not include a criterion that tracks to NIH’s innovation criterion, but does include criteria evaluating patient-centeredness and engagement) reflecting PCORI’s unique approach.

Crosswalk of PCORI Merit Review Criteria with NIH Criteria
SIGNIFICANCE	Potential for the study to fill critical gaps in evidence Potential for the study findings to be adopted into clinical practice and improve delivery of care
APPROACH	Scientific merit (research design, analysis, and outcomes) Investigator(s) and environment
PCORI-only Merit Review Criteria
PATIENT-CENTEREDNESS/ENGAGEMENT	Patient-centeredness Patient and stakeholder engagement

Below are PCORI’s merit review criteria. PCORI’s merit review panels use these criteria during the preliminary and in-person review phases to evaluate and score all submitted applications, and to ensure consistency and fairness in how applications are evaluated.

Criterion 1. Potential for the study to fill critical gaps in evidence:

The application should address the following questions:

• Does the application convincingly describe the clinical burden?
• Does the application identify a critical gap in current knowledge as noted in systematic reviews, guideline development efforts, or previous research prioritizations?
• Does the application identify a critical gap in current knowledge, evidenced by inconsistency in clinical practice and decision making?
• Would research findings from the study have the potential to fill these evidence gaps?

Criterion 2. Potential for the study findings to be adopted into clinical practice and improve delivery of care

The application should describe how evidence generated from this study could be adopted into clinical practice and delivery of care by others. The application should also address the following questions:

• Does the application identify who will make the decision (i.e., the decision maker) or use (i.e., the end-user) the study findings (not the intervention) this study produces, such as local and national stakeholders?
• Does the application identify potential end-users of study findings—such as local and national stakeholders—and describe strategies to engage these end-users?
• Does the application provide information that supports a demand for this kind of a study from end-users?
• Would this study’s research findings have the potential to inform decision making for key stakeholders? If so, provide an example. How likely is it that positive findings could be reproduced by others, resulting in improvements in practice and patient outcomes? Identify the potential barriers that could hinder adoption of the intervention by others.
• Does the application describe a plan for how study findings will be disseminated beyond publication in peer-review journals and at national conferences?

Criterion 3. Scientific merit (research design, analysis, and outcomes)

The application should show sufficient technical merit in the research design to ensure that the study goals will be met. The application should also address the following questions:

• Does the application describe a clear conceptual framework anchored in background literature which informs the design, key variables, and relationship between interventions and outcomes being tested?
• Does the Research Plan describe rigorous methods that demonstrate adherence to the PCORI Methodology Standards?
• Is the overall study design justified?
• Are the patient population and study setting appropriate for the proposed research question?
• Does the application provide justification that the outcome measures are validated and appropriate for the population?
• Are each of the comparators (e.g., active intervention arm and comparator arm) described clearly and well-justified? If “usual care” is one of the arms, is it adequately justified and will it be sufficiently measured?
• Are the sample sizes and power estimates appropriate? Is the study design (e.g., cluster randomized design, randomized controlled trial, or observational study) accounted for and is the anticipated effect size adequately justified?
• Is the study plan feasible? Is the project timeline realistic, including specific scientific and engagement milestones? Is the strategy for recruiting participants feasible? Are assumptions about participant attrition realistic, and are plans to address patient or site attrition adequate?

Criterion 4. Investigator(s) and environment

This criterion should assess the appropriateness (e.g., qualifications and experience) of the investigator(s)/team and the environment’s capacity (e.g., resources, facilities, and equipment) to support the proposed project. It should not be an assessment of the institution’s quality.

The application should also address the following questions:

• How well-qualified are the PIs, collaborators, and other researchers to conduct the proposed activities? Is there evidence of sufficient clinical or statistical expertise (if applicable)?
• Does the investigator or co-investigator have demonstrated experience conducting projects of a similar size, scope, and complexity?
• If the project is collaborative or dual-PI, do the investigators have complementary and integrated expertise? Are the leadership, governance, and organizational structures appropriate for the project?
  • (Dual-PI Option Only) Does the Leadership Plan adequately describe and justify PI roles and areas of responsibility?
• Is the level of effort for each team member appropriate for successfully conducting the proposed work?
• Does the application describe adequate availability of and access to facilities and resources (including patient populations, samples, and collaborative arrangements) to carry out the proposed research?
• Is the institutional support appropriate for the proposed research?

Criterion 5. Patient-centeredness

The application should demonstrate that the study focuses on improving patient-centered outcomes and employs a patient-centered research design (i.e., a design informed or endorsed by patients). (Note: The study can be patient-centered even if the end-user is not the patient, as long as patients will benefit from the information.)

The application should also address the following questions:

• Does the application include a thorough description about which outcomes (both benefits and harms) are important to patients, and are those outcomes included in the study plan?
• Does the application provide information that indicates that closing the evidence gap is important to patients and other stakeholders?
• Are the interventions being compared in the study available to patients now, and are they the best options for comparison (including whether they would be chosen by patients and their healthcare providers for managing the condition being studied)?

Criterion 6. Patient and stakeholder engagement

The application should demonstrate the engagement of relevant patients and other stakeholders (e.g., patients, caregivers, clinicians, policy makers, hospitals and health systems, payers [insurance], purchasers [business], industry, researchers, and training institutions) in the conduct of the study. Quality of engagement should be evaluated based on scope, form, and frequency of patient and stakeholder involvement throughout the research process.

The application should also address the following questions:

• Does the application provide a well-justified description of how the research team incorporates stakeholder involvement? Does the study include the right individuals (e.g., researchers, patients, caregivers, clinicians, policy makers and other healthcare system stakeholders) to ensure that the projects will be carried out successfully?
• Does the application show evidence of active engagement among scientists, patients, and other stakeholders throughout the research process (e.g., formulating questions, identifying outcomes, monitoring the study, disseminating, and implementing)? Is the frequency and level of patient and stakeholder involvement sufficient to support the study goals?
• Is the proposed Engagement Plan appropriate and tailored to the study?
• Are the roles and the decision-making authority of all study partners described clearly?
• Are the organizational structure and resources appropriate to engage patients and stakeholders throughout the project?

In-Person Review

During preliminary review, all administratively and scientifically compliant applications are evaluated and scored based on PCORI’s merit review criteria, including evaluation of adherence to the PCORI Methodology Standards. After PCORI completes the preliminary review, PCORI program staff members evaluate panel scores and critiques to identify a subset of applications for merit reviewers to discuss at the in-person review meeting. Not all submitted applications move forward to in-person review.

During the in-person review, merit reviewers meet to discuss applications and to clarify further the merits of the proposed research. They also identify areas for improvement. Each application is re-scored based on the content of discussion. The Panel Chair and PCORI MRO lead the in-person panel meeting and ensure that all applications receive a fair and thorough review according to the standards outlined in the PFA.

Post-Panel Review

After the in-person meeting, PCORI program staff evaluate final merit review panel scores and comments, identify duplication or synergy among funded projects, and consider the fit of applications within the programmatic vision. Program staff members then recommend projects to a Selection Committee, which includes members of the Board. The Selection Committee considers recommendations and works with staff to identify a slate of applications for possible funding based on merit review scores, programmatic balance and fit, and PCORI’s strategic priorities. This slate is then proposed to the Board for consideration and approval.

In addition, PCORI evaluates applicant risk before issuing a PCORI award. Factors considered include financial stability, quality of management systems, audit findings, and past performance on PCORI awards (e.g., compliance with PCORI reporting requirements, conformance to PCORI terms and conditions on previous awards, and timely achievement of milestones). Based on the risk assessment, PCORI may impose special terms and conditions on awardees or withhold contract issuance until such business risks are mitigated. PCORI will not award new contracts to current awardees with overdue reports (progress, interim, final, etc.) until the overdue reports have been submitted to PCORI. 

Summary Statements and Funding Recommendations

Summary statements are provided to applicants approximately two weeks before funding decisions are announced. If an application progresses to in-person discussion, the applicant will receive a summary statement inclusive of:

• In-person panel discussion notes
• Final average overall score
• Preliminary reviewer critiques
• Application quartile, which provides information for applicants to understand how they did relative to other discussed applications

Summary statements for applications that do not progress to in-person discussion include only the preliminary reviewer critiques.

Funding recommendations are made by identifying meritorious applications that fit the programmatic needs and that address the merit review criteria satisfactorily while adhering to the PCORI Methodology Standards. Programs also consider the funds allotted for the current funding announcement when deciding which applications to recommend to the Board for approval. Applicants to this current cycle’s PFA will receive summary statements in November 2017 and notification of the funding status of their application no later than November 2017.

Back to Top

IV. Templates, Checklists, and more

• These documents are no longer available.

Back to Top