Eligibility

Evidence Readiness: Applicants must propose a feasible and logical next step for disseminating and/or implementing a clinically meaningful finding or findings associated with a PCORI-funded CER study. Only projects proposing to disseminate or implement findings from PCORI CER studies that a) tested a research hypothesis and b) evaluated comparative clinical effectiveness of two or more comparators will be responsive to this announcement. (Note: These requirements do not apply to methods studies.)

PCORI Research Awardees: Recipients of PCORI research awards, including Broad PCORI Awards, Pilot Projects Program Awards, Targeted PCORI Awards, Pragmatic Clinical Study Awards, and PCORI-funded demonstration projects occurring within the National Patient-Centered Clinical Research Network (PCORnet) infrastructure (e.g. ADAPTABLE and obesity trials), are eligible to respond to this announcement. Applications associated solely with Eugene Washington PCORI Engagement Awards and Pipeline to Proposal Awards are not eligible for this limited PFA. NOTE: Although eligible to apply, Pilot Projects must still meet all of the PFA requirements in order to be considered responsive. Some of the requirements (e.g., evidence readiness) may be difficult for Pilot projects to satisfy.

Organization: Applications may be submitted by a private-sector research organization, including any nonprofit or for-profit organization, and any public-sector research organizations, including any university or college hospital or healthcare system; any laboratory or manufacturer; or any unit of local, state, or federal government. The Internal Revenue Service must recognize all U.S. applicant organizations. Nondomestic components of organizations based in the United States and foreign organizations may apply, as long as there is demonstrable benefit to the U.S. healthcare system and U.S. efforts in the area of patient-centered research can be shown clearly. Organizations may submit multiple funding applications. Individuals are not permitted to apply.

Personnel: A Letter of Support is required from the Principal Investigator (PI) of the original PCORI-funded research project if he or she is not the proposed PI of the current application.

Timing: Applicants are required to submit the draft final research report (DFRR) pertaining to their PCORI research award before submitting a full application to this announcement. Applicants may submit the mandatory LOI before submitting their DFRR. Applications submitted prior to PCORI’s receipt of the DFRR, or for which the DFRR is judged to be incomplete or otherwise inadequate by PCORI, will be administratively withdrawn. Applicants are encouraged to submit their DFRR 1-2 months before the application deadline for this PFA, so that PCORI can perform a preliminary review to determine whether the DFRR is acceptable for submission to Peer Review. PCORI will provide comments within two weeks of receiving the DFRR to provide the PI an opportunity to address these comments before the PFA’s application deadline.

After submitting their PCORI DFRR, applicants will be eligible to submit an application in response to this limited PFA for two years. During this period of eligibility, applicants will be allowed one opportunity to resubmit an application that was reviewed and not funded in a previous cycle. See the Resubmission Policy (page 3) for more detail.

Review Criteria

Please note that the merit review criteria for this announcement are different from those PCORI uses when reviewing research applications.

1. Importance of research results in the context of the existing body of evidence
2. Readiness of the research results for dissemination
3. Technical merit of the proposed dissemination or implementation project
4. Project personnel and environment
5. Patient-centeredness
6. Patient and stakeholder engagement

I. Introduction

Summary of Program

The Patient-Centered Outcomes Research Institute (PCORI) is launching this funding initiative to support the investigator-initiated dissemination and implementation (D&I) of patient-centered, comparative clinical effectiveness research (CER) findings obtained from PCORI-funded studies. This opportunity will give PCORI investigators the chance, following the generation of results from their PCORI research award, to propose the next step(s) for making their research results more useful, actionable, and accessible to targeted end-users. The overarching objective of this limited PCORI Funding Announcement (PFA) is to promote and facilitate the effective and timely use of research evidence in the real world. As such, PCORI seeks to fund projects that do the following:

• Propose a logical and feasible next step for D&I of clinically meaningful CER finding(s) in order to improve CER finding accessibility, usefulness, uptake, and/or impact among targeted end-users. Note: For the purposes of this limited PFA, a research finding includes any tools or other products that may have been developed or refined as part of the original PCORI study.
• Propose to disseminate or implement findings from PCORI CER studies that a) tested a research hypothesis and b) evaluated the comparative clinical effectiveness of two or more comparators. (Note: These requirements do not apply to methods studies.)
• Target end-users who can benefit directly from the use of the evidence that is the focus of the project. Alternatively, target end-users who are a critical link to achieving changes in health outcomes or health care.
• Clearly describe key PCORI research study findings as they relate to the research questions, specific aims, and hypotheses of the study, supported by the required tables and figures. Investigators should discuss how the study findings relate to the body of evidence in the existing literature and current state of knowledge on the particular questions. Further, the investigator should identify the decision-making context in which the research findings are relevant, beyond the study setting. The findings must be applicable and generalizable to this context.
• Ensure that the proposed D&I plan for advancing use of the evidence in practice is informed and guided by an established conceptual model or framework.
• Clearly describe how the perspectives and experiences of patients and stakeholders, including individuals living with the disease or condition of interest, have been incorporated into the project plan. Also, describe how the host delivery systems and settings in which D&I is planned have been included, and demonstrate the commitment and involvement of those required to accomplish the project successfully.
• Propose an evaluation plan focusing on an appropriate balance of measurable process and short-, intermediate-, and longer-term outcomes to capture timely information on the effectiveness of the proposed D&I activities.
• Address the potential of the proposed project to inform future D&I efforts, leading to broader uptake of the PCORI findings.

Background

U.S. healthcare organizations and agencies in the public and private sectors spend billions of dollars on research and service delivery programs each year, yet patients and stakeholders often lack sufficient information to make decisions regarding the most effective treatment strategies for their particular circumstances.[1] The gap between what we know can optimize healthcare delivery and what is actually implemented in everyday practice remains one of the most important issues hindering the healthcare system and public health.[2][3] Finding ways to enhance awareness and knowledge of useful and relevant information to help people and organizations make decisions (dissemination) and put them into practice (implementation) is essential to improving health care and health outcomes.

The concepts of dissemination and implementation are sometimes used interchangeably to characterize the complete process of bringing evidence into practice. For the purposes of this limited PFA, we make the following distinction between dissemination and implementation:

• Dissemination is the intentional, active process of identifying target audiences and tailoring communication strategies to increase awareness and understanding of evidence and to motivate its use in policy, practice, and individual choices.[4]

The purpose of dissemination is to spread and sustain knowledge and the associated evidence-based interventions. Consistent with this definition of dissemination as an active process, this limited PFA will not support projects primarily dependent on passive dissemination strategies (sometimes also called research diffusion) such as untargeted mass mailings, publication of study findings, and untargeted presentations to heterogeneous groups.[5]

• Implementation is the deliberate, iterative process of integrating evidence into policy and practice by adapting evidence to different contexts and facilitating behavior change and decision making based on evidence across individuals, communities, and healthcare systems.[4]

Dissemination and implementation share the ultimate goal of encouraging the use of evidence in individual decision making, policy, and practice. These processes involve stakeholder engagement and partnerships with people and organizations, and are enhanced through ongoing evaluation. For the current announcement, successful applications will propose context-appropriate strategies that address how research results are transmitted, interpreted, and delivered to diverse patient and stakeholder groups.

Potential Approaches

PCORI is seeking projects that propose feasible and well-informed strategies for actively disseminating and implementing patient-centered outcomes research (PCOR) results, with the aim of increasing their accessibility, usefulness, uptake, and/or impact among targeted end-users. Specific strategies proposed for disseminating and implementing results of PCORI-funded research projects will vary based on a host of factors, including the finding being disseminated, the population(s) being targeted, and the goals of the D&I effort. PCORI encourages applicants to work closely with relevant stakeholder groups to identify appropriate D&I strategies. Examples of appropriate projects include, among others, efforts to:

• Develop, demonstrate, and evaluate approaches for incorporating results of PCORI-funded research into decision-making settings for patients, providers, policy makers, and other stakeholders.
• Adapt the content, format, or vehicle for delivering CER research evidence to improve its penetration and use at the policy, health systems, clinical practice, caregiver, and patient levels.
• Demonstrate the capacity and ability to take research results found effective through PCORI research studies “to scale” in diverse settings and populations.
• De-implement or reduce the use of interventions that are not evidence based, have been widely adopted prematurely, or are harmful or wasteful.

All proposed D&I strategies must actively disseminate/implement findings to targeted end-users and evaluate the success of the D&I strategy. Applications proposing to translate or adapt a finding or product without a plan to actively disseminate or implement it will not be considered responsive.

Collaborations

Applicants may consider proposing projects involving collaboration to disseminate the results of multiple related PCORI-funded research studies. Collaborative D&I projects may take different forms. At minimum, a collaborative project must involve the partnership of two or more PCORI-funded investigators partnering to disseminate the collective results of two or more PCORI-funded research studies that address a single or closely related condition, population, decision dilemma, or evidence gap. PCORI welcomes collaborative applications and encourages use of the Greater Than Budget/Time Request mechanism (see “Funds Available” below) to request funds or time greater than the limit specified in this announcement that are appropriate and commensurate to the scope of the proposed project. Collaborative projects must have demonstrated support from the Principal Investigator (PI) of each PCORI study whose findings are being disseminated in the collaborative D&I project. Please note: Participation in a collaborative project does not preclude individual investigators from submitting a separate, individual D&I application through this mechanism. However, investigators will be expected to provide a strong justification that their individual D&I projects do not duplicate activities proposed in the collaborative D&I project.

Resubmissions

After submitting their PCORI draft final research report, applicants will be eligible for two years to submit an application in response to this limited PFA. During this period of eligibility, applicants will be allowed one opportunity to resubmit an application that completed the merit review process (i.e., the applicant received a summary statement) and was not funded in a previous cycle. Applicants may not resubmit an application for a previously submitted and reviewed application until they have received merit review feedback (summary statement) from the initial submission. All resubmitted applications require submission of a new LOI that must be administratively and programmatically responsive to the PFA. Resubmitted applications must include a Resubmission Letter. Applications that do not meet these requirements will be deemed nonresponsive and withdrawn from merit review. LOIs that are not invited to submit a full application by PCORI do not count as a submission or resubmission.

Funds Available

PCORI has devoted up to $9 million in total annual costs under this limited PFA. The total amount awarded and the number of awards made will depend upon the quality, duration, and costs of the applications received. The maximum proposed budget for individual studies is $350,000 in direct costs, and the maximum project period is up to two years.

PCORI is providing applicants who anticipate the need for funds greater than $350,000 direct costs and/or time greater than two years the opportunity to request a higher level of funding and/or a longer project period through this limited PFA. Applicants should indicate their intention to request additional funds or time, as well as a brief justification for why the additional funds and/or time are needed, when submitting the LOI. If PCORI is willing to consider funds greater than $350,000 direct costs and/or a project period of more than two years for the project proposed in the LOI and if the applicant is invited to submit a full application, the applicant will be required to complete the Greater Than Budget/Time Request form with their application. The applicant will specify the additional funds and/or time required and justify the need for additional funds in this form. This justification should include a description of how these additional funds will increase the likelihood of impact on healthcare and/or health outcomes. Applicants submitting collaborative D&I applications are encouraged to take advantage of this Greater Than Budget/Time opportunity (see “Collaborations” above).

Back to top

II. Guidance for Preparing Applications

In developing a D&I strategy and corresponding PCORI application, applicants should pay attention to four fundamental concepts[6],[7],[8],[9],[10]: Evidence Context, Setting, Engagement, and Evaluation.

Evidence Context refers to the body of existing evidence relevant to the PCORI research finding. It is seldom the case that a single finding warrants D&I independent from other research findings. Applicants should explain how the current finding supports, augments, or differs from related evidence and how the proposed D&I plan takes account of the full body of related information.

Setting. The proposed project should consider carefully the setting in which the D&I will take place (e.g., health system, organization, community, etc.). The focus of investigators’ efforts should be the logical next D&I steps associated with PCORI-funded research results. At a minimum, the project should move findings out of a controlled research setting to a more general setting, demonstrating the ability of the proposed strategy to bring PCORI findings to the targeted end-users in ways that promote uptake.⁶ Project design should reflect relationships among the stakeholders within the setting, as these will be critical to the success of the project.

Engagement involves incorporating into project design the perspectives and experiences of interested patients and stakeholders, including individuals living with the disease or condition of interest, as well as the host delivery systems and settings in which applicants intend to disseminate and implement their work. D&I efforts will not succeed without the active involvement of those central to the D&I activities, including the targeted end-users. Stakeholders should be appropriately engaged both in planning and executing the D&I strategy, and particularly in ensuring that evidence and strategies are tailored appropriately to the setting. Plans for engaging stakeholders should reflect a spirit of partnership and reciprocity.

Evaluation is essential to understanding how and why D&I activities are or are not successful, as a basis for future replication, adjustment, or reconsideration of strategies. Evaluation planning should start at the beginning of the project and should include an appropriate balance of measurable process and short- and longer-term outcomes to provide timely information on both the execution of the project intervention and its impact. In selecting evaluation measures, applicants should also consider the appropriate balance among those that can capture impact on the dissemination setting (e.g. climate, culture, capacity, readiness), measures characterizing the dissemination process (e.g. reach, adoption), impacts on patients or intermediary targets of the activity (e.g. understanding, behavior change), and contextual factors (e.g. policy climate, incentive structures) associated with the proposed approach. In general, evaluation plans that exclusively reflect impact on setting characteristics and/or dissemination process outcomes will not be considered sufficient. The following resources may be helpful for identifying an appropriate evaluation strategy.[11]

Specific Requirements

Applications for proposed projects should meet the following requirements:

• Propose a logical and feasible next step for D&I of clinically meaningful CER finding(s) in order to improve CER finding accessibility, usefulness, uptake, and/or impact among targeted end-users. Note: For the purposes of this limited PFA, a research finding includes any tools or other products that may have been developed or refined as part of the original PCORI study.
• Propose to disseminate or implement findings from PCORI CER studies that a) tested a research hypothesis and b) evaluated comparative clinical effectiveness of two or more comparators. (Note: These requirements do not apply to methods studies.)
• Target end-users who can benefit directly from the use of the evidence that is the focus of the project. Alternatively, target end-users who are a critical link to achieving changes in health outcomes or health care.
• Clearly describe key PCORI research study findings as they relate to the research questions, specific aims, and hypotheses of the study, supported by the required tables and figures. Investigators should discuss how the study findings relate to the body of evidence in the existing literature and current state of knowledge on the particular questions. Further, the investigator should identify the decision-making context in which the research findings are relevant, beyond the study setting. The findings must be applicable and generalizable to this context.
• Ensure that the proposed D&I plan for advancing use of the evidence in practice is informed and guided by an established conceptual model or framework.
• Clearly describe how the perspectives and experiences of patients and stakeholders, including individuals living with the disease or condition of interest, have been incorporated into the project plan. Also, describe how the host delivery systems and settings in which D&I is planned have been included, and demonstrate the commitment and involvement of those required to accomplish the project successfully.
• Propose an evaluation plan focusing on an appropriate balance of measurable process and short-, intermediate-, and longer-term outcomes to capture timely information on the effectiveness of the proposed D&I activities.
• Describe the projected reach of this D&I initiative in terms of:
  • Absolute number: Total number of individuals you expect to reach with this D&I initiative
  • Proportion: Expected reach of this D&I initiative relative to the broader population of individuals who stand to benefit from the results being disseminated. 
  • Representativeness: Consideration of how those being targeted with this D&I initiative are similar or different from the broader population of individuals who stand to benefit from the results being disseminated. 
• Address the potential of the proposed project to inform future D&I efforts, leading to broader uptake of the PCORI findings.

Nonresponsiveness

LOIs and applications will be considered nonresponsive to this limited PFA and may be administratively withdrawn if the proposed project:

• Primarily conducts new research, as opposed to disseminating and implementing research findings obtained from already-completed PCORI-funded studies and evaluating the success of those D&I efforts. Projects proposing to perform CER are not of interest under this funding announcement, and will cause a LOI or application to be considered nonresponsive.
• Proposes to disseminate and/or implement findings that are not associated with a PCORI-funded CER or methods study.
• Proposes to translate and/or adapt a finding without actively disseminating or implementing it.
• Proposes to develop or validate a new tool or system for patients or clinicians without the primary purpose of actively disseminating or implementing evidence. Modification or adaptation of tools and systems previously found to be effective and proposed as the primary mechanism for actively disseminating and implementing evidence will be considered, as long as their development is not the primary activity proposed.
• Proposes a passive dissemination plan (publications or presentations to heterogeneous audiences) as its primary dissemination method.
• Proposes to use contract funds to pay for the cost of the interventions being disseminated in the project. Note: In general, PCORI does not pay for the cost of the interventions being disseminated in the projects it funds. Intervention costs include, but are not limited to, salary and time compensation for personnel that are delivering the intervention as well as equipment and other material costs associated with the delivery of the intervention. These are considered direct patient care costs and PCORI expects these costs to be covered by the healthcare delivery system or other interested payers. PCORI encourages all applicants to find support from sites, payers, stakeholders, etc. in the payment or cost-sharing of the interventions. Only under special circumstances will PCORI consider, as an exception, coverage of patient care intervention costs. If you are requesting the use of PCORI funds for any portion of these costs, your application must include a detailed justification, in the “Budget Justification Template,” outlining the clear importance of the request to the overall success of the project as well as future sustainability and implementation plans for once the project has completed (i.e. how will these costs be covered in the future, post PCORI funding, for implementation of the interventions in not only the sites participating in the study but also in other communities and health care settings).  Providing a justification, however, will not guarantee that the costs will be approved.

Protection of Human Subjects

If applicable, applicants should describe the protection of human subjects involved in their proposed research. If human subject protection is not applicable, applicants should provide a justification in the event that is not necessary for their project. PCORI follows the Federal Policy for the Protection of Human Subjects (45 CFR part 46), including the Common Rule. For more detailed information, please see Section 5 titled “Human Subjects Research Policy” in the Supplemental Grant Application Instructions for All Competing Applications and Progress Reports, which was issued by the U.S. Department of Health and Human Services. PCORI does not require that applicants comply with sections of this policy that refer to requirements for federal-wide assurance or that refer to standards for including women, minorities, and children. Awardees must also comply with appropriate state, local, and institutional regulations and guidelines pertaining to the use of human subjects in research.

PCORI merit reviewers will examine plans for protection of human subjects in all applications and may provide comments regarding the plans (see How To Evaluate Human Subjects Protections). Reviewers’ comments on human subject research are not reflected in the overall application score, but PCORI staff may use them during potential funding negotiations. Final determinations about adequacy of human subject protections rest with the Institutional Review Board (IRB) or IRBs that have jurisdiction for the study.

The Awardee Institution or organization, whether domestic or foreign, bears ultimate responsibility for safeguarding the rights and welfare of human subjects in PCORI-supported activities.

Required Education of Key Personnel on the Protection of Human Subject Participants

For those projects requiring human subject protection, PCORI requires that all applicants adhere to the National Institutes of Health (NIH) policy on education in the protection of human subject participants in the conduct of research. This applies to all individuals listed as key personnel in the application. The policy and FAQs are available on the NIH website.

Back to top

III. How To Submit an Application

Letter of Intent

Applicants should download the LOI Template for the Dissemination and Implementation limited PFA from the PCORI Funding Opportunities. The LOI has a three-page limit. References should be numbered in the text and full citations provided on a separate page following the LOI. PCORI suggests including all references as in-text citations using American Medical Association citation style, but other citation styles are accepted. Complete the document and convert it to a PDF file. Do not upload additional documents as part of your LOI, including Letters of Endorsement or Support, because they are not requested at this stage. Inclusion of additional documents will result in LOI rejection without review. Please visit the PCORI Funding Opportunities for additional applicant resources, including the PFA and required templates. LOIs are a mandatory prerequisite for submission of a full application. Applicants who fail to submit an LOI will not be permitted to submit a full application to the corresponding award cycle.

The LOI for the proposed project should contain the following information:

• Title of proposed project
• Original PCORI research award number, project title, and name of Principal Investigator (PI)
• Clear identification and description of the research results to be disseminated, including a clear description of the strength of these findings that justifies interest in disseminating them
• Identification of the evidence gap the research results fill, and the significance of the finding in the context of existing literature
• Objective and specific aims of the proposed D&I project
• Clear identification and description of the setting in which dissemination will take place, the immediate target of the dissemination activity, and the ultimate link to changes in healthcare delivery and health outcomes
• Clear description of the approach for disseminating the research finding
• Description of the evaluation plan, including measurable indicators of success
• Patient and other stakeholder engagement in the planning and implementation of this D&I effort

Please address all categories in the LOI Template and then upload the document into PCORI Online. The deadline for LOI submission is July 25, 2017, by 5 p.m. (ET).

Letter of Intent Review

An LOI is required in order to submit a full application. LOIs will be screened for responsiveness to this limited PFA and to ensure compliance with administrative guidelines. Only applicants whose LOIs are deemed most responsive to this limited PFA will be invited to submit a full application. Nonresponsive LOIs and those not adhering to Application Guidelines will not be invited to submit a full application. If an invited applicant indicates the intention to request additional funds and/or time with the LOI and if PCORI is willing to consider such a request, the applicant will be required to complete the Greater Than Budget/Time Request form with their application. Notification of denial or approval to submit an application will occur no later than August 22, 2017. Please refer to the Application Guidelines for information on how to submit your LOI via PCORI Online.

Submission Dates

LOIs and applications must be submitted in accordance with the published dates and times listed in the Overview section of this document and in the PCORI Funding Opportunities.

PCORI Online

To submit an application, you must register with PCORI Online and submit both an LOI and an application for each cycle to which you are applying. Please be sure that you are applying to the funding announcement titled, “Limited PCORI Funding Announcement: Dissemination and Implementation of PCORI-Funded Patient-Centered Outcomes Research Results.”

Back to top

IV. Applicant Resources

• D&I Application Guidelines (Cycle 2 2017)

Back to top

V. Merit Review

PCORI’s merit review process is designed to support the following goals:

• Identify applications that have the strongest potential to help patients, caregivers, clinicians, policy makers, and other healthcare system stakeholders make informed decisions to improve patient outcomes
• Implement a transparent, fair, objective, and consistent process to identify these applications
• Elicit high-quality feedback that reflects a diversity of perspectives to ensure that the PCORI-funded research reflects the interests and views of patients and other stakeholders and those who care for them, and that it meets the criteria for scientific rigor
• Fund projects that fill important evidence gaps and have strong D&I potential
• Regularly evaluate and continually improve the merit review process and policies in support of PCORI’s mission

PCORI merit review is a multiphase process that includes staff evaluation of LOIs; preliminary review of full applications by review panels; in-person panel discussion of a subset of full applications (based on the preliminary review and program priorities); and programmatic review and recommendation to the Office of the Chief Engagement and Dissemination Officer (OCEDO) for funding approval.

Preliminary Review

PCORI conducts rigorous merit review of the full applications it receives. Note that PCORI may eliminate applications from the review process for scientific reasons (e.g., nonresponsiveness) or for administrative reasons. An application may be administratively withdrawn if it is incomplete; is submitted past the stated due date and time; or does not meet the formatting criteria outlined in the Application Guidelines, in the PCORI templates, and in PCORI Online.

PCORI Merit Review Officers (MROs) recruit each merit review panel based on the number and type of topic areas represented by invited LOIs. MROs recruit the Panel Chair; scientist reviewers, who are subject matter experts; patient representatives; and representatives of other stakeholder groups. All panel members receive training during the review cycle to ensure that they understand the programmatic and organizational goals of review.

Application Review Criteria

Criterion 1. Importance of research results in the context of the existing body of evidence

• Does the application sufficiently identify and describe the original evidence gap that the PCORI-funded research addressed? Does the application sufficiently demonstrate that the evidence gap persists and is important?
• Does the application clearly describe the PCORI research results being proposed for dissemination?
• Does the application sufficiently discuss how the PCORI research results relate to the body of evidence in the existing literature? Specifically, does the application address the extent to which the PCORI results align with or contradict the existing evidence base?
• Will further uptake of these results, beginning with the project proposed, constitute a change in practice and lead to improved healthcare and health outcomes?

Criterion 2. Readiness of the research results for dissemination

• Are the PCORI results well-described, including their strength in terms of clinical and statistical significance?
• Does the application sufficiently address the generalizability of the PCORI research results to populations beyond the immediate study sample?
• Does the application identify the decision-making context in which the PCORI results are most relevant beyond the initial study setting?
• If the applicant is proposing to adapt an effective intervention, is the adaptation well justified? Does the applicant provide adequate justification that the adapted intervention will be ready for dissemination?

Criterion 3. Technical merit of the proposed dissemination or implementation project (project design, outcomes, and evaluation)

• Does the application sufficiently describe the group that will be the target of the proposed dissemination activity? Does it describe the setting in which the dissemination will take place? Are these targeted users and settings generalizable?
• Does the application use a D&I framework or model to inform the project design and evaluation outcomes and to address the potential longer-term implications of the proposed D&I project on clinical practice and patient-centered outcomes?
• Does the application provide a clear approach for disseminating the described research results?
  • Are the dissemination/implementation approaches appropriate for this D&I effort? Consider the extent to which they are tested, evidence-based, and/or consistent with principles and findings from implementation science.
• Does the application propose appropriate measures and describe the plan for evaluating success in sufficient detail, including an appropriate balance of measurable process and short-, intermediate-, and longer-term outcomes to capture timely information on the effectiveness of the proposed D&I activities?
• Does the application consider factors that may help or hinder the use of research results, including specific barriers to user implementation and how to mitigate them, within the context of the proposed project?
• Does the application address scalability, including a clear path for future efforts to bring these research results into wider use?
• Is the proposed timeline realistic, including specific project milestones?

Criterion 4. Project personnel and environment

This criterion should assess the appropriateness (e.g., qualifications and experience) of the project personnel/team and capacity of the environment (e.g., resources, facilities, and equipment) to support the proposed project. It should not be an assessment of the institution’s quality.

• How well qualified is the project team (e.g., PIs, collaborators, and other stakeholders) to conduct the proposed activities?
• Does the investigator (or co-investigator) have demonstrated experience conducting projects of similar size, scope, and complexity?
• If the project is collaborative or dual-PI, do the investigators have complementary and integrated expertise? Are the leadership, governance, and organizational structures appropriate for the project?
  • (Dual-PI Option Only) Does the Leadership Plan adequately describe and justify roles and areas of responsibility of the PIs?
• Is the level of effort for each team member appropriate for successful conduct of the proposed work?
• Does the application describe adequate availability of and access to facilities and resources (e.g., collaborative or partnering arrangements) to carry out the proposed project?
• Is the institutional support appropriate for the proposed project?

Criterion 5. Patient-centeredness

• Does the application demonstrate a clear link between the proposed D&I project and ultimate benefit/value to patients?

Criterion 6. Patient and stakeholder engagement

• Does the application demonstrate that stakeholders central to the proposed project are engaged with planning and execution of the project? Have patients or their representatives been engaged to inform the dissemination strategy or other relevant aspects of the project?
• Has the application demonstrated a sufficient willingness and readiness of the healthcare systems and settings in which dissemination will occur to use and embrace these research results?
• Does the application demonstrate clear interest of personnel at target settings in the D&I of the PCORI research findings and clear commitment to participate as active partners in the project?
• Does the application demonstrate the principles of reciprocal relationships, co-learning, partnership, trust, transparency, and honesty?

In-Person Review

During preliminary review, all administratively compliant applications are evaluated and scored based on PCORI’s merit review criteria. After the preliminary review is completed, PCORI program staff members evaluate panel scores and critiques to identify a subset of applications for merit reviewers to discuss at the in-person review meeting. Not all submitted applications move on to in-person review.

During the in-person review, merit reviewers meet to discuss applications, clarify further the merits of the proposed project plan, and identify areas for improvement. In addition, each application is re-scored based on the content of discussion. The Panel Chair and PCORI MRO lead the in-person panel meeting and ensure that all applications receive a fair and thorough review according to the standards outlined in the PFA.

Post-Panel Review

After the in-person meeting, PCORI program staff evaluate final merit review panel scores and comments, identify duplication or synergy among funded projects, and consider the fit of applications within the programmatic vision. Engagement/Dissemination Program staff members then recommend projects to the OCEDO for funding approval. The D&I Program, including its funded projects portfolio, is governed by the Engagement, Dissemination, and Implementation Committee, which is a subcommittee of PCORI’s Board of Governors.

In addition, PCORI evaluates applicant risk before issuing a PCORI award. Factors considered include financial stability, quality of management systems, audit findings, and past performance on PCORI awards (e.g., compliance with PCORI reporting requirements, conformance to PCORI terms and conditions on previous awards, and timely achievement of milestones). Based on the risk assessment, PCORI may impose special terms and conditions on awardees or withhold contract issuance until such business risks are mitigated. PCORI will not award new contracts to current awardees with overdue reports (progress, interim, final, etc.) until the overdue reports have been submitted to PCORI.

Summary Statements and Funding Recommendations

Summary statements are provided to applicants the same day that funding decisions are announced. If an application progresses to in-person discussion, the applicant will receive a summary statement that includes:

• In-person panel discussion notes
• Final average overall score
• Preliminary reviewer critiques

Summary statements for applications that do not progress to in-person discussion include only the preliminary reviewer critiques.

Funding recommendations are made by identifying meritorious applications that fit the programmatic needs and that satisfactorily address the merit review criteria. Program teams also consider the funds allotted for the current PFA when deciding which applications to recommend for approval. Applicants to this current cycle’s PFA will receive summary statements and notification of the funding status of their application no later than May 2018.

Back to top