Merit Review Criteria for PLACER PFA
Phased Large Awards for Comparative Effectiveness Research (PLACER)
Criterion 1. Potential for the study to fill important gaps in evidence
The application should clearly justify the importance of the clinical or care delivery problems that the study will address and whether the proposed approach is appropriately conceptualized.
- Is this an important research question for which the answers will help decision makers (e.g., patients, clinicians, health systems, payers, policy makers)? Is there compelling evidence that the research question addresses an important health decision?
- Does the application identify a critical gap in current knowledge as noted in systematic reviews, guideline development efforts, authoritative recommendations by stakeholder organizations, or with evidence of inconsistency in clinical practice and decision making?
- Would research findings from the study have the potential to fill the demonstrated evidence gaps?
- Does the importance of the research question justify the scope and magnitude of the proposed study?
- Will the findings remain relevant and valuable given the proposed timeline to achieve them?
Criterion 2. Potential for the study findings to be adopted into clinical practice and improve delivery of care
The application should describe how evidence generated from this study can be readily adopted into clinical practice and delivery of care. The application should also address the following questions:
- Does the application identify end-users, such as local and national stakeholders who have expressed their interest in applying study findings?
- Would the study’s findings inform decision making for these key stakeholders?
- Has the applicant identified potential barriers to intervention adoption and strategies to address such barriers?
- Has the applicant identified resources or factors that would promote adoption of the intervention?
Criterion 3. Scientific merit (research design, analysis, and outcomes)
The application should show sufficient technical merit in the research design to ensure that the study goals will be met. The application should also address the following questions:
- Does the application describe a clear causal framework anchored in background literature that informs the design, key variables, and relationship between interventions and outcomes being tested?
- Does the research plan describe rigorous methods that adhere to the PCORI Methodology Standards?
- Is the overall study design justified?
- Are the patient population and study setting appropriate for the proposed research question?
- Does the application provide justification that the outcome measures are validated and appropriate for the population?
- Are each of the comparators (e.g., active intervention arm and comparator arm) described clearly, and are they well justified? What is the strength of evidence supporting their efficacy and/or widespread use? If “usual care” is one of the arms, is it adequately justified, and will it be sufficiently measured?
- Are the sample sizes and power estimates well justified and appropriate for the proposed study design (e.g., cluster randomized design, RCT) as well as the specific research question? Is the anticipated effect size adequately justified?
- Is the overall study plan for both phases well justified and coherent? Are the project timeline and milestones realistic?
For the proposed DCC:
- Are appropriate plans in place for DCC involvement in development of the final trial design and protocol, the study power and analytical approach, the statistical analysis plan, and the conduct of all proposed analyses?
- Are there adequate plans for data collection/reporting and its quality monitoring for errors, omissions, and completeness?
- Is there a well-justified plan for managing the functions of the DSMB?
For the feasibility phase:
- How likely will the major activities to be accomplished in the feasibility phase prepare for the successful execution of the full-scale trial?
- Is the scope and duration of the feasibility phase appropriate and realistic for what is proposed?
- Does the application adequately plan a defensible and realistic estimate of enrollment considering the available eligible population, acceptance of randomization by providers and participants, burden(s) of participation, attrition, and other factors influencing study entry?
- Is contingency planning adequate for potential obstacles or challenges?
Criterion 4. Investigator(s) and environment
This criterion should assess the appropriateness (i.e., qualifications and experience) of the investigator(s)/team and the environment’s capacity (e.g., resources, facilities, equipment) to support the proposed project. It should not be an assessment of the institution’s reputation, but rather of the breadth and depth of its available personnel and resources. The application should also address the following questions:
- How well qualified are the Principal Investigators (PIs), collaborators, and other researchers to conduct the proposed activities? Is there evidence of sufficient clinical, statistical, and study management expertise?
- If the project is collaborative or dual-PI, do the investigators have complementary and integrated expertise? If the leader of the DCC is not designated a dual-PI, does the application provide strong justification why this arrangement is not needed?
- Does the investigative team have adequate experience in conducting projects of a similar size, scope, and complexity, including phased studies?
- Are the leadership, governance, and organizational structures adequate for the scope of the project? Does the Leadership Plan clearly describe and justify investigator roles and areas of responsibility? Does the plan support efficient and high-quality research?
- Do the personnel who will manage engagement activities have appropriate experience, resources, and time commitments to carry out the proposed engagement of patients and stakeholders in research?
- Does the application describe adequate availability of and access to facilities and resources for the proposed research?
- Is the institutional support (including the DCC) appropriate for the proposed research? For the proposed DCC:
- Are the experience and capabilities of the DCC and the DCC leadership appropriate to the proposed study?
- Does the application clearly describe the DCC’s role and functions?
- Does the application reference the prime institution’s policies and practices to ensure DCC independence and compliance with all applicable data management and security requirements?
Criterion 5. Patient-centeredness
The application should demonstrate that the study focuses on improving patient-centered outcomes and employs a patient-centered research design (i.e., a design informed or endorsed by patients). (Note: The study can be patient centered even if the end-user is not the patient, as long as patients will benefit from the information.)
The application should also address the following questions:
- Does the application include a thorough description about which outcomes (both benefits and harms) are important to patients, and are those outcomes included in the study plan?
- Do the comparators represent challenging choices that patients confront?
- What evidence is presented or planned to assess patients’ willingness to accept the proposed comparators considering their potential benefits, risks, and burdens of time, inconvenience, out-of-pocket costs, and other factors?
Criterion 6. Patient and stakeholder engagement
The application should demonstrate the engagement of relevant patients and other stakeholders (e.g., patients, caregivers, clinicians, policy makers, hospital and health system representatives, payers [insurance], purchasers [business], industry, researchers, and training institutions) in the conduct of the study. Quality of engagement should be evaluated based on scope, form, and frequency of patient and stakeholder involvement throughout the research process. The application should also address the following questions:
- Is the proposed engagement approach appropriate and tailored to the study? Is the frequency and level of involvement of patients, providers, health system partners, and other stakeholders appropriate to support the study goals?
- Are the proposed stakeholders fully representative of the groups most likely to be impacted by the study question? Does the study plan include adequate representation of appropriate stakeholder groups from the list above, to ensure diverse perspectives throughout the research process?
- Are planned engagement activities adequate to assist in determining the acceptability (e.g., the time, resource demands, or other burdens) of the comparators, randomization, and requirements of study conduct and participation for providers and participants?
- Are there clear descriptions of the roles and contributions of all study collaborators in decision making?
Posted: January 28, 2021