6: Standards for Data Registries

DR-1: Requirements for the design of registries

Registries established for conducting PCOR must have the following characteristics.

1. Registry Purpose and Protocol
   The purpose of the registry should be clearly defined to guide the design of key registry features including, but not limited to: the target population, the research question/s to be addressed, the data source utilized, the data elements collected, data sharing policies, and the stakeholders involved in the development and use of the registry. Participants and other key stakeholders should be engaged in registry and protocol development. Registries should aim to be user-oriented in design and function.
2. Data Safety and Security
   Registry custodians should comply with IRB requirements, local and national laws, and where applicable, HIPAA. Registries should provide information describing type of data collection (primary or secondary source data), data use agreements, informed consent documents, data security protections, plans for maintaining data protection if the registry ends, and approaches to protecting security including risk and/or process for re-identification of participants, especially for medical or claims records.
3. Data Elements and Quality
   Standardized data element definitions and/or data dictionaries should be used whenever possible. When creating a new registry, published literature should be reviewed to identify existing, widely used definitions of outcomes, exposure, and confounders before drafting new definitions.
   
   When collecting primary data, conduct multi-stakeholder engagement with potential participants and data users to prioritize data collection needs. When participants support their face validity, utilize validated instruments or patient-reported outcome measures when available. If secondary data sources (e.g., electronic medical records, claims data) are utilized, describe the original purpose of the secondary data and verify the accuracy and completeness of the data, as well as the approach to and validity of the linkages performed between the primary and secondary sources.
   
   The specifics of the quality assurance plan will depend on the type of data (primary or secondary) collected by the registry. In general, the plan should address: 1) structured training tools for data abstractors/curators; 2) use of data quality checks for ranges and logical consistency for key exposure and outcome variables and covariates; and 3) data review and verification procedures, including source data verification plans (where feasible and appropriate), and validation statistics focused on data quality for the key exposure and outcome variables and key covariates. A risk-based approach to quality assurance is advisable, focused on variables of greatest importance.
4. Confounding
   Registries should identify important potential confounders pertinent to the purpose and scope of the research during the planning phase and collect reasonably sufficient data on these potential confounders to facilitate the use of appropriate statistical techniques during the analysis phase. When conducting analysis, refer to the PCORI Methodology Standards for Data Integrity and Rigorous Analyses and the Standards for Causal Inference Methods.
5. Systematic Participant Recruitment and Enrollment
   Develop a sampling plan (population-based or otherwise) of the target population and identify recruitment strategies for participants that minimize the impact of selection bias. Participants should be enrolled systematically, with similar procedures implemented at all participating sites and for each intervention of interest. Confirm adherence to agreed-upon enrollment practices.
6. Participant Follow-Up
   The objective(s) of the registry should determine the type, extent, and length of participant follow-up.
   
   Describe the frequency with which follow-up measures will be ascertained, consider linkage with other data sources such as the National Death Index to enhance long-term follow-up, and identify the date of last contact with the participant in existing registries, where appropriate. Ensure that the participants are followed in as unbiased a manner as possible, using similar procedures at all participating sites.
   
   Monitor loss to follow-up to ensure best efforts are used to achieve follow-up time that is adequate to address the main objective. At the outset of the registry, develop a retention plan that documents when a participant will be considered lost to follow-up and what actions will be taken to minimize loss of pertinent data. Retention efforts should be developed with stakeholders to ensure the efforts are suitable for the target population and anticipated challenges are addressed appropriately.

Public comments

First, in order to make a distinction between engagement in designing the registry infrastructure and specific studies, AcademyHealth would recommend modifying the second sentence of the “Registry Purpose and Protocol” language slightly to read, “Participants and other key stakeholders should be engaged in registry design and study protocol development.” Furthermore, we recommend adjusting the language in first sentence of the “Data Safety and Security” section as follows: “Registry custodians should comply with IRB requirements, the HIPPA Privacy Rule, and all other applicable state and federal laws.” Finally, unless further classification is given to the “Systematic Participant Recruitment and Enrollment” section on which sampling plans could be categorized as “otherwise,” we would recommend simply dropping the phrase “(population-based or otherwise)” from the first sentence.

Lisa Simpson, AcademyHealth, Stakeholder - Other, 04/11/2016 - 4:40pm

DR-2: Documentation and reporting requirements of registry materials, characteristics and bias

Clearly describe, document with full citations where appropriate, and make publicly available registry materials including but not limited to registry protocols; data sharing policies; operational definitions of data elements; survey instruments utilized and patient-reported outcomes captured. Modifications to any documents or data collection instruments should be clearly described and made available for registry users and participants. Characteristics of the participants in the registry should be described. Identify how the participants may differ from the target population to help assess potential selection biases. Document loss to follow-up and describe impact on the results, utilizing sensitivity analyses (pre-specified where possible) to quantify possible biases. Report the extent of bias clearly to stakeholders who may want to utilize the registry resource.

Public comments

No comments.

DR-3: Adapting Established Registries for PCORI Research

Previously established registries that intend to support new clinical research may not always have been guided by all applicable Methodology Standards. When new research will use such registries, investigators should engage key stakeholders, including registry participants, to ensure:

1. Informed consent documents are appropriately tailored to participant needs, characteristics and conditions
2. Data elements are meaningful and useful to researchers and participants
3. Recruitment and retention strategies are feasible and effective
4. Registry policies are patient-centered and the use of registry data is transparent to participants
5. Dissemination practices of PCORI-funded research using registry data are appropriate and effective at reaching the communities from which the data are collected
6. Opportunities for bi-directional benefit between participants and researchers
7. Registry materials, described in DR-2, and informed consent forms are publicly available in accessible formats

Public comments

No comments.

DR-4: Documentation requirements when using registry data

Researchers planning PCOR studies relying on registries must ensure that these meet the requirements contained in Standards DR-1 and DR-2 and must document each required feature of the registry(s) to be used (e.g., in an appendix to the funding application or study protocol). Deviations from the requirements with Standards DR-1 and DR-2 should be well documented and limitations of research related to the deviations from requirements should be addressed along with study findings.

Public comments

No comments.

General feedback on the Standards for Data Registries

Public comments

AcademyHealth recommends that PCORI encourage both existing registries and those in development to submit a registry profile to AHRQ’s Registry of Patient Registries (RoPR) to promote collaboration and encourage transparency among registry developers and users. This also represents another inconsistency with PCORI’s former guidance to researchers on registering studies; PCORI’s dissemination and communication guidance suggests that investigators working with registries should report to RoPR, but this registry isn’t referenced within the Methodology Standards.

Lisa Simpson, AcademyHealth, Stakeholder - Other, 04/11/2016 - 4:40pm

• The current and proposed standards for data registries appear to be primarily focused on good practices for designing new data registries. However, there are three uses of data registries for PCOR studies that call for somewhat different quality considerations and standards: (1) Observational PCOR studies for which a registry is being newly designed to provide data. (2) Observational PCOR studies which propose to use data elements from an existing ongoing registry to address a research question. (3) Observational PCOR studies which propose to modify an existing ongoing registry to address a research question. The elegance and strength of a patient registry is its ability to answer many questions that are often not known or thought to be needed at the inception of the registry. PCORI should augment existing standards or create new standards in this section to detail how registries can and should support “new research and research questions” that were unknown when the registry was first designed. Specific issues for consideration may be: advisory board review and approval, protocol and data collection modifications, confirmation of appropriate participant consent for the new questions, etc. We recommend that PCORI consider including standards in the report that account for the types of registries listed above and address related quality requirements for each. • When using an existing registry, the standard should require a feasibility assessment of a proposed registry based on its history of operation (taking into account potential sponsor or clinician biases) and quality to date.

Eli Lilly and Company , Industry, 03/30/2016 - 2:19pm