Public Comments - Methodology Standards

« Back to the list of all comments on the 2016 Methodology Standards

3: Standards for Data Integrity and Rigorous Analysis

IR-1: A priori, specify plans for quantitative data analysis that correspond to major aims

Researchers should describe the analytic approaches that will be used to address the major research aims before analysis is undertaken. These include definitions of key exposures, endpoints, and covariates. Also identify patient subgroups of interest, plans (if any) for how new subgroups of interest will be identified or how analysis plans may be adapted based on changing needs and scientific advances, and plans for how missing data will be handled.

Public comments

AcademyHealth appreciates Standard IR-1 and its attempt to prompt researchers to think through analytic approach issues as well as to define these issues prior to conducting a base analysis. However, we feel that this Standard is missing specified plans for robustness tests. Although PCORI mentions these later in the Standards, specifically when referring to missing data methods, robustness tests are not limited to dealing only with missing data issues. To rectify this omission, PCORI could add a simple sentence, such as, “Researchers should specify their plans for robustness and sensitivity tests in advance of doing these analyses.”
Lisa Simpson, AcademyHealth, Stakeholder - Other, 04/11/2016 - 4:34pm