Signal detection of adverse events with imperfect confirmation rates in vaccine safety studies using self-controlled case series design

Biometrical Journal (Impact Factor: 0.95). 05/2014; 56(3). DOI: 10.1002/bimj.201300012
Source: PubMed


The Vaccine Safety Datalink project captures electronic health record data including vaccinations and medically attended adverse events on 8.8 million enrollees annually from participating managed care organizations in the United States. While the automated vaccination data are generally of high quality, a presumptive adverse event based on diagnosis codes in automated health care data may not be true (misclassification). Consequently, analyses using automated health care data can generate false positive results, where an association between the vaccine and outcome is incorrectly identified, as well as false negative findings, where a true association or signal is missed. We developed novel conditional Poisson regression models and fixed effects models that accommodate misclassification of adverse event outcome for self-controlled case series design. We conducted simulation studies to evaluate their performance in signal detection in vaccine safety hypotheses generating (screening) studies. We also reanalyzed four previously identified signals in a recent vaccine safety study using the newly proposed models. Our simulation studies demonstrated that (i) outcome misclassification resulted in both false positive and false negative signals in screening studies; (ii) the newly proposed models reduced both the rates of false positive and false negative signals. In reanalyses of four previously identified signals using the novel statistical models, the incidence rate ratio estimates and statistical significances were similar to those using conventional models and including only medical record review confirmed cases.

25 Reads
  • [Show abstract] [Hide abstract]
    ABSTRACT: PurposeOutcome misclassification in retrospective epidemiologic analyses has been well-studied, but little is known about such misclassification with respect to sequential statistical analysis during surveillance of medical product-associated risks, a planned capability of the US Food and Drug Administration's Sentinel System.Methods Using a vaccine example, we model and simulate sequential database surveillance in an observational data network using a variety of outcome detection algorithms. We consider how these algorithms, as characterized by sensitivity and positive predictive value, impact the length of surveillance and timeliness of safety signal detection. We show investigators/users of these networks how they can perform preparatory study design calculations that consider outcome misclassification in sequential database surveillance.ResultsNon-differential outcome misclassification generates longer surveillance times and less timely safety signal detection as compared with the case of no misclassification. Inclusive algorithms characterized by high sensitivity but low positive predictive value outperform more narrow algorithms when detecting rare outcomes. This decision calculus may change considerably if medical chart validation procedures were required.Conclusions These findings raise important questions regarding the design of observational data networks used for pharmacovigilance. Specifically, there are tradeoffs involved when choosing to populate such networks with component databases that are large as compared with smaller integrated delivery system databases that can more easily access laboratory or clinical data and perform medical chart validation. Copyright © 2014 John Wiley & Sons, Ltd.
    Pharmacoepidemiology and Drug Safety 04/2014; 23(8). DOI:10.1002/pds.3618 · 2.94 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Surveillance for intussusception (IS) has been recommended in countries using rotavirus vaccine, but can be resource intensive. There is little data about the relative severity of rotavirus vaccine-associated IS compared with other IS cases. We collected detailed clinical data on all cases to evaluate the validity of ICD-coding for IS in routinely collected data and case severity. Hospitalisations and emergency department (ED) presentations coded as IS in infants aged <12 months from 1 July 2007 to 30 June 2010 were classified using Brighton criteria by case note review. We used self-controlled case series (SCCS) analysis to estimate IS risk following vaccination for all and only Brighton level 1 cases. Of 179 unique episodes coded as IS, 110 (61%) met Brighton level 1 criteria; SCCS analysis found a relative incidence (RI) of IS in days 1-7 following the first dose of RV1 of 11.1 (95% CI: 2.6-48.0). When all coded episodes of IS were included RI was 4.0 (95% CI: 1.3-12.7). The proportion of Brighton 1 cases requiring surgery was 39% for those within 21 days of vaccine receipt and 34% for others (p=0.67). Using ICD-coded cases without individual confirmation yielded a lower point estimate of risk for IS post rotavirus vaccination, however, the risk remained statistically compatible with that for chart confirmed cases only. Analysis using healthcare databases to evaluate risk of IS if conducted without case-confirmation may be insufficient to confirm a low level risk. IS episodes following vaccination were not more severe.
    The Pediatric Infectious Disease Journal 04/2014; 33(9). DOI:10.1097/INF.0000000000000362 · 2.72 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Healthcare databases have been used in the past four decades to identify, refine, and evaluate potential safety signals of marketed medical products. Critics have challenged this research because data are from secondary sources and because some published studies have lacked robust methods for exposure and outcome definition and failed to adequately control for biases. We review the history of healthcare databases used in pharmacovigilance for quantifying adverse outcomes associated with therapeutics, methods to improve the quality of this research, and best practices for pharmacoepidemiologic studies. Drug and vaccine safety studies increasingly use information from multiple healthcare databases, with analyses that aim to keep patient-level identifying data with local research custodians. Analytic methods, including high-dimensional exposure propensity scores, use large numbers of variables to reduce confounding and further anonymize patient data. However, due to gaps in and complexities of the available databases, the value of the research depends on experts with knowledge about the clinical context (e.g., how products are prescribed and taken, how outcomes are diagnosed and recorded, what risk factors must be considered), understanding the nuances of individual databases and the clinical practice patterns they represent, and utilizing study designs that minimize bias, particularly confounding by medication indication.
    12/2014; 1(4). DOI:10.1007/s40471-014-0026-0