Applicability, reliability, sensitivity, and specificity of six Brighton Collaboration standardized case definitions for adverse events following immunization

Immunization Safety Office, Office of the Chief Science Officer, Centers for Disease Control and Prevention, 1600 Clifton Road, Mailstop E-03, Atlanta, GA, USA.
Vaccine (Impact Factor: 3.49). 10/2008; 26(50):6349-60. DOI: 10.1016/j.vaccine.2008.09.002
Source: PubMed

ABSTRACT We evaluated the applicability, reliability, sensitivity, and specificity of six standardized case definitions for adverse events following immunization (AEFI) (for fever, generalized convulsive seizure, hypotonic-hyporesponsive episode, intussusception, nodule, and persistent crying) developed by the Brighton Collaboration using the U.S. Vaccine Adverse Event Reporting System (VAERS). The evaluation included: (a) the development of codified search strings using standardized coding terminology, and (b) for sensitivity and specificity analyses, the development of a "gold standard" for case determination by clinical expert reviews, and its comparison against the application of the definitions to VAERS reports by nonclinicians. Application of the case definitions in an automated approach proved to be valid, feasible, and unlikely to miss confirmed cases of the reported clinical event. The definitions had variable but generally high sensitivity and specificity compared to clinician review, which in itself yielded inconsistent case determination. The study demonstrated the need for the developed standardized definitions for AEFI and their usefulness in passive surveillance.

  • [Show abstract] [Hide abstract]
    ABSTRACT: The Canadian Adverse Event Following Immunization Surveillance System (CAEFISS) receives report via active syndromic surveillance for selected serious AEFI from the Canadian Immunization Monitoring Program Active (IMPACT) and via targeted passive surveillance from Federal/Provincial/Territorial health jurisdictions. Post-immunization seizure is a target of active and passive surveillance. Since 2009, the revised national AEFI reporting forms enable capture of terms specific to several Brighton Collaboration Case Definitions (BCCD) including generalized seizure and fever. To evaluate feasibility of applying the BCCD for generalized seizure to adverse event following immunization (AEFI) reports collected by IMPACT and targeted passive surveillance (non-IMPACT). Reports to CAEFISS coded as seizure in children <2 years of age (vaccination dates 1998-2011) were reviewed retrospectively. A BCCD level (1-5 or unclassifiable) was assigned. The effects of reporting source (IMPACT versus non-IMPACT), seriousness [serious (e.g., hospitalized) versus non-serious], vaccination year (1998-2008 versus 2009-2011), and data submission method to CAEFISS (electronic versus paper) were assessed by stratified analysis. There were 459 IMPACT and 908 non-IMPACT cases analyzed, of which 99.6% and 27%, respectively, were serious reports. The revised reporting form that captured the BCCD components (2009-2011) was associated with increased proportions of IMPACT and non-IMPACT cases meeting the BCCD for generalized seizure. Incorporating the BCCD components (level of consciousness, motor manifestations and fever ≥38°C) into the national reporting form and guidelines appeared to improve the feasibility of their use in AEFI surveillance. This effect was more pronounced among active syndromic surveillance compared to targeted passive surveillance reports.
    Vaccine 10/2013; DOI:10.1016/j.vaccine.2013.09.048 · 3.49 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Combination vaccines have improved the efficiency of delivery of new vaccines in low and middle-income countries. Post-authorization monitoring of adverse events (AEs) after vaccination with a liquid pentavalent DTwP-HepB-Hib combination vaccine was conducted in Guatemalan infants. A prospective observational safety study of the incidence of medical attended events (MAEs) and serious adverse events (SAEs) in children who received pentavalent and oral polio vaccines at 2, 4 and 6 months of age was conducted in two clinics at the Institute of Guatemala. Parents were contacted by telephone after each dose. All outpatient, emergency department visits, and hospitalizations were monitored. A self-controlled analysis was conducted to determine if there was evidence of increased risk of MAEs or SAEs following vaccines as compared to control time windows. Of 3000 recruited infants, 2812 (93.7%) completed the third dose and 2805 (93.5%) completed follow-up. Ten AEs in eight infants, of which four SAEs in four infants, were classified as related to the vaccine. Thirteen deaths were reported due to common illnesses of infancy, and none were judged to be related to the vaccine. The mortality rate (4.4 per 1000) was lower than expected for the population. The incidence-rate-ratio for healthcare visits was lower in post-vaccination time windows than for control windows; after the first vaccine dose, the rate ratios for the risk periods of 0-1, 2-6, and 7-30 days post-vaccination were 0.3, 0.5, and 0.7, respectively (all statistically significantly different from the reference value of 1.0 for the 31-60 day control period). The liquid pentavalent vaccine was associated with lower rates of health care visits and not associated with increases in SAEs or hospitalizations. Systems can be set up in low to middle income countries to capture all health care visits to monitor the safety of new vaccines.
    Vaccine 09/2013; 31(49). DOI:10.1016/j.vaccine.2013.09.015 · 3.49 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Establishing a Case Definition (CDef) is a first step in many epidemiological, clinical, surveillance, and research activities. The application of CDefs still relies on manual steps and this is a major source of inefficiency in surveillance and research. Describe the need and propose an approach for automating the useful representation of CDefs for medical conditions. We translated the existing Brighton Collaboration CDef for anaphylaxis by mostly relying on the identification of synonyms for the criteria of the CDef using the NLM MetaMap tool. We also generated a CDef for the same condition using all the related PubMed abstracts, processing them with a text mining tool, and further treating the synonyms with the above strategy. The co-occurrence of the anaphylaxis and any other medical term within the same sentence of the abstracts supported the construction of a large semantic network. The 'islands' algorithm reduced the network and revealed its densest region including the nodes that were used to represent the key criteria of the CDef. We evaluated the ability of the "translated" and the "generated" CDef to classify a set of 6034 H1N1 reports for anaphylaxis using two similarity approaches and comparing them with our previous semi-automated classification approach. Overall classification performance across approaches to producing CDefs was similar, with the generated CDef and vector space model with cosine similarity having the highest accuracy (0.825±0.003) and the semi-automated approach and vector space model with cosine similarity having the highest recall (0.809±0.042). Precision was low for all approaches. The useful representation of CDefs is a complicated task but potentially offers substantial gains in efficiency to support safety and clinical surveillance.
    Applied Clinical Informatics 10/2013; 4(4):515-27. DOI:10.4338/ACI-2013-04-RA-0028 · 0.39 Impact Factor