Magnitude of Potential Biases in a Simulated Case-Control Study of the Effectiveness of Influenza Vaccination

Influenza Division, Centers for Disease Control and Prevention, Atlanta, Georgia 30333, USA.
Clinical Infectious Diseases (Impact Factor: 8.89). 11/2011; 54(1):25-32. DOI: 10.1093/cid/cir750
Source: PubMed


Many influenza vaccine effectiveness estimates have been made using case-control methods. Although several forms of bias may distort estimates of vaccine effectiveness derived from case-control studies, there have been few attempts to quantify the magnitude of these biases.
We estimated the magnitude of potential biases in influenza vaccine effectiveness values derived from case-control studies from several factors, including bias from differential use of diagnostic testing based on influenza vaccine status, imperfect diagnostic test characteristics, and confounding. A decision tree model was used to simulate an influenza vaccine effectiveness case-control study in children. Using probability distributions, we varied the value of factors that influence vaccine effectiveness estimates, including diagnostic test characteristics, vaccine coverage, likelihood of receiving a diagnostic test for influenza, likelihood that a child hospitalized with acute respiratory infection had influenza, and others. Bias was measured as the difference between the effectiveness observed in the simulated case-control study and a true underlying effectiveness value.
We found an average difference between observed and true vaccine effectiveness of -11.9%. Observed vaccine effectiveness underestimated the true effectiveness in 88% of model iterations. Diagnostic test specificity exhibited the strongest association with observed vaccine effectiveness, followed by the likelihood of receiving a diagnostic test based on vaccination status and the likelihood that a child hospitalized with acute respiratory infection had influenza. Our findings suggest that the potential biases in case-control studies that we examined tend to result in underestimates of true influenza vaccine effects.

Download full-text


Available from: Jill M Ferdinands
  • Source
    • "Second, the diagnostic tests used to diagnose influenza infection were of variable sensitivity and specificity. In particular, there is concern with the use of commercially available influenza antigen detection assays, as the sensitivity and specificity of these tests can be quite variable.31,32 In the context of a case-control study, the specificity of the test used to provide laboratory confirmation of influenza infection in the cases is of great importance, as false positives can lead to substantial bias in the results.32 "
    [Show abstract] [Hide abstract]
    ABSTRACT: Young children are at increased risk of severe outcomes from influenza illness, including hospitalization. We conducted a case-control study to identify risk factors for influenza-associated hospitalizations among children in U.S. Emerging Infections Program sites. Cases were children 6-59 months of age hospitalized for laboratory-confirmed influenza infections during 2005-08. Age- and zip-code-matched controls were enrolled. Data on child, caregiver, and household characteristics were collected from parents and medical records. Conditional logistic regression was used to identify independent risk factors for hospitalization. We enrolled 290 (64%) of 454 eligible cases and 1,089 (49%) of 2,204 eligible controls. Risk for influenza hospitalization increased with maternal age <26 years (odds ratio [OR] 1.8, 95% confidence interval [CI] 1.1-2.9); household income below the poverty threshold (OR 2.2, CI 1.4-3.6); smoking by >50% of household members (OR 2.9, CI 1.4-6.6); lack of household influenza vaccination (OR 1.8, CI 1.2-2.5); and presence of chronic illnesses, including hematologic/oncologic (OR 11.8, CI 4.5-31.0), pulmonary (OR 2.9, CI 1.9-4.4), and neurologic (OR 3.8, CI 1.6-9.2) conditions. Full influenza immunization decreased the risk among children aged 6-23 months (OR 0.5, CI 0.3-0.9) but not among those 24-59 months of age (OR 1.5, CI 0.8-3.0; p-value for difference = 0.01). Chronic illnesses, young maternal age, poverty, household smoking, and lack of household influenza vaccination increased the risk of influenza hospitalization. These characteristics may help providers to identify young children who are at greatest risk for severe outcomes from influenza illness.
    Full-text · Article · Mar 2014 · The Pediatric Infectious Disease Journal
  • Source
    • "Innovative observational approaches have been developed to estimate vaccine effectiveness with laboratory-confirmed outcomes, particularly in preventing medically attended acute respiratory infections (MAARI). Recent MAARI studies in Europe, Canada, and the United States have used an approach in which vaccine effectiveness is estimated by comparing vaccination coverage in persons who test positive for influenza with those who test negative [ utilize a variation of the traditional case-control design and it is not yet clear whether they adequately account for the range of biases typically associated with such studies [6]. There has been a long tradition of using household cohorts to study incidence and transmission of respiratory illnesses of all severities [7]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background: There is a recognized need to determine influenza vaccine effectiveness on an annual basis and a long history of studying respiratory illnesses in households. Methods: We recruited 328 households with 1441 members, including 839 children, and followed them during the 2010-2011 influenza season. Specimens were collected from subjects with reported acute respiratory illnesses and tested by real-time reverse transcriptase polymerase chain reaction. Receipt of influenza vaccine was defined based on documented evidence of vaccination in medical records or an immunization registry. The effectiveness of 2010-2011 influenza vaccination in preventing laboratory-confirmed influenza was estimated using Cox proportional hazards models adjusted for age and presence of high-risk condition, and stratified by prior season (2009-2010) vaccination status. Results: Influenza was identified in 78 (24%) households and 125 (9%) individuals; the infection risk was 8.5% in the vaccinated and 8.9% in the unvaccinated (P = .83). Adjusted vaccine effectiveness in preventing community-acquired influenza was 31% (95% confidence interval [CI], -7% to 55%). In vaccinated subjects with no evidence of prior season vaccination, significant protection (62% [95% CI, 17%-82%]) against community-acquired influenza was demonstrated. Substantially lower effectiveness was noted among subjects who were vaccinated in both the current and prior season. There was no evidence that vaccination prevented household transmission once influenza was introduced; adults were at particular risk despite vaccination. Conclusions: Vaccine effectiveness estimates were lower than those demonstrated in other observational studies carried out during the same season. The unexpected findings of lower effectiveness with repeated vaccination and no protection given household exposure require further study.
    Preview · Article · Mar 2013 · Clinical Infectious Diseases
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Following the 2009 influenza A/H1N1 (pH1N1) pandemic, both seasonal and pH1N1 viruses circulated in the US during the 2010-2011 influenza season; influenza vaccine effectiveness (VE) may vary between live attenuated (LAIV) and trivalent inactivated (TIV) vaccines as well as by virus subtype. Vaccine type and virus subtype-specific VE were determined for US military active component personnel for the period of September 1, 2010 through April 30, 2011. Laboratory-confirmed influenza-related medical encounters were compared to matched individuals with a non-respiratory illness (healthy controls), and unmatched individuals who experienced a non-influenza respiratory illness (test-negative controls). Odds ratios (OR) and VE estimates were calculated overall, by vaccine type and influenza subtype. A total of 603 influenza cases were identified. Overall VE was relatively low and similar regardless of whether healthy controls (VE = 26%, 95% CI: -1 to 45) or test-negative controls (VE = 29%, 95% CI: -6 to 53) were used as comparison groups. Using test-negative controls, vaccine type-specific VE was found to be higher for TIV (53%, 95% CI: 25 to 71) than for LAIV (VE = -13%, 95% CI: -77 to 27). Influenza subtype-specific analyses revealed moderate protection against A/H3 (VE = 58%, 95% CI: 21 to 78), but not against A/H1 (VE = -38%, 95% CI: -211 to 39) or B (VE = 34%, 95% CI: -122 to 80). Overall, a low level of protection against clinically-apparent, laboratory-confirmed, influenza was found for the 2010-11 seasonal influenza vaccines. TIV immunization was associated with higher protection than LAIV, however, no protection against A/H1 was noted, despite inclusion of a pandemic influenza strain as a vaccine component for two consecutive years. Vaccine virus mismatch or lower immunogenicity may have contributed to these findings and deserve further examination in controlled studies. Continued assessment of VE in military personnel is essential in order to better inform vaccination policy decisions.
    Full-text · Article · Jul 2012 · PLoS ONE
Show more