The extent and impact of vaccine status miscategorisation on covid-19 vaccine efficacy studies

Full text

The extent and impact of vaccine status
miscategorisation on covid-19 vaccine efficacy
studies
Martin Neil (m.neil@qmul.ac.uk)
School of Electronic Engineering and Computer Science
Queen Mary, University of London
Norman Fenton (n.fenton@qmul.ac.uk)
School of Electronic Engineering and Computer Science
Queen Mary, University of London
Scott McLachlan (scott.mclachlan@kcl.ac.uk)
School of Nursing, Midwifery and Palliative Care
Kings College London
Abstract
It is recognised that many studies reporting high efficacy for Covid-19 vaccines
suffer from various selection biases. Systematic review identified thirty-nine studies
that suffered from one particular and serious form of bias called miscategorisation
bias, whereby study participants who have been vaccinated are categorised as
unvaccinated up to and until some arbitrarily defined time after vaccination
occurred. Simulation demonstrates that this miscategorisation bias artificially
boosts vaccine efficacy and infection rates even when a vaccine has zero or
negative efficacy. Furthermore, simulation demonstrates that repeated boosters,
given every few months, are needed to maintain this misleading impression of
efficacy. Given this, any claims of Covid-19 vaccine efficacy based on these
studies are likely to be a statistical illusion.
Keywords: simulation; covid-19; evidence-based medicine; mis-categorisation;
selection bias; observational studies; public health; vaccine effectiveness.
—————————— u ——————————
1. Introduction
Considerable attention has been given to the reported high efficacy for the Covid-19 vaccines and
how many of these studies have exhibited signs of selection bias (Reeder, 2021, Fung, Jones &
Doshi, 2023; Heying & Weinstein, 2023; Ioannidis, 2022; Fenton & Neil, 2023). One major kind of
selection bias takes the form of miscategorisation, whereby study participants who have been
vaccinated are miscategorised as unvaccinated up to and until some arbitrarily defined time after
vaccination occurred (typically up to 14 or 21 days). This selection bias, which has been seen to take
several different types, all of which help exaggerate vaccine efficacy, has recently become known
colloquially as the ‘cheap trick’ (Fenton & Neil, 2023).
To identify the different types of miscategorisation bias and evaluate how widespread it is, we
conducted a review of Covid-19 vaccine studies to identify those studies that have employed
miscategorisation selection bias and we have simulated the effects of this selection bias on measures
of vaccine efficacy.
This review reveals that, up to February 2024, 39 research studies on Covid-19 vaccines have
employed different types of this bias, with variants including straightforward miscategorisation from
one category to another, miscategorising the vaccinated as having unverified vaccination status,
1

uncontrolled reporting of vaccination status and excluding those vaccinated from the study. Many of
the studies have applied one or more of these biases within time periods from one week to three.
Our simulation model demonstrated that this selection bias artificially boosts vaccine efficacy in all
cases, and with the application of repeated ‘booster’ vaccinations, the efficacy of repeated Covid-19
vaccines could be maintained at artificially high levels in perpetuity. Furthermore, in tandem with this
the infection rate would likewise be artificially elevated and would be lower for the unvaccinated cohort
compared to the vaccinated cohort, further compounding misleading claims that a Covid-19 vaccine
reduces infection rates when it does not.
The paper is structured as follows: In Section 2 we review the work on biases in Covid-19 vaccine
studies. In Section 3 we describe the search method by which relevant studies were selected. In
Section 4 we classify each of the relevant studies according to novel types of miscategorisation
selection biases exhibited. In Section 5 we simulate the vaccine efficacy results that would be
observed during peak rollout of both a placebo and negative efficacy vaccine under the various
selection biases. Section 6 offers our conclusions.
2. Background
Several studies have investigated bias in Covid-19 vaccine studies, including: (i) outcome reporting
bias affecting interpretation of vaccine efficacy where studies report relative risk reduction (RRR)
rather than actual risk reduction (ARR) (Brown, 2021); confounding bias in test-negative studies
where other acute respiratory infections (ARI) are assumed to occur or be independent to Covid-19
(Doll et al, 2022), where authors promote the use of recently vaccinated individuals as a negative
control (Hitchings et al, 2022), due to imperfect sensitivity and/or specificity of the test used to
diagnose the disease (Eusebi et al, 2023; Williams et al, 2022); state bias wherein limited uptake, or
vaccine hesitancy, is said to occur because the general public prefer domestically produced vaccines
over foreign-made (Kobayashi et al, 2021) and alternatively, confirmation bias that causes people to
disregard public information and results in the same hesitancy (Malthouse, 2023); self-selection bias
where participants who have been vaccinated are more likely to also willingly present for swab
collection and testing (Glasziou et al, 2022); and collider stratification bias where rather than the usual
approach of reporting the relative risk of the disease, Covid-19, test-negative studies use the recently
created alternative approach of reporting the relative risk of infection given a second variable,
vaccination (Ortiz-Brizuela et al, 2023). The studies discussed here are approximately evenly divided
between those that report biases that have exaggerated factors of vaccine safety and efficacy, and
those reporting biases have negatively impacted assessment of these factors and resulting public
perception.
We focus explicitly on miscategorisation selection biases, which inevitably exaggerate vaccine
efficacy. We identify five types of such bias (defined in detail in Section 4), namely: (i)
Miscategorisation (the type most closely associated with the miscategorisation selection bias); (ii)
Unverified; (iii) Uncontrolled; (iv) Excluded; and (v) Undefined. Previous work (Ioannidis, 2021; Fung,
Jones & Doshi, 2023, Lataster 2024) has largely focused only on miscategorisation, so our review is
novel as well as more extensive than previous work. Ioannidis (2021) considers miscategorisation in
terms of vaccination self-reporting by participants, the need for investigators to provide definitions for
what it means to be vaccinated and whether categorisation as vaccinated occurs immediately after
vaccination or after some period, and they discuss the possibility for these definitions to themselves
cause miscategorisation of vaccination status. Fung et al (2023) examine this issue in terms of a
case-counting window bias, in which investigators do not begin counting cases in the fully vaccinated
until the arbitrary period after vaccination had passed. They also found that investigators could apply
this period to both the vaccine and placebo arms of their study, or to the vaccine group alone.
3. Method
A search was conducted of PubMed and Scopus seeking literature presenting either a retrospective
health records or prospective clinical trial of one or more Covid-19 vaccines with efficacy or safety as
an endpoint. The search term used was:
[covid] and [vaccine] and [ecacy] and [safety]
2

The initial search returned 2,209 results. 476 Duplicates were removed, as well as 1,562 that while
discussing or mentioning vaccines for Covid-19 did not present a study of vaccine efficacy or safety
and 134 single-page works that were a mix of protocol disclosures and abstracts of results. Of the 37
remaining, 35 provided sufficient detail of the inclusion and exclusion criteria for inclusion in this study.
A further 4 papers were identified through citation mining of included papers. Each paper was
evaluated for a range of aspects that included the manufacturer and type of vaccine, the control
cohort comparator (placebo or unvaccinated), the primary outcomes (prevention of infection,
hospitalisation, ICU admission or death), the author’s potential conflicts of interest (declared and
undeclared) and whether they included one or more types of miscategorisation selection bias. This
work reports on the last of these factors.
4.Types of miscategorisation selection bias
Our review identified the following five types of the miscategorisation selection bias:
(a) Miscategorisation: During the arbitrarily defined period the vaccinated are categorised as
unvaccinated, twice vaccinated categorised as single vaccinated, or boosted categorised
as twice vaccinated (e.g.: Buchan et al, 2022; Stock et al, 2022).
(b) Unverified: Participants whose vaccination status is unknown or unverified are
categorised as unvaccinated (e.g.: Rosenberg et al, 2021; Lyngse et al, 2022b).
(c) Uncontrolled: Participants are allowed to self-administer or self-report their vaccination or
infection status, became unblinded or sought vaccination outside the study (e.g.: Angel et
al, 2021; Wu et al, 2023).
(d) Excluded: Participants who are vaccinated but who become infected or died during the
arbitrarily defined period are neither categorised as unvaccinated or vaccinated but are
instead simply removed from analysis (e.g.: Tabarsi et al, 2023; Heath et al, 2023);
(e) Undefined: The authors of the study fail to provide definitions for either or both vaccinated
and unvaccinated cohorts (e.g.: Bermingham et al, 2023b; Nordstrom et al, 2022).
Table 1 lists the incidence and frequency of use for each type of miscategorisation selection bias in
Covid-19 vaccine effectiveness research studies. Use of the arbitrary miscategorisation type was
ubiquitous, identified in 100% of the reviewed studies. Further, nearly one-third (31%) also used one
or more of the other types of bias.
Table 1 Research studies containing miscategorisation
selection bias (see appendix for full citation list)
Citation (a) (b) (c) (d) (e) Defined Period
Dagan et al (2021) X 14 days
Haas et al (2021) X 7 days
Rosenberg et al (2021) X X 14 days
Thomas et al (2021) X 7 days
Angel et al (2021) X X 7 days
NSW Health (2021) X X 14 days
Ali et al (2021) X 14 days
Pilishvili et al (2021) X X 14 days / 7 days
Andrews et al (2022) X 28 days
Buam et al (2022) X 21 days / 14 days
Buchan et al (2022) X 7 days
Carazo et al (2022) X 14 days
Chung et al (2022) X 7 days
Palinkas et al (2022) X 7 days
Ferdinands et al (2022) X X X 14 days
Lyngse et al (2022) X 7-15 days
Lyngse et al (2022b) X X 7-15 days
3

Nordstrom et al (2022) X X X 14 days
Petras et al (2022) X 14 days
Robles-Fontan et al (2022) X 14 days
Arbel et al (2022) X 7 days
Paternina et al (2022) X 14 days
Stock et al (2022) X 21 days / 14 days
Bermingham et al (2023) X 21 days
Yau et al (2023) X Until fully vaccinated
Mitchell et al (2023) X 14 days
Tan et al (2023) X 7 days
Al Kaabi et al (2023) X 14 days
Tabarsi et al (2023) X X X 14 days
Heath et al (2023) X X 7 days
Nadeem et al (2023) X 14 days
Anez et al (2023) X 7 days
Munoz et al (2023) X 7 days
Wu et al (2023) X X 28 days
Bermingham et al (2023b) X X 21 days
Liu et al (2023) X 7 days
Kitano et al (2023) X 7 days / 14 days
Polack et al (2020) X X 7 days
Khairullin et al (2022) X 14 days
39 4 5 4 2
5. Simulation of vaccine effectiveness
We used a deterministic temporal simulation to illustrate the effects of the miscategorisation selection
bias on vaccine effectiveness and the reported infection rates for different cohorts, vaccinated and
unvaccinated. We simulated a hypothetical vaccination campaign starting at week 1 and completing
on week 6 with 85% of the observed population vaccinated by that time.
Here we examine several scenarios showing the effect of a one-week, two-week and three-week
selection biases for miscategorisation (a) and exclusion (c) and the effects of repeated vaccination, by
boosting, on vaccine efficacy and infection reported rates. Two scenarios present a placebo (zero-
efficacy) vaccine, which does not affect infection rates, and compare this with a negative-efficacy
vaccine, whereby those vaccinated suffer slightly elevated infection rates compared to the
unvaccinated.
Note that observational studies might suffer from many sources of additional confounding biases so
this model is a simplification and should not be taken as representative of population level data.
The scenarios simulated cover an eleven-week period with an assumed constant weekly infection rate
of 1% in the placebo scenario, and a slightly elevated infection rate, 1.25%, for the vaccinated cohort
in the negative-efficacy scenario. This is used in both the miscategorisation, (a), and excluded, (c),
simulations. To simulate the effects of boosters we assume a population that is repeatedly vaccinated
every twelve weeks, with those who are vaccinated miscategorised (a) within one week of each
vaccination.
The results of the five scenarios are presented in Figure 1.
4

Figure 1 Five scenarios A-E. A: Miscategorisation, (a) with placebo vaccine; B: Miscategorisation, (a), with negative
efficacy vaccine; C: Exclusion, (c) with placebo vaccine; D: Exclusion, (c), with negative efficacy vaccine; E: Boosting
with miscategorisation, (a), with placebo vaccine
In practice, most studies do not report vaccine efficacy in the initial week(s) (when no cases are
categorised as vaccinated) as this would show up as 100% efficacy. However, note that in all
scenarios in the first weeks where efficacy would be reported the starting point for efficacy is over
90%.
In scenario A, miscategorisation, (a), with a placebo, high vaccine effectiveness falls towards zero
after one, two or three-week periods, accompanied by an increase in the reported infection rate for the
unvaccinated cohort from the start of the vaccination campaign. After seven weeks the reported
infection rates for the vaccinated and unvaccinated cohorts converge on the true infection rate. In
scenario B, miscategorisation, (a), with a negative effectiveness vaccine, the reported vaccine
effectiveness is negative from week six onwards, and again the reported infection rate for the
unvaccinated is overestimated from the start of the vaccination campaign. However, by the end of the
campaign the reported infection rates for the vaccinated would be greater than that for the
unvaccinated.
Scenarios C and D are simply the same as scenarios A and B, except for the fact that they are for the
excluded type, (c), of selection bias. Note that here the reported infection rate for the vaccinated
remains unbiased whilst that for the vaccinated rises to match the true rate for the placebo and
negative efficacy scenarios.
In Scenario E, boosting with miscategorisation, (a), we can see that repeated application of the
vaccine at twelve-week intervals restores vaccine efficacy to high levels after each booster and,
assuming a constant infection rate, elevates the reported infection rate in the unvaccinated cohort
between each booster campaign, giving rise to bias and gross overestimation.
Our simulation model has demonstrated that the effects of this selection bias are to artificially boost
vaccine efficacy in all cases, and with the application of repeated ‘booster’ vaccinations, the efficacy
of repeated Covid-19 vaccines could be maintained at these artificial levels in perpetuity should
boosting be continued indefinitely. Furthermore, in tandem with this the infection rate is likewise
5

artificially elevated for the unvaccinated cohort compared to the vaccinated cohort, further
compounding false claims that a Covid-19 vaccine reduces infection rates. Note that other metrics of
vaccine effectiveness, such as mortality or morbidity improvements, are capable of being mis-reported
in a similar way because of the same bias.
6. Conclusions
Our reviews reveals that a serious form of selection bias, miscategorisation, is pervasive throughout
the many research studies that aim to measure Covid-19 vaccine efficacy. The effect of this bias is to
artificially inflate vaccine efficacy and present the misleading impression that these vaccines are
effective and that the non-vaccinated suffer from higher Covid-19 infection rates compared to the
vaccinated.
We presented a simulation model to demonstrate the effects of this selection bias and show it
artificially boosts vaccine efficacy in all cases, and with the application of repeated ‘booster’
vaccinations, the efficacy of repeated Covid-19 vaccines could be maintained at artificial levels in
perpetuity should boosting be continued indefinitely. This effect occurs with a both a zero-efficacy
(placebo) vaccine and a negative-efficacy vaccine that increases, rather than reduces, infection rates
in those vaccinated.
This miscategorisation is guaranteed to lead to initially very high efficacy claims (usually above 90%)
during peak vaccine rollout even if the vaccine were a placebo or worse. Efficacy then falls toward
zero a few weeks later. This pattern of high initial efficacy, tapering off after 3 months is also
consistently observed in real-world studies, and is often used as justification for additional, booster
vaccinations to maintain efficacy. The corresponding Covid-19 infection rate is also likewise artificially
elevated in the unvaccinated cohort compared to the vaccinated cohort. These issues apply to other
measures of vaccination effectiveness related to mortality and morbidity.
Thus, we conclude that any claims of Covid-19 vaccine efficacy based on these studies are likely to
be a statistical illusion.
References
Brown, R. B. (2021). Outcome reporting bias in COVID-19 mRNA vaccine clinical trials. Medicina,
57(3), 199.
Doll, M. K., Pettigrew, S. M., Ma, J., & Verma, A. (2022). Effects of confounding bias in coronavirus
disease 2019 (COVID-19) and influenza vaccine effectiveness test-negative designs due to
correlated influenza and COVID-19 vaccination behaviors. Clinical Infectious Diseases, 75(1),
e564-e571.
Elliot, C. (2014). Relationships between physicians and Pharma: Why physicians should not accept
money from the pharmaceutical industry. Neurology Clinical Practice, 4(2).
https://doi.org/10.1212%2FCPJ.0000000000000012
Eusebi, P., Speybroeck, N., Hartnack, S., Stærk-Østergaard, J., Denwood, M. J., & Kostoulas, P.
(2023). Addressing misclassification bias in vaccine effectiveness studies with an application
to Covid-19. BMC Medical Research Methodology, 23(1), 55.
Fenton, N. & Neil, M. (2023). The Very Best of Cheap Trick. Where are the Numbers. Last accessed:
February 20, 2023. Sourced from: https://wherearethenumbers.substack.com/p/the-very-best-
of-cheap-trick
Fung, K., Jones, M., & Doshi, P. (2023). Sources of bias in observational studies of covid‐19 vaccine
effectiveness. Journal of Evaluation in Clinical Practice.
Glasziou, P., McCaffery, K., Cvejic, E., Batcup, C., Ayre, J., Pickles, K., & Bonner, C. (2022). Testing
behaviour may bias observational studies of vaccine effectiveness. Official Journal of the
Association of Medical Microbiology and Infectious Disease Canada, 7(3), 242-246.
Haven, T., Gopalakrishna, G., Tijdink, J., van der Schot, D., & Bouter, L. (2022). Promoting trust in
research and researchers: How open science and research integrity are intertwined. BMC
Research Notes, 15(1), 302.
Hitchings, M. D., Lewnard, J. A., Dean, N. E., Ko, A. I., Ranzani, O. T., Andrews, J. R., & Cummings,
D. A. (2022). Use of recently vaccinated individuals to detect bias in test-negative case–
6

control studies of COVID-19 vaccine effectiveness. Epidemiology (Cambridge, Mass.), 33(4),
450.
Ioannidis, J. P. (2022). Factors influencing estimated effectiveness of COVID-19 vaccines in non-
randomised studies. BMJ Evidence-Based Medicine, 27(6), 324-329.
Kobayashi, Y., Howell, C., & Heinrich, T. (2021). Vaccine hesitancy, state bias, and Covid-19:
Evidence from a survey experiment using Phase-3 results announcement by BioNTech and
Pfizer. Social Science & Medicine, 282, 114115.
Lataster, R. (2024). How the adverse effect counting window affected vaccine safety calculations in
randomised trials of COVID‐19 vaccines. Journal of Evaluation in Clinical Practice.
Malthouse, E. (2023). Confirmation bias and vaccine-related beliefs in the time of COVID-19. Journal
of Public Health, 45(2), 523-528.
Ortiz-Brizuela, E., Carabali, M., Jiang, C., Merckx, J., Talbot, D., & Schnitzer, M. E. (2023). Potential
biases in test-negative design studies of COVID-19 vaccine effectiveness arising from the
inclusion of asymptomatic individuals. medRxiv, 2023-11.
Saul, S. (2008). Experts conclude Pfizer manipulated studies. New York Times (8 October, 2008).
Last accessed: 28 December, 2023. Sourced from:
https://www.nytimes.com/2008/10/08/health/research/08drug.html
Shienke, E. (2023). Bioethics 533: Ethical dimensions of renewable energy and sustainable systems -
2.1 Falsification, Fabrication, Plagiarism. Penn State University. Last accessed: 28
December, 2023. Sourced from: https://www.e-education.psu.edu/bioet533/node/654
Weinstein, B. & Heying. H. (2023). Science Strong and Fragile. Dark Horse Podcast. Last accessed:
February 20, 2024. Sourced from: https://www.youtube.com/watch?v=3AWOHZ2Y_KQ
(Starting at: 31:30)
Williams, L. R., Ferguson, N. M., Donnelly, C. A., & Grassly, N. C. (2022). Measuring Vaccine Efficacy
Against Infection and Disease in Clinical Trials: Sources and Magnitude of Bias in
Coronavirus Disease 2019 (COVID-19) Vaccine Efficacy Estimates. Clinical Infectious
Diseases, 75(1), e764-e773. https://onlinelibrary.wiley.com/doi/10.1111/jep.13839
7

Appendix: Research studies containing miscategorisation as a
selection bias
Al Kaabi, N., Yang, Y., Eldin Hussein, S., Yang, T., Abdalla, J., Wang, H., ... & Yang, X. (2023). Efficacy and
Safety of a Booster Vaccination with Two Inactivated SARS-CoV-2 Vaccines on Symptomatic COVID-19
Infection in Adults: Results of a Double-Blind, Randomized, Placebo-Controlled, Phase 3 Trial in Abu
Dhabi. Vaccines, 11(2), 299.
Ali, K., Berman, G., Zhou, H., Deng, W., Faughnan, V., Coronado-Voges, M., ... & McPhee, R. (2021). Evaluation
of mRNA-1273 SARS-CoV-2 vaccine in adolescents. New England Journal of Medicine, 385(24), 2241-
2251.
Andrews, N., Tessier, E., Stowe, J., Gower, C., Kirsebom, F., Simmons, R., ... & Lopez Bernal, J. (2022).
Duration of protection against mild and severe disease by Covid-19 vaccines. New England Journal of
Medicine, 386(4), 340-350.
Áñez, G., Dunkle, L. M., Gay, C. L., Kotloff, K. L., Adelglass, J. M., Essink, B., ... & Corey, L. (2023). Safety,
Immunogenicity, and Efficacy of the NVX-CoV2373 COVID-19 Vaccine in Adolescents: A Randomized
Clinical Trial. JAMA Network Open, 6(4), e239135-e239135.
Angel, Y., Spitzer, A., Henig, O., Saiag, E., Sprecher, E., Padova, H., & Ben-Ami, R. (2021). Association between
vaccination with BNT162b2 and incidence of symptomatic and asymptomatic SARS-CoV-2 infections
among health care workers. Jama, 325(24), 2457-2465.
Arbel, R., Wolff Sagy, Y., Hoshen, M., Battat, E., Lavie, G., Sergienko, R., ... & Netzer, D. (2022). Nirmatrelvir use
and severe Covid-19 outcomes during the Omicron surge. New England Journal of Medicine, 387(9),
790-798.
Bermingham, C., Gethings, O., & Nafilyan, V. (2023). Covid-19 vaccine effectiveness estimated using census
2021 variables, England: 31 March 2021 to 20 March 2022. Last accessed: February 5, 2023. Sourced
from: https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/
bulletins/covid19vaccineeffectivenessestimatedusingcensus2021variablesengland/
31march2021to20march2022
Bermingham, C., Nafilyan, V., Andrews, N., & Gethings, O. (2023b). Estimating the effectiveness of COVID-19
vaccination against COVID-19 hospitalisation and death: a cohort study based on the 2021 Census,
England. medRxiv, 2023-06.
Baum, U., Poukka, E., Leino, T., Kilpi, T., Nohynek, H., & Palmu, A. A. (2022). High vaccine effectiveness against
severe COVID-19 in the elderly in Finland before and after the emergence of Omicron. BMC Infectious
Diseases, 22(1), 1-9.
Buchan, S. A., Chung, H., Brown, K. A., Austin, P. C., Fell, D. B., Gubbay, J. B., ... & Kwong, J. C. (2022).
Estimated effectiveness of COVID-19 vaccines against Omicron or Delta symptomatic infection and
severe outcomes. JAMA network open, 5(9), e2232760-e2232760.
Carazo, S., Skowronski, D. M., Brisson, M., Barkati, S., Sauvageau, C., Brousseau, N., ... & De Serres, G.
(2022). Protection against Omicron BA. 2 reinfection conferred by primary Omicron or pre-Omicron
infection with and without mRNA vaccination. medRxiv, 2022-06.
Chung, H., Austin, P. C., Brown, K. A., Buchan, S. A., Fell, D. B., Fong, C., ... & Kwong, J. C. (2022, September).
Effectiveness of COVID-19 vaccines over time prior to omicron emergence in Ontario, Canada: test-
negative design study. In Open Forum Infectious Diseases (Vol. 9, No. 9, p. ofac449). US: Oxford
University Press.
Dagan, N., Barda, N., Kepten, E., Miron, O., Perchik, S., Katz, M. A., ... & Balicer, R. D. (2021). BNT162b2 mRNA
Covid-19 vaccine in a nationwide mass vaccination setting. New England Journal of Medicine, 384(15),
1412-1423.
Ferdinands, J. M., Rao, S., Dixon, B. E., Mitchell, P. K., DeSilva, M. B., Irving, S. A., ... & Fireman, B. (2022).
Waning of vaccine effectiveness against moderate and severe Covid-19 among adults in the US from
the VISION network: test negative, case-control study. bmj, 379.
Haas, E. J., Angulo, F. J., McLaughlin, J. M., Anis, E., Singer, S. R., Khan, F., ... & Alroy-Preis, S. (2021). Impact
and effectiveness of mRNA BNT162b2 vaccine against SARS-CoV-2 infections and COVID-19 cases,
hospitalisations, and deaths following a nationwide vaccination campaign in Israel: an observational
study using national surveillance data. The Lancet, 397(10287), 1819-1829.
Heath, P. T., Galiza, E. P., Baxter, D. N., Boffito, M., Browne, D., Burns, F., ... & Toback, S. (2023). Safety and
efficacy of the NVX-CoV2373 Coronavirus disease 2019 vaccine at completion of the placebo-controlled
phase of a randomized controlled trial. Clinical Infectious Diseases, 76(3), 398-407.
Khairullin, B., Zakarya, K., Orynbayev, M., Abduraimov, Y., Kassenov, M., Sarsenbayeva, G., ... & Kutumbetov, L.
(2022). Efficacy and safety of an inactivated whole-virion vaccine against COVID-19, QazCovid-in®, in
healthy adults: a multicentre, randomised, single-blind, placebo-controlled phase 3 clinical trial with a 6-
month follow-up. EClinicalMedicine, 50.
Kitano, T., Thompson, D. A., Engineer, L., Dudley, M. Z., & Salmon, D. A. (2023). Risk and benefit of mRNA
COVID-19 vaccines for the omicron variant by age, sex, and presence of comorbidity: a quality-adjusted
life years analysis. American Journal of Epidemiology, kwad058.
8

Liu, B., Stepien, S., Dobbins, T., Gidding, H., Henry, D., Korda, R., ... & Macartney, K. (2023). Effectiveness of
COVID-19 vaccination against COVID-19 specific and all-cause mortality in older Australians: a
population-based study. The Lancet Regional Health–Western Pacific, 40.
Lyngse, F. P., Mølbak, K., Denwood, M., Christiansen, L. E., Møller, C. H., Rasmussen, M., ... & Kirkeby, C. T.
(2022). Effect of vaccination on household transmission of SARS-CoV-2 Delta variant of concern.
Nature Communications, 13(1), 3764.
Lyngse, F. P., Mortensen, L. H., Denwood, M. J., Christiansen, L. E., Møller, C. H., Skov, R. L., ... & Kirkeby, C.
T. (2022). Household transmission of the SARS-CoV-2 Omicron variant in Denmark. Nature
Communications, 13(1), 5573.
Mitchell, R., Cayen, J., Thampi, N., Frenette, C., Bartoskzo, J., Choi, K. B., ... & Pelude, L. (2023). Trends in
severe outcomes among adult and pediatric patients hospitalized with COVID-19 in the Canadian
Nosocomial Infection Surveillance Program, March 2020 to May 2022. JAMA Network Open, 6(4),
e239050-e239050.
Muñoz, F. M., Sher, L. D., Sabharwal, C., Gurtman, A., Xu, X., Kitchin, N., ... & Gruber, W. C. (2023). Evaluation
of BNT162b2 Covid-19 vaccine in children younger than 5 years of age. New England Journal of
Medicine, 388(7), 621-634.
Nadeem, I., Ul Munamm, S. A., Ur Rasool, M., Fatimah, M., Abu Bakar, M., Rana, Z. K., ... & McLoughlin, H.
(2023). Safety and efficacy of Sinopharm vaccine (BBIBP-CorV) in elderly population of Faisalabad
district of Pakistan. Postgraduate medical journal, 99(1171), 463-469.
Nordström, P., Ballin, M., & Nordström, A. (2022). Risk of infection, hospitalisation, and death up to 9 months
after a second dose of COVID-19 vaccine: a retrospective, total population cohort study in Sweden. The
Lancet, 399(10327), 814-823.
NSW Health, (2021). Vaccination among Covid-19 cases in the NSW Delta outbreak. Reporting period: 16 June
to 7 October 2021. Last accessed: February 5, 2024. Sourced from:
https://www.health.nsw.gov.au/Infectious/covid-19/Documents/in-focus/covid-19-vaccination-case-
surveillance-051121.pdf
Pálinkás, A., & Sándor, J. (2022). Effectiveness of COVID-19 vaccination in preventing all-cause mortality among
adults during the third wave of the epidemic in hungary: nationwide retrospective cohort study. Vaccines,
10(7), 1009.
Paternina-Caicedo, A., Jit, M., Alvis-Guzman, N., Fernandez, J. C., Hernandez, J., Paz-Wilches, J. J., ... & De La
Hoz-Restrepo, F. (2022). Effectiveness of CoronaVac and BNT162b2 COVID-19 mass vaccination in
Colombia: A population-based cohort study. The Lancet Regional Health–Americas, 12.
Petráš, M., Lesná, I. K., Večeřová, L., Nyčová, E., Malinová, J., Klézl, P., ... & Adámková, V. (2021). The
effectiveness of post-vaccination and post-infection protection in the hospital staff of three Prague
hospitals: a cohort study of 8-month follow-up from the start of the COVID-19 Vaccination Campaign
(COVANESS). Vaccines, 10(1), 9.
Pilishvili, T., Gierke, R., Fleming-Dutra, K. E., Farrar, J. L., Mohr, N. M., Talan, D. A., ... & Schrag, S. J. (2021).
Effectiveness of mRNA Covid-19 vaccine among US health care personnel. New England Journal of
Medicine, 385(25), e90.
Polack, F. P., Thomas, S. J., Kitchin, N., Absalon, J., Gurtman, A., Lockhart, S., ... & Gruber, W. C. (2020). Safety
and efficacy of the BNT162b2 mRNA Covid-19 vaccine. New England journal of medicine, 383(27),
2603-2615.
Reeder, M. (2021) Use of a null assumption to re-analyze data collected through a rolling cohort subject to
selection bias due to informative censoring. Unpublished manuscript.
https://zenodo.org/records/5243901
Robles-Fontán, M. M., Nieves, E. G., Cardona-Gerena, I., & Irizarry, R. A. (2022). Effectiveness estimates of
three COVID-19 vaccines based on observational data from Puerto Rico. The Lancet Regional Health–
Americas, 9.
Rosenberg, E. S., Dorabawila, V., Easton, D., Bauer, U. E., Kumar, J., Hoen, R., ... & Zucker, H. A. (2022).
Covid-19 vaccine effectiveness in New York state. New England Journal of Medicine, 386(2), 116-127.
Stock, S. J., Carruthers, J., Calvert, C., Denny, C., Donaghy, J., Goulding, A., ... & Wood, R. (2022). SARS-CoV-
2 infection and COVID-19 vaccination rates in pregnant women in Scotland. Nature medicine, 28(3),
504-512.
Tabarsi, P., Anjidani, N., Shahpari, R., Mardani, M., Sabzvari, A., Yazdani, B., ... & Barati, S. (2023). Evaluating
the efficacy and safety of SpikoGen®, an Advax-CpG55. 2–adjuvanted severe acute respiratory
syndrome coronavirus 2 spike protein vaccine: a phase 3 randomized placebo-controlled trial. Clinical
Microbiology and Infection, 29(2), 215-220.
Tan, W. C., Tan, J. Y. J., Lim, J. S. J., Tan, R. Y. C., Lee, A. R. Y. B., Leong, F. L., ... & Tan, K. B. (2023).
COVID-19 Severity and Waning Immunity After up to 4 mRNA Vaccine Doses in 73 608 Patients With
Cancer and 621 475 Matched Controls in Singapore: A Nationwide Cohort Study. JAMA oncology, 9(9),
1221-1229.
Thomas SJ, Kitchin N, et al. (2021) Safety and Efficacy of the BNT162b2 mRNA 404. Covid-19 Vaccine. N Engl J
Med. 383(27):2603-2615. 405
Wu, Q., Tong, J., Zhang, B., Zhang, D., Chen, J., Lei, Y., ... & Chen, Y. (2024). Real-World Effectiveness of
BNT162b2 Against Infection and Severe Diseases in Children and Adolescents. Annals of Internal
Medicine.
9

Yau, J. W. K., Lee, M. Y. K., Lim, E. Q. Y., Tan, J. Y. J., Tan, K. B. J. C., & Chua, R. S. B. (2023). Genesis,
evolution and effectiveness of Singapore's national sorting logic and home recovery policies in handling
the COVID-19 Delta and Omicron waves. The Lancet Regional Health–Western Pacific.
10