Content uploaded by Ronald Meester

Author content

All content in this area was uploaded by Ronald Meester on Dec 14, 2021

Content may be subject to copyright.

COVID-19 vaccinations and mortality - a Bayesian analysis

Ronald Meester, Wouter Aukema, and Theo Schetters

Summary

In this note, we report about the results of a statistical investigation into the relation

between vaccination against COVID-19 and the mortality rates in the three weeks

after vaccination, in the age groups 12 - 65, 65 - 80 and 80+ respectively in The

Netherlands. We investigate whether the observed mortality during this period can

be better explained by the expected mortality as computed similarly to the method of

the CBS (Statistics Netherlands), or by a scenario in which vaccination leads to a

small increase in the mortality in the three weeks following the vaccination. We found

that in the cohort 12 - 65, the two scenarios predict the actual data roughly equally

well, but for the cohort 65 - 80, the second scenario explains the data much better. In

the 80+ cohort, there is a difference between men and women: for men, the second

scenario explains the data much better, but for women the contrary is the case. We

discuss the interpretation of these results, and conclude that they urgently call for

further investigations into the effects of vaccination, and publication of the possible

risks of vaccination against COVID-19.

Introduction

It is clear that a statistical investigation into the relation between the mass

vaccination program in The Netherlands and the mortality rate is difficult and subtle.

The Netherlands has seen very high mortality rates in the fall of 20211, but the

reason for this is not so easy to understand. Various explanations could be offered:

COVID-19 mortality, delayed health care, the effect of mass vaccination, aging, and

probably other ones as well. A statistical analysis of these high mortality rates

requires more detailed information on the individual level – information which is not

(yet) available.

1https://www.cbs.nl/nl-nl/reeksen/sterfte-per-week/

There is, however, another relation between vaccination and mortality that can in fact

be statistically investigated without detailed individual information, namely the

mortality rates directly following vaccination. There is detailed information about the

number of vaccinations per week for each age group, and together with the observed

mortality per week per age group, this enables us to statistically investigate the direct

effect of vaccination on the mortality rates, without the results being confounded by

other effects. This analysis is carried out in the present paper.

Methods

Population data for the period 2015 - 2021 was downloaded from Eurostat.2

Mortality data for the period 2015 to 2021 (week 41) was also downloaded from

Eurostat.3Expected mortality was calculated using the same method as described

by the CBS.4Vaccination data was obtained from the RIVM (National Institute for

Public Health and the Environment).5

We performed a Bayesian analysis. In such an analysis, one computes the

probability of the actual outcome under each of the two hypotheses. If one

hypothesis assigns a (much) higher probability to the outcome than the other, then

the first one explains the data (much) better than the second one and is therefore a

(much) better explanation than the other hypothesis. The ratio of the probabilities of

the outcome under each of the scenarios is called the likelihood ratio (LR): it conveys

which of the two hypotheses explains the data best, and by what factor.

Here is a very simple example of using this method that might help understanding

the method. Suppose you are uncertain about the fairness of a coin. You consider

two hypotheses: the first hypothesis is that the coin is fair, the second hypothesis is

that the coin is biased in a 40-60 way. Now we flip the coin 100 times. If 51 heads

5https://www.rivm.nl/covid-19-vaccinatie/cijfers-vaccinatieprogramma. Percentage uptake for

each age-group was equally distributed along the corresponding dimensions for sex,

age-group and time, and recorded as the number of doses administered for dosis one and

two.

4https://www.cbs.nl/en-gb/news/2020/49/higher-mortality-in-week-48/expected-mortality. We

chose not to apply the adjustments CBS made internally since details of how these

adjustments were applied by CBS across the age-group, sexe and time dimension, were not

available.

3https://ec.europa.eu/eurostat/databrowser/view/demo_r_mweek3/default/table?lang=en.

2https://ec.europa.eu/eurostat/databrowser/view/DEMO_PJAN/default/table?lang=en.

come up, then this is much more likely under the first than under the second

hypothesis, and hence this is evidence in favor of the first. If 65 heads come up then

this is much more likely under the second than under the first hypothesis, and hence

this is evidence in favor of the second. In the former case, the LR will be larger than

1, in the latter it will be smaller.

The evidence in a Bayesian approach should always be understood in the context.

Without going into the mathematics, strong evidence in favor of a hypothesis does

not necessarily imply that you are convinced that this hypothesis describes the truth

well. An outrageous hypothesis requires more evidence to become believable than a

hypothesis that one deems believable from the outset. We will come back to this in

the discussion.

The hypotheses

Here is the first of our hypotheses:

CBS Hypothesis: Each week, the probability of death for every person in the

population is calculated based on the expected number of deaths similar to the CBS.

You could interpret this as the ‘baseline’ scenario. In our calculations, we distinguish

between the sexes, something which turned out to be relevant only for the 80+

cohort.

The alternative hypothesis states that someone who is vaccinated has an increased

chance of dying in the three weeks after vaccination. We increased the probability for

the three weeks after vaccination by 10% compared to the probability coming from

the CBS hypothesis. Both the period (three weeks) and the extra opportunity (10%)

are of course flexible, and can also be chosen differently. We carried out a sensitivity

analysis for these two choices, and the results were very stable for large ranges of

other possible choices.

Increased Risk Hypothesis: A person who is vaccinated in a certain week has a

mortality probability in the following three weeks that is 10% per week larger than in

the CBS Hypothesis. After that period of three weeks, this person has the CBS

mortality probability again.

Results

Age group 12 - 65: in this age group, there seems to be no discernable direct

mortality effect of vaccination; see Figure 1.

Figure 1. See the text for an explanation.

We explain the figure now. In the first column, we have written the week numbers of

2021. In the rightmost column we have written the number of vaccinations (including

both first and second dose) in this cohort in the specific week, and we have

visualized this with the blue bars. The second and third column contain the LR per

week, both numerically and visualized. A LR larger than 1 favors the increased risk

hypothesis. A red bar indicates that the evidence favors Increased Risk, and a green

bar that it favors the CBS Hypothesis. The length of the bars reflects the logarithm of

the LR. The overall LR for this cohort is around 11, so the two hypotheses explain

the data roughly equally well, with a slight advantage for the Increased Risk

Hypothesis. Note that the red and green bars do not represent numbers of deaths;

they only indicate which hypothesis explains the data best and by what factor. We

have also made computations for both sexes separately, but there were no relevant

differences between these two groups.

Age group 65 - 80: in this age group, the situation is very different than in the

previous one; see Figure 2.

Figure 2. See the text for an explanation.

We see that the Increased Risk Hypothesis explains the data much better than the

CBS Hypothesis. The data is overall much more likely under the Increased Risk

Hypothesis than under the CBS Hypothesis. The overall LR is incredibly large, and

hence the evidence very strongly favors the Increased Risk Hypothesis. We remark

that the scale of the visualization of the LR is different than in Figure 1. The

computed LR’s are much, much larger here.

Age group 80+: in this age group there is a remarkable difference between the

sexes, see Figure 3.

Figure 3. See the text for an explanation.

Discussion

A Bayesian analysis constitutes a comparison between two hypotheses. A large LR

does not necessarily imply that the favored one is likely to be the ‘correct’ one: bóth

hypotheses could be far from the truth. If we compare two false hypotheses, this can

nevertheless result in a high LR favoring one of them, compared to the other.

Furthermore, a LR should be understood in context. A large LR favoring a certain

hypothesis does not make that hypothesis immediately credible. If the hypothesis

was considered very unlikely prior to the evidence, then a LR will typically make it

more credible, but not necessarily to the extent that the hypothesis becomes

believable or likely to be true.

In our situation, the two competing hypotheses only differ around the time of

vaccination of the relevant cohort. Therefore, away from the vaccination period, the

hypotheses predict the same, and the data will not be distinguishing between them.

This is confirmed in all the figures above, where the LR is essentially 1 outside the

vaccination weeks.

Figure 1 shows that the data does not distinguish strongly between the two

hypotheses of interest for the 12 - 65 cohort. This is to be expected, since the

mortality rate in the cohort is low in any case. We find it reassuring that the weekly

LR’s do not consistently point in either direction, and variations can be due to chance

fluctuations. In other words, the CBS Hypothesis seems to be in good agreement

with the data, and the slightly larger mortality rate under the Increased Risk

Hypothesis performs only slightly better.

In Figure 2, for the 65 - 80 cohort, the situation is dramatically different. The

Increased Risk Hypothesis outperforms the CBS Hypothesis by many orders of

magnitude. Clearly, the actual mortality was higher than was predicted by the CBS,

and the Increased Risk Hypothesis explains the data much, much better than the

CBS Hypothesis.

Is there a way to say anything meaningful about the prior belief in the Increased Risk

Hypothesis? We have received reports about adverse events following vaccination,

but to the best of our knowledge, no detailed study of this has been carried out. On

the other hand, Martin Neil, Norman Fenton and their co-authors have reached the

conclusion that there might be a serious direct mortality effect of vaccination.6

Hence, the hypothesis that vaccination comes with a certain risk is not outrageous at

all. The reported overall LR, together with a certain prior plausibility, makes the

Increased Risk Hypothesis rather credible for this cohort. It seems, therefore, that

the public should be informed about the plausibility of this hypothesis, something

which has not been done so far.

We finally mention that in the 65 - 80 cohort, no serious differences were observed

when we distinguished between the sexes.

For the 80+ cohort, things are different. For women, the data strongly favors the CBS

Hypothesis, but for men it favors the Increased Risk Hypothesis. Part of an

explanation could be that over 2021, the mortality rate among women was very low,

possibly partly due to the higher mortality rate during the first waves of the pandemic.

In any case, we do not seem to be able to draw strong conclusions from our

outcomes for this cohort.

We plan to perform a similar analysis to investigate the protection effect of

vaccinations in the near future. Since our population data comes from Eurostat, we

invite our colleagues to perform a similar analysis in other countries as well. Finally,

our spreadsheet is available upon request.

Conclusion

Our results for the 65 - 80 cohort strongly suggest that further investigation into the

direct mortality effects of COVID-19 vaccinations is urgently called for, and that the

current mass vaccination campaign should not be continued without proper public

information about the risks.

6Martin Neil, Norman Fenton, Joel Smalley, Clare Craig, Joshua Guetzkow, Scott McLachlan,

Jonathan Engler and Jessica Rose (2021), “Latest statistics on England mortality data suggest

systematic miscategorisation of vaccine status and uncertain effectiveness of Covid-19 vaccination”,

http://dx.doi.org/10.13140/RG.2.2.14176.20483.

Appendix

All data points were collected into an Excel spreadsheet where we developed a

custom function (using Excel VBA) to calculate probability under the Increased Risk

Hypothesis as follows:

Function pdf(deaths, population, expected, vaccinated, added_risc) As Double

vpdf = 0

If vaccinated = 0 Then

vpdf = WorksheetFunction.Binom_Dist(deaths, population, expected / population,

False)

GoTo EndFunction

End If

If vaccinated >= population Then

vpdf = WorksheetFunction.Binom_Dist(deaths, vaccinated, _

(expected / population) * (1 + added_risc), False)

GoTo EndFunction

End If

For n = 0 To deaths

U = WorksheetFunction.Binom_Dist(deaths - n, population - vaccinated, expected /

population, False) 'Unvaccinated Population

V = WorksheetFunction.Binom_Dist(n, vaccinated, (expected / population) _

* (1 + added_risc), False) 'Vaccinated Population

vpdf = vpdf + V * U

Next n

EndFunction:

pdf = vpdf

End Function