PreprintPDF Available

Discrepancies and inconsistencies in UK Government datasets compromise accuracy of mortality rate comparisons between vaccinated and unvaccinated

Authors:
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

UPDATE: A significantly revised version of this report is here:http://www.eecs.qmul.ac.uk/~norman/papers/inconsistencies_vaccine.pdf To determine the overall risk-benefit of Covid-19 vaccines it is crucial to be able to compare the all-cause mortality rates between the vaccinated and unvaccinated in each different age category. However, current publicly available UK Government statistics do not include raw data on mortality by age category and vaccination status. Hence, we are unable to make the necessary comparison. In attempting to reverse engineer estimates of mortality by age category and vaccination status from the various relevant public Government datasets we found numerous discrepancies and inconsistencies which indicate that the Office for National Statistics reports on vaccine effectiveness are grossly underestimating the number of unvaccinated people. Hence, official statistics may be underestimating the mortality rates for vaccinated people in each age category. Although we have not subjected this data to statistical testing the potential implications of these results on the effects of vaccination on all-cause mortality, and by implication, the future of the vaccination programme is profound
Content may be subject to copyright.
1
Discrepancies and inconsistencies in UK Government datasets compromise
accuracy of mortality rate comparisons between vaccinated and
unvaccinated
Martin Neil, Norman Fenton and Scott McLachlan
Queen Mary, University of London, UK
20 October 2021
Abstract
To determine the overall risk-benefit of Covid-19 vaccines it is crucial to be able to
compare the all-cause mortality rates between the vaccinated and unvaccinated in
each different age category. However, current publicly available UK Government
statistics do not include raw data on mortality by age category and vaccination status.
Hence, we are unable to make the necessary comparison. In attempting to reverse
engineer estimates of mortality by age category and vaccination status from the
various relevant public Government datasets we found numerous discrepancies and
inconsistencies which indicate that the Office for National Statistics reports on vaccine
effectiveness are grossly underestimating the number of unvaccinated people. Hence,
official statistics may be underestimating the mortality rates for vaccinated people in
each age category. Although we have not subjected this data to statistical testing the
potential implications of these results on the effects of vaccination on all-cause
mortality, and by implication, the future of the vaccination programme is profound
Introduction
In a previous article
1
we argued that the overall risk/benefit of vaccines was best measured by
comparing all-cause mortality between the vaccinated and unvaccinated. A simple summary of our
arguments for this is provided in the Appendix. Our article reviewed the Public Health England
(PHE)/Office for National Statistics (ONS) data on age-adjusted mortality rates by vaccination status
[1]. In the most recent week of that report (week 26, ending 2 July 2021) the ONS provided the data,
which is shown in Table 1, along with the ASMR (age standardized mortality rate) that is derived from
it.
Table 1 All cause deaths by vaccination status (week 26)
Population
size
Total all-cause
deaths
Unadjusted
mortality rate
(UMR)
Age standardized
mortality rate
(ASMR)*
Unvaccinated
9,531,364
436
4.57
25.3
One dose
6,404,395
576
8.99
89.3
Two doses
23,309,568
5944
25.5
14.7
Total
39,245,327
6,956
17.72
-
* Note that the ASMR here is derived by weighting the ASMR values for Covid-19 (Table 4) and Non Covid-19 deaths (Table 5) from [1]
1
https://probabilityandlaw.blogspot.com/2021/09/all-cause-mortality-rates-in-england.html
2
The ‘unadjusted’ mortality rate (UMR) is simply the number of deaths per 100K people (i.e., we divide
the total all-cause deaths by the population size and multiply by 100K). The ASMR [2] is a derived
metric to adjust for the different age distributions among the different groups (the ONS did not
provide the ASMR for the total population, hence the blank entry in the final column of Table 1)
The total UMR of 17.72 for week 26 represents a worrying increase of between 15-18% mortality
compared with the same week over the previous decade when the average UMR in England for all-
cause mortality was 15.47 (with a min/max range 14.88-15.84) [11]. This is despite just 109 (less than
1.6% of the total 6956 deaths) being classified as Covid-19.
The large differences both within and between the UMR and ASMR numbers in Table 1 seem highly
counter-intuitive and, while some differences have obvious explanations, others do not. Specifically:
The unvaccinated group has the lowest UMR (five times lower than the rate for people with
two doses), but the two-dose vaccinated group has the lowest ASMR. As explained in our
previous article, this ‘reversal’ is an example of Simpson’s paradox and is explained by the fact
that most deaths occur in the older age categories which are also the categories with the
highest percentage of double vaccinated people.
While the ASMR for the two-dose vaccinated group is lower than the UMR, the one-dose
vaccinated group ASMR is ten times higher than the equivalent UMR. This is hard to explain,
as is the extremely high ASMR for those one-dose vaccinated.
The total population size of 39,245,327 in the ONS/PHE mortality by vaccination status report
[1] is supposed to include everybody in England aged at least 10 but is at least 10 million fewer
than that estimated by ONS in the 2011 census [9] and in their 2021 population estimate [10].
This is partly explained in [9], but we provide further insights based on direct communication
with ONS in Appendix B.
While the ASMR can be useful in many epidemiological and medical contexts, we believe it is both
unnecessarily complex and somewhat redundant in this context. The ASMR maps any population
onto a notional European standard age population profile [2], and its calculation depends on the
population size and number of deaths in each of a full range of age stratification categories for each
vaccination category. The fundamental problem we noted in our article was that the ONS did not
provide this raw data and so it was therefore impossible to verify their ASMR calculations.
If we had the raw age-categorized data we would be able to simply compare, for each age category
and week, the all-cause mortality rate for vaccinated and unvaccinated. This would make the ASMR
redundant and allow the direct comparison we seek. The ONS have told us in direct communications
that release of this age-categorized data is planned for future versions of the vaccination status
reports and have committed to make this release within the next three weeks.
Discrepancies and Inconsistencies in ONS datasets
While the data are not yet directly available, we believed it should be possible to reverse engineer
reasonably accurate mortality estimates for the individual age categories, by vaccination status, by
stitching together data available from various ONS sources.
However, it turns out that this reverse engineering from other data sources is not realistically possible
given that there are fundamental discrepancies and inconsistencies between the various relevant ONS
sources of data a problem that has been highlighted in [8] and which we discuss further below. Of
most concern is the observation that the ONS data may significantly underestimate the total
population of unvaccinated people. This means that, even when in future the ONS releases the age-
3
categorized mortality data, it is likely that in many age categories the mortality rate for the
unvaccinated will be overestimated (since the ‘denominator’ will be lower than it should be). This
also means that the mortality rates presented in Table 1 are likely to be exaggerating the unvaccinated
mortality rate (both for UMR and ASMR figures).
There are four relevant datasets that we considered, all of which are publicly available online:
PHE/ONS data on age-adjusted mortality rates by vaccination status (PHE/ONS mortality)
[1].
NIMS (National Immunisation Management Service), national flu and COVID-19 surveillance
reports 01 July 2021 Week 26 (NIMS vaccination survey’) [3].
ONS population estimates for the UK, England and Wales, Scotland and Northern Ireland: mid-
2020 (ONS population survey’) [4].
Deaths registered weekly in England and Wales by age and sex from ONS. (ONS registered
deaths’) [5].
Discrepancies and inconsistencies in the data are identified as follows (bearing in mind that the
PHE/ONS mortality data has been restricted to England only):
According to the NIMS vaccination survey the population of England is 61,941,471, whereas
the ONS population survey estimate is 56,550,138. The PHE/ONS mortality reports an even
lower estimate still (see below). Inevitably, these differences lead to very different estimates
of the crucial total number of people in each age category. The inaccuracy of NIMS population
data is noted in [8] and to remedy it they argue that ONS population estimates should be
preferred.
Because the PHE/ONS mortality report omits children under the age of 10, we must restrict
our analysis to the remaining population of England. According to the ONS population survey
this sub-population totals 49,771,233. However, in the latest week of the PHE/ONS mortality
report we considered (week 26 ending 2 July 2021) this sub-population was recorded as just
39,245,327 (this is the total of all unvaccinated plus all categories of vaccinated). Given that
NIMS vaccine survey estimates an even higher population than in the PHE/ONS mortality
report, this means the PHE/ONS mortality report data is missing at least 10 million people
(49,771,233 - 39,245,327 = 10,525,906). Appendix B explains why the ONS estimate is so low
and how this may result in systematic biases in estimating the population proportion
vaccinated.
The fact that the PHE/ONS mortality report is ‘missing’ millions of people is confirmed by their
count of all-cause deaths. The ONS registered deaths reports age stratified all-cause deaths
for England and Wales combined, so the first assumption we must make in our analysis is to
account for the fact that the ONS reports the total deaths in England and in Wales separately.
For week 26 the ONS registered deaths lists a total of 8,808 all-cause deaths in people in
England and Wales and 8,227 in England. Hence, we can apply the proportion (8,227/8,880)
across each age group to estimate the expected deaths per age group and then remove the
estimated under 10s from the England total. Alternatively, we could do the same adjustment
by total population of each nation; this results in similar estimates. This results in an estimate
of 8,192 deaths in the over 10s in England in week 26.
However, in stark contrast, the PHE/ONS mortality report gives a total of only 6,956 deaths
for over 10s in England. Thus 1,236 deaths are unaccounted for.
4
How bad might the discrepancies be?
As we said we cannot reverse engineer the mortality estimates for each age category due to the
unreliability of population denominators and death counts. To further explore this unreliability, we
will attempt to derive the population estimates for each age category and the expected deaths from
other ONS sources. Note that we are not claiming that these are reliable but using them to highlight
the magnitude of potential errors that might be involved. We can do this using the NIMS vaccine
survey, the ONS population survey and the ONS registered deaths data for week 26, ending 2 July
2021. We chose week 26 because: (i) it is the latest week for which data were available from the ONS
at the time; (ii) it had the lowest Covid-19 mortality in any month up to that date in 2021 (there were
only 109 Covid-19 deaths); and (iii) it is in summer and thus mortality data are generally less affected
by influenza-like illnesses and other seasonal factors. Likewise, and perhaps even more importantly,
compared to the period December 2020 March 2021, by July the vaccination programme had
vaccinated many millions of people over the age of 18, thus providing a huge and representative
sample size on which to base any statistical or arithmetical estimates.
The analysis proceeds as follows:
1. Use the NIMS vaccination survey and the ONS 2020 population survey to estimate the
population vaccinated and unvaccinated, in each age category using the ONS/PHE mortality
report population of 39,245,327 people.
2. Sum over the age categories to ‘reverse engineer’ the total population in each of the
vaccination categories, and then compare these to the ONS totals. Ideally, they should be
“reasonably” close, but if not it then there are significant, perhaps worrying, differences
between NIMS and the ONS mortality report.
3. Perform the steps above for the death counts.
4. Compare the results against ONS actuarial life tables for expected mortality.
Are the ONS underestimating the number of people unvaccinated?
The NIMS vaccine survey for week 26 has a total population of 61,941,461 of which 54,977,393 are
listed as 10 years and over. From the NIMS vaccine survey, we can obtain the percentage by
vaccination status for each age group shown in Table 2.
Because, for our reverse engineering of the figures, we are reliant on the PHE/ONS mortality report
for total deaths by vaccination status, all our population sizes by age category must therefore be ‘pro-
rated’ down to the population figure used therein, which is 39,245,327. If we apply these percentages
to the ONS population size of 39,245,327 we get the distribution and totals shown in Table 3.
Note that in Table 3 the estimated totals are significantly different, but dramatically so for the
unvaccinated category. Nearly 3 million of those we estimate to be classified as unvaccinated, using
NIMS, are classified as two-dose vaccinated compared with the PHE/ONS survey. Appendix B further
discusses the extent to which this discrepancy is due to either NIMS underestimating the number of
vaccinated or ONS underestimating the proportion of unvaccinated.
5
Table 2 England population percentage by age category and vaccination status for week 26 (NIMS)
Age
Unvaccinated
1 dose
2 doses
80 - 100
4.76%
2.13%
93.10%
75 - 79
4.59%
1.37%
94.04%
70 - 74
5.60%
1.44%
92.96%
65 - 69
7.84%
2.05%
90.11%
60 -64
9.84%
3.32%
86.84%
55 -59
11.83%
4.21%
83.96%
50 - 54
14.47%
5.01%
80.51%
45 - 49
19.63%
18.44%
61.93%
40 - 44
25.92%
27.36%
46.72%
35 - 39
33.05%
37.77%
29.18%
30 - 34
39.15%
37.77%
23.08%
25 - 29
44.43%
36.47%
19.09%
18 - 24
48.50%
36.11%
15.39%
10 - 17
98.62%
0.87%
0.51%
Table 3 England population by age category and vaccination status for week 26 (Estimated from
NIMS versus PHE/ONS actual)
PHE/ONS Survey Population Sizes estimated
using NIMS survey percentages
Age
Unvaccinated
1 dose
2 doses
80 - 100
95,143
42,640
1,860,177
75 - 79
68,126
20,295
1,395,722
70 - 74
114,869
29,542
1,906,242
65 - 69
161,773
42,369
1,859,099
60 -64
242,309
81,724
2,138,458
55 -59
344,018
122,541
2,442,592
50 - 54
436,365
151,169
2,427,187
45 - 49
559,952
525,964
1,766,329
40 - 44
762,316
804,650
1,373,909
35 - 39
1,062,460
1,214,192
938,243
30 - 34
1,325,233
1,278,374
781,135
25 - 29
1,406,056
1,154,221
604,162
18 - 24
1,808,472
1,346,375
573,908
10 - 17
3,922,069
34,554
20,394
Estimated Total
12,309,162
6,848,610
20,087,556
PHE/ONS Total
9,531,364
6,404,395
23,309,568
Difference
2,777,798
444,215
-3,222,012
6
Is the pattern of mortality what we might expect to see?
Next, we can use the death counts from the ONS registered deaths in week 26 to estimate the
expected deaths for each age category in the unreleased PHE/ONS report. This is done using
proportional allocation, which assumes the all-cause mortality rates are independent of vaccination
status (and hence implicitly assuming vaccines have no impact on all-cause mortality). When the
PHE/ONS release their data, we can test whether the pattern deviates significantly from this assumed
independence. For now, all we can do is compare the total death counts given this is what PHE/ONS
have released.
As we have said we have had to pro-rate the ONS registered deaths in week 26, to take account of
differences between England & Wales and England. We can do this again for deaths in each age
category and then use the ONS 2020 population survey to calculate the UMR for each age category.
When we apply this UMR to the estimated populations in Table 3, we get the results shown in Table
4.
Table 4 England expected population percentage per age category and vaccination status all-cause
deaths for week 26, using ONS 2021 death registration and ONS 2020 population data and Table 3
Estimated Populations
All-cause deaths estimated in PHE/ONS
using ONS deaths data and Table 3
Estimated Populations
UMR*
Age
Unvaccinated
1 dose
2 doses
80 - 100
143
64
2,790
149.97
75 - 79
37
11
756
54.14
70 - 74
36
9
595
31.23
65 - 69
31
8
354
19.02
60 -64
34
12
301
14.08
55 -59
30
11
215
8.79
50 - 54
23
8
129
5.33
45 - 49
23
21
72
4.08
40 - 44
23
24
41
2.96
35 - 39
18
21
16
1.72
30 - 34
14
13
8
1.05
25 - 29
8
6
3
0.54
18 - 24
12
9
4
0.68
10 - 17
12
0
0
0.31
Expected Total
444
218
5,284
PHE/ONS Total
436
576
5,944
Percentage Ratio
Actual/Expected
98%
265%
112%
* Note that the UMR here is the same for all categories of vaccination
Notice that the Expected Total all-cause deaths (summing the totals of the three columns) is 5,945
whilst the PHE/ONS Total sums to 6,956. This is a significant difference. Likewise, when we compare
the expected deaths versus actual deaths for each of the vaccination categories there is close
alignment for the unvaccinated categories (444 versus 436), less so for the 2-dose vaccinated (5,284
7
versus 5,944) and much less so still for the single dose vaccinated (218 versus 576). The ratio of actual
to expected is over 250% in the single dose vaccinated and 112% in the two-dose vaccinated.
We already noted earlier those 1,236 deaths of over-10s deaths in England during week 26 are
unaccounted for (based on our estimate of 8,192 and the PHE/ONS mortality report number 6,956).
It might be reasonable to expect that these occurred in the ‘missing’ 10 million.
Conclusions
All the caveats about confounders etc. that were discussed in our previous analyses and reports
(notably in [7]) apply. However, the sample sizes here are sufficiently large for most of these to apply
consistently across all classes of vaccination. We suspect that all confounders are likely to be present
in all sub-populations to a similar degree. However, possible systemic biases might include:
We are at the mercy of the grossly different and variable data and statistics available from the
ONS in myriad form hence, needless to say, the reliability of our analysis is entirely reliant on
the ONS data.
Vaccinated people may be more likely to have co-morbidities (which may partly explain higher
mortality rates), although this is less relevant the lower the age and the higher the population
in that age group that are vaccinated.
While our analysis is restricted to just the most recent week where death data by vaccination status
is available (week 26), the previous four weeks show a similar pattern.
Our analysis has discovered that over 10 million people are missing from the PHE/ONS analysis and
1,236 deaths that occurred during week 26 are also missing. The vaccination status of this group is
unknown. Furthermore, by reverse engineering the estimates from other ONS sources we have
discovered that the PHE/ONS mortality report is underestimating the number of vaccinated people,
from an approximate total of 39 million, by over 2 million people. Similarly, we believe the ONS may
be underestimating the number of single dose vaccinated people by just over four hundred thousand.
Given this, there is the possibility that as many as 22 million people, in week 26, were unvaccinated
rather than the 9.5 million reported.
Our analysis clearly suggests that, when compared to ONS death figures from week 26, all-cause
mortality (UMR) for vaccinated people, compared to unvaccinated people, is certainly higher in single
dosed individuals and slightly higher in those who are double dosed.
Any analysis that relied solely on the PHE/ONS mortality data would be systematically biased by the
fact that it would be conditioned on the available data, and how it is queried from available databases,
rather than on the prevailing vaccination status of the population at large. In attempting to reverse
engineer estimates of mortality by age category and vaccination status from the various relevant ONS
datasets we found numerous discrepancies and inconsistencies which indicate that the PHE/ONS
reports on vaccine effectiveness are grossly underestimating the number of unvaccinated people.
Although we have not subjected this data to statistical testing the potential implications of these
results on the effects of vaccination on all-cause mortality, and by implication, the future of the
vaccination programme is profound. Hence, if our estimates are inconsistent with the (unreleased)
raw data, it is incumbent on the ONS to provide the raw data along with an explanation of why our
estimates are wrong. We look forward to them releasing the data forthwith.
8
Acknowledgments
We would like to acknowledge Dr Clare Craig and Jonathan Engler for commenting on earlier drafts
and Dr Vahe Nafilyan and Dr Charlotte Bermingham from the ONS for their advice and help.
References
[1] Public Health England/Office for National Statistics (PHE/ONS). Age-standardised mortality rates
for deaths by vaccination status, England: deaths occurring between 2 January and 2 July 2021.
https://www.ons.gov.uk/file?uri=%2fpeoplepopulationandcommunity%2fbirthsdeathsandmarriages
%2fdeaths%2fdatasets%2fdeathsbyvaccinationstatusengland%2fdeathsoccurringbetween2januarya
nd2july2021/dataset.xlsx
[2] Revised European Standard Population 2013.
https://webarchive.nationalarchives.gov.uk/ukgwa/20160106020035/http:/www.ons.gov.uk/ons/gu
ide-method/user-guidance/health-and-life-events/revised-european-standard-population-2013--
2013-esp-/index.html
[3] National Immunisation Management Service (NIMS) National flu and COVID-19 surveillance
reports (PHE/ONS) 01 July 2021 Week 26.
https://www.gov.uk/government/statistics/national-flu-and-covid-19-surveillance-reports
[4] ONS. Population estimates for the UK, England and Wales, Scotland and Northern Ireland: mid-
2020.
https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationesti
mates/bulletins/annualmidyearpopulationestimates/mid2020
[5] Deaths registered weekly in England and Wales by age and sex: covid-19, 2021. ONS.
https://www.ons.gov.uk/datasets/weekly-deaths-age-sex/editions/covid-19/versions/51
[7] Fenton N.E., Neil M., McLachlan S., (2021) “Paradoxes in the reporting of Covid19 vaccine
effectiveness: Why current studies (for or against vaccination) cannot be trusted and what we can
do about it” http://dx.doi.org/10.13140/RG.2.2.32655.30886
[8] Barnes, O. and Burn-Murdoch J. “Covid response Hampered by population data glitches”,
Financial Times, 11 October 2021.
[9] NHS England. Denominators for COVID-19 vaccination statistics.
https://www.england.nhs.uk/statistics/wp-content/uploads/sites/2/2021/05/Denominators-for-
COVID-19-vaccination-statistics.docx
[10] 2011 Census: Population Estimates by five-year age bands, and Household Estimates, for Local
Authorities in the United Kingdom.
https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationesti
mates/datasets/2011censuspopulationestimatesbyfiveyearagebandsandhouseholdestimatesforlocal
authoritiesintheunitedkingdom
[11] Deaths registered weekly in England and Wales, provisional. ONS (consulted 19 October 2021).
https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/datase
ts/weeklyprovisionalfiguresondeathsregisteredinenglandandwales
9
Appendix A
Why all “all-cause mortality” is the most appropriate measure for overall risk-benefit analysis of
Covid vaccines
If Covid is as dangerous as claimed - and if the vaccine is as effective as claimed - we should
by now have seen many more Covid related deaths among the unvaccinated than the
vaccinated (in each age group).
If the vaccine is as safe as claimed, then there should have been very few more deaths from
causes unrelated to Covid among the vaccinated than the unvaccinated (in each age group).
So, the count of all-cause deaths should be higher among the unvaccinated than the
vaccinated (in each age group), confirming that the benefits of vaccination outweigh the
risks.
Counting all-cause deaths completely bypasses the problem of defining what constitutes a
‘Covid case’ or a ‘Covid related death’ (definitions which can be easily manipulated to fit
different narratives).
We define a person as ‘vaccinated’ if they have received at least one dose. As we are not
interested in whether a person becomes a ‘Covid case’, any other definition is flawed as it
will fail to acknowledge that adverse reactions (including death) from vaccines usually occur
shortly after vaccination.
The fact that the US CDC (Centre for Disease Control) now counts a person as ‘unvaccinated’
if they die within 14 days of the second dose, or after just one dose, might make some sense
if we are interested only in the vaccine’s ability to stop infection. But in the context of death
attribution, it is nothing less than fraudulent.
10
Appendix B - Under- and Over-reporting of Vaccinations
There are several ways that potential under- and over-reporting of COVID-19 vaccinations might
occur.
The greatest potential for systematic errors in the estimation of people unvaccinated arises from the
way the ONS reduces the total England population to a denominator of 39 million - represented by
the dashed red line shown in Figure 1.
ONS staff explained to the authors of this paper how they begin with three datasets:
1. The population as reported from the 2011 Census, from which they remove any individual
verified as deceased.
2. The General Practice Extraction Service (GPES), from which they remove those who:
a. are no longer registered with a GP clinic;
b. no longer reside at the English postcode recorded on their Electronic Health Record
(EHR), and;
c. any records of patients reported as deceased or who were not present in England to
have been counted in the 2011 census.
3. The NIMS dataset on vaccination status.
The population used in the PHE/ONS mortality report represents those individuals remaining across
all three of these datasets - effectively only a subset of each and almost twenty million people short
of the estimated population of England for 2020.
Two issues arise from the conspicuous absence of these people. First, the vaccination status of these
‘missing’ is unknown. It cannot be automatically assumed that they are vaccinated, as the PHE/ONS
statistics that report vaccinated figures are based on the reduced (red line) population. Second, the
all-cause mortality figures used to develop the numerators includes everyone who has died during
week 26, but the population denominator is missing a significant proportion of the population. If the
majority of the missing individuals are, as we suspect, unvaccinated - their absence significantly
inflates the UMR and ASMR for that group.
Figure 1: The ONS population conundrum
11
With the help from NHS Digital and NHS Trust Informatics team members we investigated potential
pathways through the NHS recording keeping systems that might lead under and over-inflation, as
describe in Figure 2.
Figure 2: Sources for potential under- and over-reporting
Based on the weekly NHS vaccination site lists [1], Figure 2 shows three primary ways in which
vaccination occurs:
1. Community-based popup venue or vaccination centre which have included tents in university
quads, large public facilities such as racecourses and cricket grounds, shopping centres,
council-owned car parks and town halls.
2. Hospital, either as a walk-in or during an in-patient stay.
3. The GP clinic where the individual is registered as a patient.
Potential Underreporting of vaccinated:
Underreporting may occur in all three pathways but is potentially more prevalent on the first. In
particular:
Vaccination cannot be refused where the individual declines to provide identification, an
address or their immigration status [2]; while the total number of such individuals vaccinated
may be recorded, they will not appear in any list of vaccinated.
While the vaccinated individual is provided with a card that identifies the dose they received
(first or second) and the batch number of the administered vaccine [3], it can depend on the
site as to whether their vaccination status is reported on the NHS vaccine database (using
either Outcomes4Health or NIVS) [4].
At some sites, problems with IT result in staff having to record vaccinations manually using
paper and pen [5]. There is obvious potential for error and missing records when manual entry
must later be used to reconcile paper records with electronic records [6]. Any individual who
declines to identify themselves cannot be appropriately recorded and reported to NIMS.
12
The onus falls to the recipient to:
o enter their vaccination details into the NHS App or website, or
o take their vaccination card to their GP to have their record updated, and
o to later remember to verify that their COVID PASS correctly reflects their status
Some individuals may only need the vaccination card [3] to demonstrate their status to an
employer and may therefore not need or bother to check whether their status has been
reported and recorded in NIMS.
As with any critical data-based infrastructure, security and complexity can introduce delay,
from hours to weeks [6], in transferring new or updated information between different
records systems. This can even actively prevent data transfer from occurring [7].
Potential Overreporting of vaccinated:
Overreporting can also arise on all three pathways. However, it is more likely to occur where an
individual person receives their COVID-19 vaccinations at different sites. Overreporting may arise
when:
Individuals who do not identify themselves at popup venues or vaccination centres:
o may not have been counted in the ONS census population count, or
o may have resided outside England but get counted in England’s vaccination figures.
It is possible for vaccination to get recorded against the wrong NHS number [7]. If the
vaccination recipient realises and shows their vaccination card to their GP, their vaccination
status can be updated. However, unless the NHS number that the vaccination was erroneously
recorded against it is known, or the individual who that NHS number belongs to becomes
aware of the error and reports it, that other record may persist. As a result, it is possible for
two different individuals’ records to report as vaccinated from a single vaccination event. This
can occur because of:
o Errors in data entry.
o The existence of multiple NHS records for the same individual [8], or
o when one vaccination event is recorded against a married person’s maiden name, and
the second recorded against their married name - or vice versa [9].
Confusion has arisen due to two individuals using the same NHS number [8], such as when an
individual who is not entitled to free NHS maternity care uses the NHS number of a family
member who is. If that individual is vaccinated during the antenatal period, this vaccination
event will be recorded against the NHS number’s true owner who will then be reported in the
national counts as vaccinated when they are not.
In either case, whether under- or over-reporting, it can take many weeks and cause considerable
frustration when individuals seek to correct errors in vaccination records once they have propagated
between the myriad of databases, data systems and across organisations [10, 11] - and confusion over
rules for the details that are recorded on vaccination certificates, vaccine passports and the degree of
detail necessary when details are recorded from identity documents has only added to the confusion
[12].
Up to 70% of medical records contain errors, with 23% being important or serious enough to affect
future care potentially adversely [13], and many of these organisations have a notoriously poor record
for ensuring all records pertaining to an error are corrected and ‘cleaned up’. All of which give rise to
potential under and over-reporting errors that we may not have realised in this paper.
13
References Appendix B
[1] https://www.england.nhs.uk/coronavirus/publication/vaccination-sites/
[2] https://www.london.gov.uk/coronavirus/covid-19-vaccine/getting-covid-vaccine-if-you-are-not-
registered-gp
[3] https://bit.ly/3pdGfbu
[4] https://www.nhs.uk/conditions/coronavirus-covid-19/covid-pass/
[5] https://www.lbc.co.uk/news/it-crash-causes-clinicians-to-log-covid-jabs-with-pen-and-paper/
[6] https://digital.nhs.uk/coronavirus/vaccinations/covid-19-vaccination-record-queries
[7] https://www.bbc.co.uk/news/uk-scotland-58475922
[8] https://digital.nhs.uk/services/national-back-office-for-the-personal-demographics-service/data-
quality-incidents
[9] https://www.telegraph.co.uk/news/2021/07/14/married-women-may-barred-flights-vaccine-
certificates-maiden/
[10] https://www.eveningexpress.co.uk/fp/news/local/i-am-just-lost-aberdeen-womans-two-week-
battle-to-change-name-on-covid-vaccine-certificate/
[11] https://www.england.nhs.uk/nhse-nhsi-privacy-notice/joint/how-to-access-your-personal-
information-or-make-a-request-in-relation-to-other-rights/
[12] https://www.dailyrecord.co.uk/in-your-area/lanarkshire/lanarkshire-family-vaccine-passport-
warning-24927385
[13] Pyper, C., Amery, J., Watson, M., & Crook, C. (2004). Patients' experiences when accessing their
on-line electronic patient records in primary care. British Journal of General Practice, 54(498), 38-43.
... Our recent articles [1,2] have argued that the simplest and most objective way to assess the overall risk/benefit of Covid-19 vaccines is to compare all-cause mortality rates of the unvaccinated against the vaccinated in each separate age-group. For such an assessment we need accurate periodic data on both age-categorized deaths and the number of vaccinated/unvaccinated people in each age group for that period. ...
... The UK Government has been better than most countries in providing detailed data on Covid cases and deaths indexed by vaccine status. However, in [1] we highlighted the absence of relevant agecategorized mortality data for England, and major inconsistencies in the data provided by different agencies. Of most concern are the very different estimates provided by UKHSA (United Kingdom Health Security Agency) and the ONS (Office for National Statistics) of the number of vaccinated and unvaccinated people. ...
... They claimed that the ONS data 'fixed' this bias and hence properly adjusted the results. However, as we pointed out in [1], while the NIMS data may indeed overestimate the number of vaccinated, it is likely that it also underestimates the number of unvaccinated (a much more difficult number to estimate than those vaccinated). ...
Preprint
Full-text available
The risk/benefit of Covid vaccines is arguably most accurately measured by comparing the all-cause mortality rate of vaccinated against unvaccinated, since it not only avoids most confounders relating to case definition but also fulfils the WHO/CDC definition of "vaccine effectiveness" for mortality. We examine two of the most recent UK ONS vaccine mortality surveillance reports, which provide the necessary information to monitor this crucial comparison over time. At first glance the ONS data suggest that, in each of the older age groups, all-cause mortality is lower in the vaccinated than the unvaccinated. This conclusion is cast into doubt upon closer inspection of the data due to a range of fundamental inconsistencies and anomalies in the data. Whatever the explanations for these are, it is clear that the data is both unreliable and misleading. It has been suggested that the anomalies are the result of healthy vaccinee selection bias and population differences. However, we show why the most likely explanations for the observed anomalies are a combination of systemic miscategorisation of deaths between the different categories of unvaccinated and vaccinated; delayed or non-reporting of vaccinations; systemic underestimation of the proportion of unvaccinated; and/or incorrect population selection for Covid deaths. We also find no evidence that socio-demographic or behavioural differences between vaccinated and unvaccinated can explain these anomalies.
... Our recent articles [1,2] have argued that the simplest and most objective way to assess the overall risk/benefit of Covid-19 vaccines is to compare all-cause mortality rates of the unvaccinated against the vaccinated in each separate age-group. For such an assessment we need accurate periodic data on both age-categorized deaths and the number of vaccinated/unvaccinated people in each age group for that period. ...
... The UK Government (through its various relevant agencies) has been better than most countries in providing detailed data on Covid cases and deaths indexed by vaccine status. However, in [1] we highlighted the absence of relevant age-categorized mortality data for England, and major inconsistencies in the data provided by different agencies. Of most concern are the very different estimates provided by UKHSA (United Kingdom Health Security Agency) and the ONS (Office for National Statistics) of the number of vaccinated and unvaccinated people. ...
... They claimed that the ONS data 'fixed' this bias and hence properly adjusted the results. However, as we pointed out in [1], while the NIMS data may indeed overestimate the number of vaccinated, it is likely that it also underestimates the number of unvaccinated (a much more difficult number to estimate than those vaccinated). ...
Preprint
Full-text available
This paper has been updated and the new version can be found here: Official mortality data for England suggest systematic miscategorisation of vaccine status and uncertain effectiveness of Covid-19 vaccination UPDATED WITH ONS DECEMBER DATA RELEASE & HEALTHY VACCINEE/MORIBUND ANALYSIS http://dx.doi.org/10.13140/RG.2.2.28055.09124 https://www.researchgate.net/publication/357778435_Official_mortality_data_for_England_suggest_systematic_miscategorisation_of_vaccine_status_and_uncertain_effectiveness_of_Covid-19_vaccination ------- The risk/benefit of Covid vaccines is arguably most accurately measured by an all-cause mortality rate comparison of vaccinated against unvaccinated, since it not only avoids most confounders relating to case definition but also fulfils the WHO/CDC definition of "vaccine effectiveness" for mortality. We examine the latest UK ONS vaccine mortality surveillance report which provides the necessary information to monitor this crucial comparison over time. At first glance the ONS data suggest that, in each of the older age groups, all-cause mortality is lower in the vaccinated than the unvaccinated. Despite this apparent evidence to support vaccine effectiveness-at least for the older age groups-on closer inspection of this data, this conclusion is cast into doubt because of a range of fundamental inconsistencies and anomalies in the data. Whatever the explanations for the observed data, it is clear that it is both unreliable and misleading. While socio-demographical and behavioural differences between vaccinated and unvaccinated have been proposed as possible explanations, there is no evidence to support any of these. By Occam's razor we believe the most likely explanations are systemic miscategorisation of deaths between the different categories of unvaccinated and vaccinated; delayed or non-reporting of vaccinations; systemic underestimation of the proportion of unvaccinated; and/or incorrect population selection for Covid deaths.
... 34 Under-and over-reporting of vaccination may potentially occur in several ways, like failure in digital record, not correctly identified individuals and errors in data entry. 35 Moreover, a not fully effective vaccine may convey an overall perception of low protection in the population. ...
Article
The success of mass vaccination campaigns may be jeopardized by human risky behaviors. For example, high level of vaccination coverage may induce early relaxation of social distancing. In this paper, we focus on the mutual influence between the decline in prevalence, due to the rise in the overall immunization coverage, and the consequent decrease in the compliance to social distancing measures. We consider an epidemic model where both the vaccination rate and the disease transmission rate are influenced by human behavior, which in turn depends on the current and past information about the spread of the disease. We highlight the impact of the information-related parameters on the transient and asymptotic behavior of the system that is on the early stage of the epidemic and its final outcome. Among the main results, we evidence that sustained oscillations may be triggered by the behavioral memory in the prevalence-dependent vaccination rate. However, the relaxation of social distancing may induce a switch from a cyclic regime to damped oscillations.
Preprint
Full-text available
U hrvatskim je medijima sve više govora o cijepljenju djece protiv covid-19, unatoč maloj ulozi djece u prijenosu novog koronavirusa i njihovom malom riziku od teških simptoma, postojanju drugih oblika prevencije, činjenici da klinička ispitivanja nisu dovršena, raznih problema u provedenim ispitivanjima i rastućoj zabrinutosti oko sigurnosti cjepiva i mogućih štetnih učinaka. Cilj je ovog kratkog pregleda odabrane znanstvene literature potaknuti kvalitetnu javnu raspravu prije donošenja potencijalno ishitrenih odluka.
Preprint
Full-text available
Given the limitations of the randomized controlled trials (RCTs) for Covid19 vaccines, we must increasingly rely on data from observational studies to determine vaccine effectiveness. But over-simplistic reporting of such data can lead to obviously flawed conclusions due to statistical paradoxes. For example, if we just compare the total number of Covid19 deaths among the vaccinated and unvaccinated then we are likely to reach a different conclusion about vaccine effectiveness than if we make the same comparison in each age category. But age is just one of many factors that can confound the overall results in observational studies. Differences in the way we classify whether a person is vaccinated or is a Covid19 case can also result in very different conclusions. There are many critical interacting causal factors that can impact the overall results presented in studies of vaccine effectiveness. Causal models and Bayesian inference can in principle be used to both explain observed data and simulate the effect of controlling for confounding variables. However, this still requires data about relevant factors and much of these data are missing from the observational studies (and the RCTs). Hence their results may be unreliable. In the absence of such data, we believe the simplest and most conclusive evidence of vaccine evidence is to compare all-cause deaths for each age category between those who were unvaccinated and those who had previously had at least one vaccine dose.
Article
Full-text available
Patient access to on-line primary care electronic patient records is being developed nationally. Knowledge of what happens when patients access their electronic records is poor. To enable 100 patients to access their electronic records for the first time to elicit patients' views and to understand their requirements. In-depth interviews using semi-structured questionnaires as patients accessed their electronic records, plus a series of focus groups. Secure facilities for patients to view their primary care records privately. One hundred patients from a randomised group viewed their on-line electronic records for the first time. The questionnaire and focus groups addressed patients' views on the following topics: ease of use; confidentiality and security; consent to access; accuracy; printing records; expectations regarding content; exploitation of electronic records; receiving new information and bad news. Most patients found the computer technology used acceptable. The majority found viewing their record useful and understood most of the content, although medical terms and abbreviations required explanation. Patients were concerned about security and confidentiality, including potential exploitation of records. They wanted the facility to give informed consent regarding access and use of data. Many found errors, although most were not medically significant. Many expected more detail and more information. Patients wanted to add personal information. Patients have strong views on what they find acceptable regarding access to electronic records. Working in partnership with patients to develop systems is essential to their success. Further work is required to address legal and ethical issues of electronic records and to evaluate their impact on patients, health professionals and service provision.
Population estimates for the
  • England Uk
  • Scotland Wales
  • Northern Ireland
ONS. Population estimates for the UK, England and Wales, Scotland and Northern Ireland: mid-2020. https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationesti mates/bulletins/annualmidyearpopulationestimates/mid2020
Covid response Hampered by population data glitches
  • O Barnes
  • J Burn-Murdoch
Barnes, O. and Burn-Murdoch J. "Covid response Hampered by population data glitches", Financial Times, 11 October 2021.
Denominators for COVID-19 vaccination statistics
  • Nhs England
NHS England. Denominators for COVID-19 vaccination statistics. https://www.england.nhs.uk/statistics/wp-content/uploads/sites/2/2021/05/Denominators-for-COVID-19-vaccination-statistics.docx