PreprintPDF Available

A suggested effect of air temperature on severity of COVID-19 in hospitalized patients by Kifer et al. (Effects of environmental factors on severity and mortality of COVID-19; medRxiv 10.1101/2020.07.11.20147157) is most likely an artifact

Authors:
Preprints and early-stage research may not have been peer reviewed yet.
1
A suggested effect of air temperature on severity of COVID-19 in hospitalized patients by
Kifer et al. (Effects of environmental factors on severity and mortality of COVID-19;
medRxiv 10.1101/2020.07.11.20147157) is most likely an artifact- Rev1
Vladimir Trkulja1, Ivan Kodvanj1, Jan Homolak1
1 Department of Pharmacology, University of Zagreb School of Medicine, Zagreb, Croatia
Vladimir Trkulja, MD, PhD
Department of Pharmacology
University of Zagreb School of Medicine
Zagreb, Croatia
vtrkulja@mef.hr
Introduction
A few weeks ago, a manuscript was pre-printed (1) that suggested an association (and in part of
data interpretation by the authors, even a causal effect was implied) between increasing air
(ambient) temperature that occurred from February to July 2020 (as between any February and
any July since the origin of time) and reduced severity of COVID-19 disease. The manuscript (1)
is freely available (medRxiv 10.1101/2020.07.11.20147157) and has received quite some
attention in the “general” or “lay” public (e.g., https://www.rtl.hr/vijesti-
hr/korona/3850946/znanstvenik-lauc-objavio-odgovorili-smo-na-jedno-od-velikih-otvorenih-
pitanja-u-ovoj-pandemiji/), but also in some non-lay circles
(https://www.kcl.ac.uk/news/covid-19-worse-in-colder-weather).
In brief (1), the authors gathered data on patients who were, during this period (late February-
early July), hospitalized for COVID-19 disease in 8 European cities (southern, northern and
central-eastern European countries) a total of 6306 subjects - and one city in China
(Zehnjiang), contributing 608 subjects hospitalized during late January/early February. Data
were generated from the hospital records or from the official statistics (1). Several outcomes,
intended to serve as measures of disease severity among the hospitalized patients, were
reported (1), of which we focus on two that we find to be the most illustrative ones: proportion
of hospitalized patients who died and proportion of hospitalized patients who, at any time
during the index hospitalization, needed admission to an intensive care unit (ICU). The main
points made in the pre-printed manuscript (1) were that (a) with elapsing calendar time, odds
2
of both outcomes declined and (b) with increasing temperature (over the observed period),
odds of both outcomes declined.
With no intention to evaluate overall evidence about the potential impact of environmental
factors on the epidemiological and clinical course of COVID-19, we would like to draw the
attention to methodological weaknesses of the pre-printed manuscript (1) (exemplified by the
two mentioned outcomes) in order to point-out how inappropriate approach to observational
data and to meta-analysis may result in inaccurate estimates of the reality (i.e., misleading/false
associations). We focus on the 8 European cities: the Chinese city contributed a minor part of
the data and only for a calendar period during which no data were provided from European
cities; the total number of events (death or need for ICU admission) was low; and it was omitted
from most of the analyses in the pre-printed manuscript (1).
The text is organized as follows: we first outline the methodological approach (to the two
outcomes) employed by the authors (1); next, we briefly comment on what appear to be obvious
methodological (and logical) drawbacks of this process; finally, we reconstruct and re-analyze
data pertaining to the two outcomes, using the numbers displayed in the pre-printed (1)
manuscript.
Outline of the methodological approach used in the pre-printed manuscript (1)
The number of hospitalized patients contributed from the 8 European cities varied between 89
(city of Coburg, Germany) to 1786 (city of Barcelona, Spain). Other cities (Bergamo, Italy;
Helsinki, Finland; Milan, Italy; Nottingham, UK; Warsaw, Poland and Zagreb, Croatia)
contributed variable numbers (between 100 from Helsinki to 995 from Bergamo) (1).
Analysis of both proportions - of those who died and of those who needed ICU - followed the
same pattern (1): a) each city was considered as a separate cohort and was analyzed separately;
b) in the first step, logistic regression models were fitted to the proportions, with calendar date
of the hospital admission, age and sex as independents to obtain ORs of death/need for ICU. The
ORs yielded for the factor “calendar date of admission were then meta-analytically pooled
across the 8 cities. Since the pooled ORs for both outcomes were <1.0, it was concluded (1) that
mortality/need for ICU among hospitalized patients declined over time, i.e., that elapsing time
(“higher” calendar date) was associated with lower odds of either outcome; c) in the next step,
identical models were fitted to the probability of dying/needing ICU, except that the
independent “calendar date of admission” was replaced by an independent “air temperature” –
an average of the mean daily temperatures recorded during the hospitalization period
[retrieved from the Climate Data Online (National Centers for Environmental Information,
NCEI) database] (1). The ORs yielded for the factor “air temperature” where then pooled across
3
the cities. Since the pooled ORs for both outcomes were <1.0, it was concluded that
mortality/need for ICU among hospitalized patients declined with increasing air temperature,
i.e., that higher temperature was associated with lower odds of either outcome.
Some methodological (and logical) drawbacks of the reported analysis
In the Discussion section (1), the authors extend implications/generalizations of the reported
findings to different areas pertaining to the COVID-19 pandemic. We would like to focus
specifically to the actual question addressed severity of the disease among hospitalized (and
thus, pre-selected) patients, illustrated by mortality and need for ICU. Gathering and analysis of
observational data (e.g., 2, 3) as well as meta-analysis (e.g., 4, 5) are delicate procedures.
Addressing all the finesse of the art of their implementation, which is hugely relevant for the
matter, is by far beyond our scope and reach, however we draw the attention to the following:
a) The number of patients contributed by individual cities largely varied, overall and by
calendar period. For some cities, data pertained to quite a limited period of time (e.g., Milan
from late February till the end of April), while for others data extended from late March to early
July (e.g., Warsaw), resulting in the fact that some calendar periods were “represented” by data
from only two, three or one city (e.g., late February, late June, early July, respectively);
b) While death is a rather obvious outcome, the other outcome considered in this comment
(admission to an ICU) bears a huge potential for generation of bias: there is no mention about
individual (by city/hospital) policies/criteria on which decisions were made about ICU
admissions. In some instances, patients might have been admitted to the ICU not due to the
severity of disease (at presentation or at any later time), but simply because ICUs served as
isolation units. For example, all patients reported from Warsaw (n=122) were reported to
have been admitted to ICU, i.e., there were no other hospitalized patients.
c) Beyond the independents of interest (calendar date of admission or, alternatively, air
temperature), the logistic models considered only age and sex as adjustments: the number of
confounders (time-fixed, i.e., those present at the moment of hospitalization, as well as time-
varying pertaining to those relevant for the hospitalization period of each particular patient,
as well as those pertaining to the entire observed period) not accounted for is endless, both at
the patient level (e.g., anthropometrics, comorbidities, delivered treatments) and at the level of
general developments regarding the disease [e.g., local criteria for hospitalizations; evolving
knowledge related to the strategy of oxygen therapy and (non)implementation of other
treatments; lock-down and other epidemiological measures; epidemiological/organizational
status at individual hospitals, e.g., hospital infections, spread of the disease among hospital staff
4
and staff availability, availability of equipment, work overload, healthcare system accessibility
and so on.];
d) Separate analysis of the associations between the outcomes and “calendar date of admission”
and “average daily temperature” was likely due to a high correlation between the two (average
temperature increased with the later calendar date) generating the collinearity issue if both
were included in the same model. However separate analysis for each of the two independents
of interest was a poor choice and did not resolve the problem - if one is to detect the association
between high(er) temperature and the outcome, it needs to be separated from the effect of
“evolving calendar time” (which brings along a number of time-varying effects, as mentioned,
not only increasing daily temperature) by adequate adjustments;
e) No particular methodological aspects of the meta-analytical procedure were provided, except
that it was by “random-effects, inverse-variance method” (1). Also, no measures of
heterogeneity/inconsistency were reported with any of the pooled effects (1). Pooling data from
observational studies (each city was treated as a separate observational cohort/study), in this
case log(OR), from a limited number of cohorts (studies) that largely vary in size, number
(proportion) of events and pertain to a variable observational periods is a “tricky business”. The
choice of the method of pooling and of the variance estimator, as well as reporting the measures
of heterogeneity are essential (6).
Re-analysis of the reconstructed data
Methods. Data were reconstructed using numerical values depicted in the pre-printed
manuscript (1), and were used exactly as displayed and defined. Specifically: a) Figure 1A (1)
displays numbers of patients who died and numbers of patients who were discharged alive
[proportion dying among hospitalized= died/(died+discharged)] by 2-week intervals
(admission dates), starting January 27 (start of the 1st 2-week period, data only for Zehnjiang).
This 1st 2-week period was depicted as period 1, and each subsequent 2-week period till
beginning of July 2020 was consecutively enumerated. There were overall 13 intervals, and
European cities contributed data from period 3 (two cities, starting February 12) till the end of
period 13 (only one city, beginning of July). Varying numbers of cities contributed data for each
of the observed periods; b) Supplementary Figure 3A (1) displays the number of patients
needing ICU as n (needing ICU)/N (total number hospitalized) over these same intervals; c)
Supplementary Figure 1A (1) displays average age per the respective interval and we used
displayed medians as city-period specific age (or mean of the individual ages displayed when ≤5
subjects per city-interval); d) for each city, we retrieved average daily temperature for each of
the observed days and calculated overall mean for the respective 2-week period. For the cities
with more than one measurement station, we used data from all stations. Data were obtained
5
from the National Oceanic and Atmospheric Administration (NOAA, www.noaa.gov), except for
the city of Coburg, for which data were not available [and they were not available from the
ncdc.noaa.gov, used in the pre-printed manuscript (1)], and were obtained from ogimet.com.
Each period-by-city was assigned its specific overall mean temperature (for the respective
fortnight). For each period-by-city we also calculated average mean daily temperature for the
index 2-week period, one preceding and one subsequent 2-week period. Data were analyzed in
three steps. First, we performed a simple random-effects meta-analysis of raw proportions of
patients who died and of proportions of patients needing ICU admission by the reported 2-week
intervals by fitting random intercept logistic regression models to logit-transformed
proportions (2-week periods treated as subgroups, method GLMM, maximum-likelihood
estimator for 2 with Hartung-Knapp adjustment). We then performed multiple meta-
regressions by fitting GLMMs to logit-transformed proportions (ML estimator for 2 with
Hartung-Knapp adjustment, with a permutation test) with moderators (i) age, period (as an
ordered number of consecutive intervals) and age*period interaction (both centered) to
evaluate “period” (i.e., elapse of calendar time) as a moderator; (ii) age, mean temperature for
the index fortnight and age*mean temperature (centered) interaction, to evaluate “mean
temperature for the index 2-week period” as a moderator (period was excluded to avoid
collinearity); (iii) age, mean temperature for a 6-week period “around” the hospitalization date,
and their interaction, to evaluate “mean temperature averaged across the preceding, the index
and a subsequent 2-week period”. Meta-analyses and meta-regressions were performed using
package metafor in R (7). Finally, reconstructed data on proportion dying and proportion
needing ICU admission were analyzed (separately) as if coming from a single multi-site
observational study, using the event/trial syntax in Proc Glimmix in SAS 9.4 for Windows (SAS,
Cary, NC) (8), by fitting generalized linear mixed-models [distribution binomial, logit link,
maximum likelihood with Gauss-Hermite quadrature, empirical (sandwich) MBN variance
estimator] (8), with age, overall mean temperature and their interaction (both mean-centered)
as the fixed effects. Individual 2-week periods were clustered into months (February, March,
April, May, June, July) and city*month interaction term was used as a random effect (allowing
for different intercepts for each city-by-month combination). In an additional run, the air
temperature assigned to a particular observed 2-week period was the average of the overall
means for the index, preceding and a subsequent 2-week period.
Results. Meta-analyses of raw proportions (based on the reconstructed data) (Figure 1)
demonstrated a large variation of the number of cities contributing data across the individual 2-
week periods (from the late February to the early July; considered as subgroups) and a large
variation in the number of patients and events. For most of the periods, pooled proportion
6
estimates were largely imprecise and heterogeneity was considerable (Figure 1). Pooled
estimates did not indicate any clear trend of declining proportions of hospitalized patients who
died or of those who needed ICU admission over the observed time (Figure 1). Multiple meta-
regression analyses (Table 1) did not indicate elapsing time or mean daily temperature (mean
for the index fortnight or average of the mean daily temperatures over the index, the preceding
and the subsequent 2-week period) as moderators of proportion dying/needing ICU admission
(Table 1). Higher age appeared consistently associated with higher odds of both outcomes
(Table 1).
Figure 1. Meta-analysis of raw proportions of hospitalized patients who died (A) and of those
needing ICU admission (B) at any time during the index hospitalization (hospitalization dates
grouped by 2-week intervals).
7
Table 1. Summary of the meta-regression models fitted to proportions of hospitalized patients
who died and those who needed ICU admission at any time during the index hospitalization.
Probability of dying
Models and moderators
OR (95%CI)
P
OR (95%CI)
P
Model 1 elapse of calendar time1
2 0.172, I2 69.0%, R2 45.5%
Calendar period (by 2 weeks)
0.96 (0.89-1.05)
0.440
1.09 (0.89-1.33)
0.350
Age (by 1 year)
1.04 (1.02-1.06)
0.001
1.04 (1.00-1.08)
0.056
Age*calendar period
1.00 (0.99-1.01)
0.537
1.00 (0.98-1.02)
0.711
Model 2 air temperature index period2
2 0.162, I2 67.9%, R2 48.8%
Mean temperature (by 1C)
0.98 (0.94-1.03)
0.440
0.97 (0.87-1.07)
0.452
Age (by 1 year)
1.04 (1.02-1.06)
0.001
1.03 (0.99-1.07)
0.075
Age*temperature
1.00 (0.99-1.00)
0.558
1.00 (0.99-1.01)
0.280
Model 3 air temperature peri-index period3
2 0.166, I2 68.5%, R2 47.5%
Mean temperature (by 1C)
0.98 (0.93-1.04)
0.557
0.97 (0.87-1.08)
0.542
Age (by 1 year)
1.04 (1.02-1.06)
0.002
1.03 (0.99-1.08)
0.081
Age*temperature
1.00 (0.99-1.00)
0.498
1.00 (0.99-1.02)
0.358
1Observed 2-week periods (subsuming hospitalization dates) as ordered numbers
2Mean of the mean daily temperatures during the index 2-week period
3Average of the mean daily temperatures for the index 2-week period, a preceding and a
subsequent 2-week period
2- residual heterogeneity, I2- residual unaccounted variability, R2- heterogeneity accounted for
Analysis of data as if coming from a single multi-site observational study (Table 2) did not
suggest associations between the probability of dying/needing ICU with mean temperature.
Older age consistently appeared associated with higher odds of both outcomes (Table 2).
Table 2. Summary of the analysis of probability of dying or needing ICU admission as in a single
multi-site study. City*month was a random effect.
Probability of dying
Probability of ICU admission
Models and effects
OR (95%CI)
P
OR (95%CI)
P
Model 1 “temperature” is the mean of daily temperatures for the index 2-week period
Mean temperature (by 1C)
1.01 (0.94-1.08)
0.775
0.92 (0.81-1.04)
0.187
Age (by 1 year)
1.05 (1.01-1.08)
0.014
1.04 (1.00-1.08)
0.102
Age*temperature
1.00 (0.99-1.01)
0.454
1.00 (0.99-1.01)
0.609
-2 LL, AIC, Chi2/df, 2 intercepts
272.03, 282.03, 0.77, 0.261
308.54, 319.69, 0.75, 6.027
Model 2 “temperature” is average of the mean daily temperatures for the preceding, index and a subsequent 2-week period
Mean temperature (by 1C)
1.01 (0.93-1.09)
0.858
0.89 (0.75-1.07)
0.206
Age (by 1 year)
1.05 (1.01-1.08)
0.012
1.04 (0.99-1.08)
0.071
Age*temperature
1.00 (0.99-1.01)
0.438
1.00 (0.99-1.01)
0.364
-2 LL, AIC, Chi2/df, 2 intercepts
272.13, 282.13, 0.79, 0.249
309.24, 319.24, 0.73, 6.313
-2 LL -2 Log Likelihood; AIC- Akaike’s information criterion; Chi2/df Pearson’s Chi2 over
degrees of freedom; 2 intercepts variance of the intercepts (city*month combinations)
8
Discussion
A number of manuscripts (>300 can be retrieved from PubMed, Scopus, WHO Covid database
and websites publishing pre-prints) have addressed the relationship between meteorological
phenomena and specifically air (ambient) temperature and various clinical and epidemiological
aspects of the COVID-19 pandemic topics beyond our scope. Our aim was to draw the
attention to the importance of adequate gathering, analysis and interpretation of such data. To
exemplify the point, we used a recently pre-printed manuscript (1) that attracted quite some
public attention. Its considerable methodological limitations [as exemplified by two of the
several outcomes reported by the authors (1)] strongly suggest that what was reported by the
authors (1) was an artifact, i.e., a misleading/useless estimate of the reality that arose from a
naïve approach to observational data and to meta-analysis. We would like to point-out that,
considering all the flaws in the structure and quality of the data, the present re-analysis is
equally meaningless it only served the purpose of emphasizing the fact that even such
“naïve” data do not support the claims made by the authors of the pre-printed manuscript (1).
References
1. Kifer D et al. Effects of environmental factors on severity and mortality of COVID-19.
medRxiv 10.1101/2020.07.11.20147157 2020.
2. Rosenbaum PR. Observation and experiment. An introduction to causal inference. Harvard
University Press, Cambridge, MA, 2017.
3. Rothman KJ, Greeland S, Lash TJ (eds). Modern epidemiology, 3rd edition. LWW,
Philadelphia, PA, 2008.
4. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (eds). Introduction to meta-analysis.
Wiley, Chichester, UK, 2009.
5. Cooper H, Hedges LV, Valentine JV (eds). The handbook of research synthesis and meta-
analysis, 3rd ed. Sage Foundation, New York, NY, 2019.
6. Blettner M, Krahn U, Schlattmann P. Meta-analysis in epidemiology. In: Handbook of
epidemiology, 2nd ed, Ahrens W, Pigeot I (eds), Springer, New York, NY, 2017, pp.1377-
1411.
7. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Software
2010; 36:1-48.
8. Kiernan K. Insights into using the GLIMMIX procedure to model categorical outcomes with
random effects. Paper SAS2179-2018.
https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-
proceedings/2018/2179-2018.pdf
Article
Full-text available
The metafor package provides functions for conducting meta-analyses in R. The package includes functions for fitting the meta-analytic fixed- and random-effects models and allows for the inclusion of moderators variables (study-level covariates) in these models. Meta-regression analyses with continuous and categorical moderators can be conducted in this way. Functions for the Mantel-Haenszel and Peto&apos;s one-step method for meta-analyses of 2 x 2 table data are also available. Finally, the package provides various plot functions (for example, for forest, funnel, and radial plots) and functions for assessing the model fit, for obtaining case diagnostics, and for tests of publication bias.
Effects of environmental factors on severity and mortality of COVID-19
  • D Kifer
Kifer D et al. Effects of environmental factors on severity and mortality of COVID-19. medRxiv 10.1101/2020.07.11.20147157 2020.
The handbook of research synthesis and metaanalysis, 3 rd ed. Sage Foundation
  • H Cooper
  • L V Hedges
  • J V Valentine
Cooper H, Hedges LV, Valentine JV (eds). The handbook of research synthesis and metaanalysis, 3 rd ed. Sage Foundation, New York, NY, 2019.
Insights into using the GLIMMIX procedure to model categorical outcomes with random effects
  • K Kiernan
Kiernan K. Insights into using the GLIMMIX procedure to model categorical outcomes with random effects. Paper SAS2179-2018.