PreprintPDF Available

Short-term forecasting of COVID-19 in Germany and Poland during the second wave – a preregistered study

Authors:

Abstract and Figures

We report insights from ten weeks of collaborative COVID-19 forecasting for Germany and Poland (12 October – 19 December 2020). The study period covers the onset of the second wave in both countries, with tightening non-pharmaceutical interventions (NPIs) and subsequently a decay (Poland) or plateau and renewed increase (Germany) in reported cases. Thirteen independent teams provided probabilistic real-time forecasts of COVID-19 cases and deaths. These were reported for lead times of one to four weeks, with evaluation focused on one- and two-week horizons, which are less affected by changing NPIs. Heterogeneity between forecasts was considerable both in terms of point predictions and forecast spread. Ensemble forecasts showed good relative performance, in particular in terms of coverage, but did not clearly dominate single-model predictions. The study was preregistered and will be followed up in future phases of the pandemic.
Content may be subject to copyright.
A preview of the PDF is not available
... Efforts to model and forecast COVID-19 transmission dynamics must therefore meet the challenges of a long and ongoing pandemic spread over an unprecedented scale. Modelling groups around the world have attempted to meet one or both challenges with various analyses conducted at a sub-national scale [51], at a national scale for a specific country [23,[52][53][54], and for several countries across the globe [34,[55][56][57][58]. In contrast to models built for a region ...
... Some of these metrics, such as coverage probability, have well-defined targets that an ideal model should achieve. However, a "good" or "acceptable" range for other commonly used metrics such as absolute error or mean relative error is not always well-defined, and these metrics are typically used to generate a relative ranking of different models [52,62]. Future research could explore the development of standardised performance indicators and evaluation frameworks for forecasts that are tied to public health objectives. ...
... into the models. In addition to the weekly reports that we publish, our work contributed to other international forecasting efforts [23,43,52]. ...
Article
Full-text available
Since 8th March 2020 up to the time of writing, we have been producing near real-time weekly estimates of SARS-CoV-2 transmissibility and forecasts of deaths due to COVID-19 for all countries with evidence of sustained transmission, shared online. We also developed a novel heuristic to combine weekly estimates of transmissibility to produce forecasts over a 4-week horizon. Here we present a retrospective evaluation of the forecasts produced between 8th March to 29th November 2020 for 81 countries. We evaluated the robustness of the forecasts produced in real-time using relative error, coverage probability, and comparisons with null models. During the 39-week period covered by this study, both the short- and medium-term forecasts captured well the epidemic trajectory across different waves of COVID-19 infections with small relative errors over the forecast horizon. The model was well calibrated with 56.3% and 45.6% of the observations lying in the 50% Credible Interval in 1-week and 4-week ahead forecasts respectively. The retrospective evaluation of our models shows that simple transmission models calibrated using routine disease surveillance data can reliably capture the epidemic trajectory in multiple countries. The medium-term forecasts can be used in conjunction with the short-term forecasts of COVID-19 mortality as a useful planning tool as countries continue to relax public health measures.
... Good forecasts are of great interest to decision makers in various fields like finance (Timmermann 2018;Elliott and Timmermann 2016), weather predictions (Gneiting and Raftery 2005;Kukkonen et al. 2012) or infectious disease modeling (Reich et al. 2019;Funk et al. 2020;Cramer et al. 2021;Bracher et al. 2021b;European Covid-19 Forecast Hub 2021). Throughout the COVID-19 pandemic, forecasts from different research institutions on COVID-19 targets like reported cases and deaths have been systematically collated by several collaborative ensemble forecasting efforts ("Forecast Hubs") in the US, Germany and Poland, and Europe. ...
... It also provides functionality to facilitate comparing forecasters even when individual forecasts are missing and offers a range of plotting functions to visualise different aspects of forecast performance. The scoringutils package is also unique in its extensive support for forecasts in a quantile format as used by various COVID-19 Forecast Hubs (Cramer et al. 2021;Bracher et al. 2021b;European Covid-19 Forecast Hub 2021;Bracher et al. 2021c), which also make use of scoringutils for their evaluations. ...
Preprint
Full-text available
Evaluating forecasts is essential in order to understand and improve forecasting and make forecasts useful to decision-makers. Much theoretical work has been done on the development of proper scoring rules and other scoring metrics that can help evaluate forecasts. In practice, however, conducting a forecast evaluation and comparison of different forecasters remains challenging. In this paper we introduce scoringutils, an R package that aims to greatly facilitate this process. It is especially geared towards comparing multiple forecasters, regardless of how forecasts were created, and visualising results. The package is able to handle missing forecasts and is the first R package to offer extensive support for forecasts represented through predictive quantiles, a format used by several collaborative ensemble forecasting efforts. The paper gives a short introduction to forecast evaluation, discusses the metrics implemented in scoringutils and gives guidance on when they are appropriate to use, and illustrates the application of the package using example data of forecasts for COVID-19 cases and deaths submitted to the European Forecast Hub between May and September 2021
... Several attempts have been made to propose rigorous evaluation strategies to compare the forecast capabilities of different models; see, e.g., [25,26]. Ensemble approaches combining multiple available forecasts from different models have proved to outperform single models in forecasting the early phases of pandemic [27,28,29], as well as influenza [30,31], dengue fever [32], and Ebola [33] outbreaks. The superior performance of ensemble forecast have been shown in different disciplines and they are typically associated to the ability of integrating the information coming from different models producing accurate predictions with well-calibrated uncertainty [34,35], avoiding, at the same time, the risk of delayed (or missing) forecasts or premature interruption of the forecast supply, which may occur when relying on a single specific model. ...
Article
Full-text available
Several epidemiological models have been proposed to study the evolution of COVID-19 pandemic. In this paper, we propose an extension of the SUIHTER model, to analyse the COVID-19 spreading in Italy, which accounts for the vaccination campaign and the presence of new variants when they become dominant. In particular, the specific features of the variants (e.g. their increased transmission rate) and vaccines (e.g. their efficacy to prevent transmission, hospitalization and death) are modeled, based on clinical evidence. The new model is validated comparing its near-future forecast capabilities with other epidemiological models and exploring different scenario analyses.
... Several attempts have been made to propose rigorous evaluation strategies to compare the forecast capabilities of different models; see, e.g., [21,22]. Ensemble approaches combining multiple available forecasts from different models have proved to outperform single models in forecasting the early phases of COVID-19 pandemic [23,24,25], as well as influenza [26,27], dengue fever [28], and Ebola [29] outbreaks. The superior performance of ensemble forecast have been shown in different disciplines and they are typically associated to the ability of integrating the information coming from different models producing accurate predictions with well-calibrated uncertainty [30,31], avoiding, at the same time, the risk of delayed (or missing) forecasts or premature interruption of the forecast supply, which may occur when relying on a single specific model. ...
Preprint
Several epidemiological models have been proposed to study the evolution of COVID-19 pandemic. In this paper, we propose an extension of the SUIHTER model, first introduced in [Parolini et al, Proc R. Soc. A., 2021] to analyse the COVID-19 spreading in Italy, which accounts for the vaccination campaign and the presence of new variants when they become dominant. In particular, the specific features of the variants (e.g. their increased transmission rate) and vaccines (e.g. their efficacy to prevent transmission, hospitalization and death) are modeled, based on clinical evidence. The new model is validated comparing its near-future forecast capabilities with other epidemiological models and exploring different scenario analyses.
... Viboud et al. [30] proposed a simple generalized-growth model for the early phase of disease outbreaks. In [31], different stochastic and deterministic compartment and agent-based models for Germany and Poland from, i.a., [16,[32][33][34][35][36], were compared. ...
Article
Full-text available
Background Despite the vaccination process in Germany, a large share of the population is still susceptible to SARS-CoV-2. In addition, we face the spread of novel variants. Until we overcome the pandemic, reasonable mitigation and opening strategies are crucial to balance public health and economic interests. Methods We model the spread of SARS-CoV-2 over the German counties by a graph-SIR-type, metapopulation model with particular focus on commuter testing. We account for political interventions by varying contact reduction values in private and public locations such as homes, schools, workplaces, and other. We consider different levels of lockdown strictness, commuter testing strategies, or the delay of intervention implementation. We conduct numerical simulations to assess the effectiveness of the different intervention strategies after one month. The virus dynamics in the regions (German counties) are initialized randomly with incidences between 75 and 150 weekly new cases per 100,000 inhabitants (red zones) or below (green zones) and consider 25 different initial scenarios of randomly distributed red zones (between 2 and 20% of all counties). To account for uncertainty, we consider an ensemble set of 500 Monte Carlo runs for each scenario. Results We find that the strength of the lockdown in regions with out of control virus dynamics is most important to avoid the spread into neighboring regions. With very strict lockdowns in red zones, commuter testing rates of twice a week can substantially contribute to the safety of adjacent regions. In contrast, the negative effect of less strict interventions can be overcome by high commuter testing rates. A further key contributor is the potential delay of the intervention implementation. In order to keep the spread of the virus under control, strict regional lockdowns with minimum delay and commuter testing of at least twice a week are advisable. If less strict interventions are in favor, substantially increased testing rates are needed to avoid overall higher infection dynamics. Conclusions Our results indicate that local containment of outbreaks and maintenance of low overall incidence is possible even in densely populated and highly connected regions such as Germany or Western Europe. While we demonstrate this on data from Germany, similar patterns of mobility likely exist in many countries and our results are, hence, generalizable to a certain extent.
Article
Full-text available
This paper presents a description of the methodology developed for estimation of pathogen transmission in transport and the results of the case study application for long-distance passenger transport. The primary objective is to report the method developed and the application for case studies in various passenger transport services. The most important findings and achievements of the presented study are the original universal methodology to estimate the probability of pathogen transmission with full mathematical disclosure and an open process formula, to make it possible to take other specific mechanisms of virus transmission when providing transport services. The results presented conducted an analysis on the mechanisms of transmission of SARS-CoV-2 virus pathogens during the transport process, to examine the chain of events as a result of which passengers may be infected. The author proposed a new method to estimate the probability of transmission of viral pathogens using the probability theory of the sum of elementary events. This is a new approach in this area, the advantage of which is a fully explicit mathematical formula that allows the method to be applied to various cases. The findings of this study can facilitate the management of epidemic risk in passenger transport operators and government administration. It should be clearly emphasised that the developed method and estimated values are the probabilities of pathogen transmission. Estimating the probability of transmission of the SARS-CoV-2 virus pathogen is not the same as the probability of viral infection, and more so the probability of contracting COVID-19. Viral infection strongly depends on viral mechanisms, exposure doses, and contact frequency. The probability of contracting COVID-19 and its complications depends on the individual characteristics of the immune system, even with confirmed viral infection. However, it is undoubtedly that the probability of transmission of the SARS-CoV-2 virus pathogen is the most reliable measure of infection risk, which can be estimated according to the objective determinants of pathogen transmission.
Article
Background: In the past, two studies found ensembles of human judgement forecasts of COVID-19 to show predictive performance comparable to ensembles of computational models, at least when predicting case incidences. We present a follow-up to a study conducted in Germany and Poland and investigate a novel joint approach to combine human judgement and epidemiological modelling. Methods: From May 24th to August 16th 2021, we elicited weekly one to four week ahead forecasts of cases and deaths from COVID-19 in the UK from a crowd of human forecasters. A median ensemble of all forecasts was submitted to the European Forecast Hub. Participants could use two distinct interfaces: in one, forecasters submitted a predictive distribution directly, in the other forecasters instead submitted a forecast of the effective reproduction number Rt. This was then used to forecast cases and deaths using simulation methods from the EpiNow2 R package. Forecasts were scored using the weighted interval score on the original forecasts, as well as after applying the natural logarithm to both forecasts and observations. Results: The ensemble of human forecasters overall performed comparably to the official European Forecast Hub ensemble on both cases and deaths, although results were sensitive to changes in details of the evaluation. Rt forecasts performed comparably to direct forecasts on cases, but worse on deaths. Self-identified “experts” tended to be better calibrated than “non-experts” for cases, but not for deaths. Conclusions: Human judgement forecasts and computational models can produce forecasts of similar quality for infectious disease such as COVID-19. The results of forecast evaluations can change depending on what metrics are chosen and judgement on what does or doesn't constitute a "good" forecast is dependent on the forecast consumer. Combinations of human and computational forecasts hold potential but present real-world challenges that need to be solved.
Article
We report on a course project in which students submit weekly probabilistic forecasts of two weather variables and one financial variable. This real-time format allows students to engage in practical forecasting, which requires a diverse set of skills in data science and applied statistics. We describe the context and aims of the course, and discuss design parameters like the selection of target variables, the forecast submission process, the evaluation of forecast performance, and the feedback provided to students. Furthermore, we describe empirical properties of students’ probabilistic forecasts, as well as some lessons learned on our part.
Article
Full-text available
Zusammenfassung Hintergrund Zeitdynamische Prognosemodelle spielen eine zentrale Rolle zur Steuerung von intensivmedizinischen COVID-19-Kapazitäten im Pandemiegeschehen. Ein wichtiger Vorhersagewert (Prädiktor) für die zukünftige intensivmedizinische (ITS-)COVID-19-Bettenbelegungen ist die Anzahl der SARS-CoV-2-Neuinfektionen in der Bevölkerung, die wiederum stark von Schwankungen im Wochenverlauf, Meldeverzug, regionalen Unterschieden, Dunkelziffer, zeitabhängiger Ansteckungsrate, Impfungen, SARS-CoV-2-Virusvarianten sowie von nichtpharmazeutischen Eindämmungsmaßnahmen abhängt. Darüber hinaus wird die aktuelle und auch zukünftige COVID-ITS-Belegung maßgeblich von den intensivmedizinischen Entlassungs- und Sterberaten beeinflusst. Methode Sowohl die Anzahl der SARS-CoV-2-Neuinfektionen in der Bevölkerung als auch die intensivmedizinischen COVID-19-Bettenbelegungen werden bundesweit flächendeckend erfasst. Diese Daten werden tagesaktuell mit epidemischen SEIR-Modellen aus gewöhnlichen Differenzialgleichungen und multiplen Regressionsmodellen statistisch analysiert. Ergebnisse Die Prognoseergebnisse der unmittelbaren Entwicklung (20-Tage-Vorhersage) der ITS-Belegung durch COVID-19-Patienten*innen werden Entscheidungsträgern auf verschiedenen überregionalen Ebenen zur Verfügung gestellt. Schlussfolgerung Die Prognosen werden der Entwicklung von betreibbaren intensivmedizinischen Bettenkapazitäten gegenübergestellt, um frühzeitig Kapazitätsengpässe zu erkennen und kurzfristig reaktive Handlungssteuerungen, wie etwa überregionale Verlegungen, zu ermöglichen.
Article
Full-text available
An operationally implementable predictive model has been developed to forecast the number of COVID-19 infections in the patient population, hospital floor and ICU censuses, ventilator and related supply chain demand. The model is intended for clinical, operational, financial and supply chain leaders and executives of a comprehensive healthcare system responsible for making decisions that depend on epidemiological contingencies. This paper describes the model that was implemented at NorthShore University HealthSystem and is applicable to any communicable disease whose risk of reinfection for the duration of the pandemic is negligible.
Article
Full-text available
For practical reasons, many forecasts of case, hospitalization, and death counts in the context of the current Coronavirus Disease 2019 (COVID-19) pandemic are issued in the form of central predictive intervals at various levels. This is also the case for the forecasts collected in the COVID-19 Forecast Hub (https://covid19forecasthub.org/). Forecast evaluation metrics like the logarithmic score, which has been applied in several infectious disease forecasting challenges, are then not available as they require full predictive distributions. This article provides an overview of how established methods for the evaluation of quantile and interval forecasts can be applied to epidemic forecasts in this format. Specifically, we discuss the computation and interpretation of the weighted interval score, which is a proper score that approximates the continuous ranked probability score. It can be interpreted as a generalization of the absolute error to probabilistic forecasts and allows for a decomposition into a measure of sharpness and penalties for over- and underprediction.
Article
Full-text available
The novel coronavirus (SARS-CoV-2), identified in China at the end of December 2019 and causing the disease COVID-19, has meanwhile led to outbreaks all over the globe with about 2.2 million confirmed cases and more than 150,000 deaths as of April 17, 2020. In this work, mathematical models are used to reproduce data of the early evolution of the COVID-19 outbreak in Germany, taking into account the effect of actual and hypothetical non-pharmaceutical interventions. Systems of differential equations of SEIR type are extended to account for undetected infections, stages of infection, and age groups. The models are calibrated on data until April 5. Data from April 6 to 14 are used for model validation. We simulate different possible strategies for the mitigation of the current outbreak, slowing down the spread of the virus and thus reducing the peak in daily diagnosed cases, the demand for hospitalization or intensive care units admissions, and eventually the number of fatalities. Our results suggest that a partial (and gradual) lifting of introduced control measures could soon be possible if accompanied by further increased testing activity, strict isolation of detected cases, and reduced contact to risk groups.
Article
Full-text available
Seasonal influenza results in substantial annual morbidity and mortality in the United States and worldwide. Accurate forecasts of key features of influenza epidemics, such as the timing and severity of the peak incidence in a given season, can inform public health response to outbreaks. As part of ongoing efforts to incorporate data and advanced analytical methods into public health decision-making, the United States Centers for Disease Control and Prevention (CDC) has organized seasonal influenza forecasting challenges since the 2013/2014 season. In the 2017/2018 season, 22 teams participated. A subset of four teams created a research consortium called the FluSight Network in early 2017. During the 2017/2018 season they worked together to produce a collaborative multi-model ensemble that combined 21 separate component models into a single model using a machine learning technique called stacking. This approach creates a weighted average of predictive densities where the weight for each component is determined by maximizing overall ensemble accuracy over past seasons. In the 2017/2018 influenza season, one of the largest seasonal outbreaks in the last 15 years, this multi-model ensemble performed better on average than all individual component models and placed second overall in the CDC challenge. It also outperformed the baseline multi-model ensemble created by the CDC that took a simple average of all models submitted to the forecasting challenge. This project shows that collaborative efforts between research teams to develop ensemble forecasting approaches can bring measurable improvements in forecast accuracy and important reductions in the variability of performance from year to year. Efforts such as this, that emphasize real-time testing and evaluation of forecasting models and facilitate the close collaboration between public health officials and modeling researchers, are essential to improving our understanding of how best to use forecasts to improve public health response to seasonal and emerging epidemic threats.
Article
Full-text available
Infectious disease outbreaks play an important role in global morbidity and mortality. Real-time epidemic forecasting provides an opportunity to predict geographic disease spread as well as case counts to better inform public health interventions when outbreaks occur. Challenges and recent advances in predictive modeling are discussed here. We identified data needs in the areas of epidemic surveillance, mobility, host and environmental susceptibility, pathogen transmissibility, population density, and healthcare capacity. Constraints in standardized case definitions and timely data sharing can limit the precision of predictive models. Resource-limited settings present particular challenges for accurate epidemic forecasting due to the lack of granular data available. Incorporating novel data streams into modeling efforts is an important consideration for the future as technology penetration continues to improve on a global level. Recent advances in machine-learning, increased collaboration between modelers, the use of stochastic semi-mechanistic models, real-time digital disease surveillance data, and open data sharing provide opportunities for refining forecasts for future epidemics. Epidemic forecasting using predictive modeling is an important tool for outbreak preparedness and response efforts. Despite the presence of some data gaps at present, opportunities and advancements in innovative data streams provide additional support for modeling future epidemics.
Article
Full-text available
Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and bias of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time for Western Area, Sierra Leone, during the 2013–16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but model predictions were increasingly unreliable at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making based on predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts.
Article
Full-text available
Infectious disease forecasting is gaining traction in the public health community; however, limited systematic comparisons of model performance exist. Here we present the results of a synthetic forecasting challenge inspired by the West African Ebola crisis in 2014-2015 and involving 16 international academic teams and US government agencies, and compare the predictive performance of 8 independent modeling approaches. Challenge participants were invited to predict 140 epidemiological targets across 5 different time points of 4 synthetic Ebola outbreaks, each involving different levels of interventions and "fog of war" in outbreak data made available for predictions. Prediction targets included 1-4 week-ahead case incidences, outbreak size, peak timing, and several natural history parameters. With respect to weekly case incidence targets, ensemble predictions based on a Bayesian average of the 8 participating models outperformed any individual model and did substantially better than a null auto-regressive model. There was no relationship between model complexity and prediction accuracy; however, the top performing models for short-term weekly incidence were reactive models with few parameters, fitted to a short and recent part of the outbreak. Individual model outputs and ensemble predictions improved with data accuracy and availability; by the second time point, just before the peak of the epidemic, estimates of final size were within 20% of the target. The 4th challenge scenario - mirroring an uncontrolled Ebola outbreak with substantial data reporting noise - was poorly predicted by all modeling teams. Overall, this synthetic forecasting challenge provided a deep understanding of model performance under controlled data and epidemiological conditions. We recommend such "peace time" forecasting challenges as key elements to improve coordination and inspire collaboration between modeling groups ahead of the next pandemic threat, and to assess model forecasting accuracy for a variety of known and hypothetical pathogens.
Article
To rapidly evaluate the safety and efficacy of COVID-19 vaccine candidates, prioritizing vaccine trial sites in areas with high expected disease incidence can speed endpoint accrual and shorten trial duration. Mathematical and statistical forecast models can inform the process of site selection, integrating available data sources and facilitating comparisons across locations. We recommend the use of ensemble forecast modeling - combining projections from independent modeling groups - to guide investigators identifying suitable sites for COVID-19 vaccine efficacy trials. We describe an appropriate structure for this process, including minimum requirements, suggested output, and a user-friendly tool for displaying results. Importantly, we advise that this process be repeated regularly throughout the trial, to inform decisions about enrolling new participants at existing sites with waning incidence versus adding entirely new sites. These types of data-driven models can support the implementation of flexible efficacy trials tailored to the outbreak setting.
Mitigation and herd immunity strategy for covid-19 is likely to fail
  • B Adamik
  • M Bawiec
  • V Bezborodov
  • W Bock
  • M Bodych
  • J P Burgard
  • T Götz
  • T Krueger
  • A Migalska
  • B Pabjan
  • T Ożański
  • E Rafaj Lowicz
  • W Rafaj Lowicz
  • E Skubalska-Rafaj Lowicz
  • S Ryfczyńska
  • E Szczurek
  • P Szymański
Adamik, B., Bawiec, M., Bezborodov, V., Bock, W., Bodych, M., Burgard, J. P., Götz, T., Krueger, T., Migalska, A., Pabjan, B., Ożański, T., Rafaj lowicz, E., Rafaj lowicz, W., Skubalska-Rafaj lowicz, E., Ryfczyńska, S., Szczurek, E., and Szymański, P. (2020). Mitigation and herd immunity strategy for covid-19 is likely to fail. medRxiv, https://www.medrxiv.org/content/10.1101/2020.03.25.20043109v1.
Quo vadis coronavirus? Rekomendacje zespo lów epidemiologii obliczeniowej na rok 2021
  • A Afelt
  • R Bartczuk
  • P Biecek
  • M Bodych
  • A Gambin
  • K Gogolewski
  • A Kaczorek
  • J Kisielewski
  • T Krüger
  • Migalska
  • R Miko Lajczyk
  • A Moszyński
  • K Niedzielewski
  • A Nowosielski
  • B Pabjan
  • M Radwan
  • F Rakowski
  • M Rosińska
  • M Semeniuk
  • J Zieliński
Afelt, A., Bartczuk, R., Biecek, P., Bodych, M., Gambin, A., Gogolewski, K., Kaczorek, A., Kisielewski, J., Krüger, T., Migalska, Miko lajczyk, R., Moszyński, A., Niedzielewski, K., Nowosielski, A., Pabjan, B., Radwan, M., Rakowski, F., Rosińska, M., Semeniuk, M., and Zieliński, J. (2020). Quo vadis coronavirus? Rekomendacje zespo lów epidemiologii obliczeniowej na rok 2021. Available online at https://quovadis. crs19.pl/ (last accessed 22 December 2020).