Content uploaded by Drazen Ignjatovic
Author content
All content in this area was uploaded by Drazen Ignjatovic on Jan 20, 2022
Content may be subject to copyright.
THE BENEFITS OF TRIALS FOR PANDEMIC
MANAGEMENT
Georg Neubauer, Dražen Ignjatović, Gerald Schimak
AIT Austrian Institute of Technology GmbH
georg.neubauer@ait.ac.at, drazen.ignjatovic@ait.ac.at, gerald.schimak@ait.ac.at
Adam Widera, Michael Middelhoff
University Munster
adam.widera@wi.uni-muenster.de, michael.middelhoff@wi.uni-muenster.de
Sabrina Scheuer, Georg Aumayr
Johanniter Österreich
sabrina.scheuer@johanniter.com, georg.aumayr@johanniter.at
Eva Jaho
EXUS
e.jaho@exus.co.uk
Karin Rainer
AGES Austrian Agency for Health and Food Safety
karin.rainer@ages.at
Keywords
disaster, pandemic management, COVID 19, trials
Abstract
Evaluations in the frame of the EU research projects DRIVER+ and STAMINA revealed that, aside
from the multitude of solutions that can be applied to manage a pandemic (restraining measures,
protective equipment, tests for epidemiological observations, therapies and vaccines, tools for
predictions, resource planning and monitoring, IT solutions for public services), there is also a need
for procedures on how to systematically evaluate such solutions. End users, e.g. in medical services
have to ensure that new solutions fit to their established pandemic management processes in order
to avoid unexpected side effects on the execution of their tasks. In this paper we show a new
methodology designed to trial pandemic solutions and provide insights based on the different
national trials of STAMINA.
1. Introduction
The analysis of national and international surveys conducted in 2020 as part of the European flagship
project DRIVER+ has shown that there is a multitude of very different challenges in the ongoing
management of the COVID-19 crisis. This is including not only aspects of resource availability, but
also issues of data management and communication (VISUAL CAPITALIST, 2020). At the
beginning of the pandemic, the requirements to facilitate mitigation measures and their effectiveness
were mostly unclear. Specifically in regards of transmission ways and infectiousness, the knowledge
base rapidly changed and developed as a requirement of the changing/mutating virus (RKI 2021).
Due to the very large number of potentially available solutions, stakeholders worldwide and also in
Austria were exposed to an information overload of proposals for solutions such as tests, masks or
IT solutions and of approaches for implementing measures, especially in the early phase of the
COVID-19 crisis (John Hopkins University, 2020). This shows the complexity of the challenge: the
stakeholders involved in pandemic management, such as state and federal authorities, operators of
infrastructure such as hospitals and emergency organisations, need optimised processes and
appropriate solutions to carry them out. However, there was and is no systematic overview of the
very heterogeneous set of solutions such as suitable diagnostic options, masks but also IT solutions.
There is also no possibility to automatically check the suitability of the solutions for the specific
requirements and to compare them with other offers. And finally, in case of identification of one or
several solutions potentially suitable to close gaps and needs according to the available descriptions,
the end user faces the problem on how to systematically validate the efficacy and efficiency of the
solutions.
In September 2020 the H2020 project STAMINA
1
was started. The purpose of the project is to
support overcoming the challenges arising from infectious diseases that may result in serious cross-
border public health threats represented by e.g. COVID-19. STAMINA has multiple goals such as
the provision of novel monitoring and decision support tools, encompassing an early warning system,
crisis simulation tools, real time web and social media analytics as well as a Common Operational
Picture platform. Moreover, new rapid tests for increased diagnostic capability for Point of Care
1
https://stamina-project.eu/
application and smart wearable devices for health monitoring are provided. These solutions are all
going to be the tested in trials according the STADEM Methodology, which is based on the Trial
Guidance Methodology (TGM). In order to address the need for enabling practitioners to
systematically and objectively investigate the performance of solutions, the TGM was developed in
the frame of the European flagship project DRIVER+, Driving Innovation for European Resilience
(DRIVER+, 2020). The methodology is described in detail in the CEN Workshop Agreement
17514:2020 (CEN Workshop Agreement 2020). The introduction of new solutions can be risky for
practitioners, because they are often associated with significant costs and in case they are not
appropriate, significant workflow disruptions. The objective of the Trial Guidance Methodology is
to investigate the impact of new solutions on existing processes, and will be described in Section 1
of this paper.
2. Method – the STADEM Approach
The Trial Guidance Methodology (TGM), created by the EU research project DRIVER+ is designed
to develop a rigorous, yet pragmatic methodology for the assessment of innovative solutions in the
area of crisis management (Fonio, Widera 2020). It follows three ambitious objectives:
• enable practitioners to systematically assess the added value of an innovative solution,
• enhance dialogue and co-operation among solution providers and practitioners, and
• support the goal to have a more objective assessment in the procurement process.
The TGM was already executed and evaluated in multiple trial events (see tgm.ercis.de). A summary
of the key phases of the TGM for the application and further development in the domain of pandemic
management is given below. After that, necessary adaptations of the TGM towards the STADEM
methodology of STAMINA are discussed.
According to the TGM a trial takes into account three dimensions (Widera et al. 2018), which need
to be considered in order to allow the evaluation of a trial. The solution dimension states the
objectives and functions that the individual solution tries to reach in the trial context. The crisis
management dimension states the objectives for the improvement in the activities to address the
situation and the roles, responsibilities and processes needed for that. The trial dimension states the
objectives for the overall trial and the sessions it is composed of. All three dimensions contribute to
dedicated performance measurements to assess their individual influence on the overall achieved
outcome.
2.1.1. Overview on the phases and steps
The trial guidance methodology consists of three phases: preparation, execution and evaluation. The
goal of a trial is the systematic assessment of an innovative solution’s added value to a specific crisis
management (CM) context and gap or need. Hence the evaluation phase is the most important phase
and should be taken into account from the beginning.
The core of a trial are the gaps that should be addressed by conducting an assessment. Every gap is
based on the underlying need to improve the current situation and is thereby directly related to CM
functions (DRIVER+ Project Public Reports 2019). A gap should be reasoned by formulating the
need to improve and describe current deficiencies, as this facilitates the understanding for non-CM
professionals. Hence, it is recommended that CM professionals are involved in the identification and
formulation of the trial gaps.
As the gap depicts a potential for improvement, it is linked to roles, responsibilities, situations,
equipment etc. that describe the situation in which this specific gap occurs.
Based on a proper identification of gaps and the trial context, the trial design is developed in a six
step approach (Fonio, C. and Widera, A. 2020). The six step approach is an iterative process in which
individual steps, in follow up iterations, might not require an update and hence can be skipped.
The process is co-creative, which means that all stakeholders should be involved. .
The execution phase also foresees preparation, rehearsals and tests in order to assure a smooth event.
This will also allow for the possibility to adjust and check that all that is needed for the trial is in
place and is functioning properly.
A trial integration meeting is the first joint physical meeting of the CM practitioners, the technical
test-bed and the solution providers. This trial integration meeting is essential to align the perspectives
of the practitioners, the solution providers and the trial committee.
A first dry run one is a technical rehearsal, and its main purpose is to test the technical set-up (the
trial design and the technical test-bed), the connection of the solutions to the test-bed and the
information exchange between solutions as well as the data collection set-up. The training on
solutions should also be validated.
The second dry run two is a full rehearsal possibly still with a limited number of participants. This is
the final possibility to adjust the trial set-up. The data collection plan and the trial script should be
approved after dry run two, otherwise the trial owner might setup another dry run before the trial
itself.
The trial run aims at collecting all the data necessary for the evaluation of the solutions by comparing
the baseline with the use of the innovative selected solutions. The total duration of the trial run
depends on the complexity of the scenario, as well as the number of gaps and solutions to be assessed.
The main objective of the evaluation phase is to make sense of the data collected during the trial.
This sense making process is key to understand if and to what extent the solutions are relevant and
innovative for a CM organization. The evaluation phase is dedicated to analyse the quality of the data
and the data, to synthesize it and finally to disseminate the results..
2.1.2. Adaptations towards STADEM
The development and evaluation of the TGM was so far carried out in the context of the DRIVER+
project. While the TGM is a very good starting point as methodology to be used in STAMINA, some
changes are necessary and potential further improvements could be evaluated. This section discusses
first steps in the development of the STADEM methodology. During the first steps of preparing the
STAMINA trials, practitioners presented a national pandemic management map for the respective
country the trial is planned in. This map is a schematic representation of the pandemic response units
and authorities in a country and how they are linked to each other. Such an overview can evidently
support the definition and description of the trial context in the preparation phase. It was therefore
suggested to all trial owners to prepare such a pandemic management map in the trial preparation.
Later evaluations will show if such a tool can effectively support the trial design and whether this is
a pandemic specific utility. A pandemic management map is intended to become a new outcome in
the step zero. One clear difference from the TGM to STAMINA objectives is the omitting of a
solution selection process. STAMINA foresees a predefined solution set for the trials, which means
that the solutions to be assessed are already known.
On the one hand, this limits the openness of the methodology since the solution selection is not based
on the trial context, on the other hand this should accelerate the iterative six step approach and allows
trial owners to directly assess one or more already predefined solutions. The first iteration can already
consider solutions in the trial context and no later changes due to newly identified solutions are
required. Yet, this demands that the trial scenario considers the capabilities of the solutions right from
the beginning, in order to conclude on achievable trial objectives. Evaluations will result in necessary
changes to the phases for STADEM. The TGM and STADEM both focus on CM practitioner interests
and support the collaboration of trial owners, practitioners and solution providers together with CM
experts. The pragmatic applicability of STADEM for the involved stakeholders is therefore a key
performance indicator.
3. Setting of the STAMINA Trials
A central goal of STAMINA is to validate if the solutions provided in the project can close pandemic
management gaps specified by end users. As the current practice shows, this is a highly relevant step
in the process of optimizing the current, mainly organically grown structures adapted from other
management practices and thus not always suitable for the highly dynamic and fluid demands of a
global or cross-border event as pandemics or epidemics. The methodology applied is described in the
previous chapter. In this chapter we give an overview on the different trials of STAMINA and
describe the Austrian trial in an exemplary way in more detail.
STAMINA does not focus on SARS-CoV-2 only, but also deals with influenza, the West Nile Virus,
measles and e.Coli as well. For the trials, this means that the test scenario should be elaborated in
such a way that at least one of the above mentioned pathogens have to be involved.
Additionally, each trial focuses on one or more types of gaps arising during a pandemic.
Different types of solutions
2
are provided in STAMINA, namely:
• STAMINA predictive models (1),
• a portable health status monitoring device (2)
• rapid tests for diagnostic capability at Point of Care (3)
• and the STAMINA toolset encompassing an
o Early warning system (4)
o Crisis simulation tool (5)
o Real time web and social media analytics (6)
o Preparedness Pandemic Training tool (7)
o Common Operational Picture application (8)
In a similar manner as with the pathogens each trial needs to be designed by the specific trial
coordination team in such a way that the performance of several of the solutions can be investigated.
In total up to 12 national trials and one cross border trial are going to be executed in STAMINA.
Table 1 gives insight about the planned trials. It should be noticed that all trials are in the early
planning stages, therefore all parameters are potentially a subject to change. In particular the degree
of specification of objectives as well as gaps is going to be continuously sharpened.
2
The numbers in parenthesis are used in table 1 to indicate if these types of solutions are intended to be applied in the
trials
Country3
Objectives/Gap(s)
Applied solutions
Austria
Back tracing of potential infected persons
2,3,7,8
France
Rapid diagnostic to identify West Nile
Virus, identify people at risk
3,4,6,7
Czech Republic
Training of control of pathogen, situational
awareness
3,4,8,7
Greece
Cooperation between federal institutions
2,3,4,7
Lithuania
Evaluation of health threats a point of
entries
3,8,7
Romania
Management of the outbreak of an
unknown pathogen in parallel to an
earthquake
5,8
Slovenia
Hospital resource management, rapid
testing
3,4,5,6,8
Spain
Coordination between authorities and first
responders
4,6,8
Turkey
Prediction of shortage of hospital resources
1,2,4,8
UK
Understanding and modeling the trajectory
of pandemics
1,8
Table 1: Overview on STAMINA trials objectives and gaps as well as potentially applied solutions
A highly pertinent cross-border trial will be also implemented in the frame of the project that will be
prepared. It can be seen from Table 1 that the application of the different (types of) solutions is not
balanced, e.g. the predictive models are only involved in two trials so far, whereas the COP is
intended to be applied in 80 % of the trials. So far, there also exists also a predominance of the
pathogen SARS-CoV-2 across trials, whereas other pathogens are so far not in the focus of the trial
committees. It is intended to balance the selection of pathogens, too. In order to get a better insight
in the planning of a specific trial, the Austrian trial is described in more detail here.
The Austrian trial is planned and executed by the Johanniter Österreich Ausbildung und Forschung
gemeinnützige GmbH, who are part of the STAMINA project as a medical first responder. During
normal operation the Johanniter have 62 ambulances with more than 700 paramedics on a 24/7 basis
in Vienna in action. Lately the transport of infected patients is of increased concern, due to COVID-
19. If an ambulance is transporting an infected patient without prior knowledge, this leads to a
potential spreading of the infection in the organization as well as the later approached hospitals,
because the ambulance car is not disinfected sufficiently concerning the special infectious disease
(e.g. MRSA; Scabies) after every transport. Therefore, the fast and efficient tracing of the contact
points of the ambulance car is the identified gap of the Austrian trial. To test the relevant solutions
(2, 3, 7, 8) addressing this gap, four different trial objectives were identified:
3
Two additional trials are being considered to be held in the Netherlands and Tunisia
1. Information processing and visualization through a new situational awareness and decision
support system in the control center of the Johanniter Vienna;
2. New tools for the preparedness, early detection and control of pathogens and diseases in the
patient and emergency transport should be assessed;
3. Establishment and testing of new communication guidelines and a documentation system in
the organization of the Johanniter;
4. Assessment of innovative contact tracing solutions with portable PCR devices and SmartKo
wearable diagnostic devices in the organization of the Johanniter in Vienna.
Within the trial, the contact points of the respective ambulance car should be identified by the
command center through the documentation and contacted via a predefined procedure which was
already tested and trained with the Preparedness Pandemic Training tool. The ambulance team and
close contacts of them will get a health monitoring device and are tested with the rapid diagnostic
tools at different times. The whole situation should be monitored using the COP-Tool.
4. Conclusion and Outlook
The main purpose of trials is to investigate whether solutions can satisfyingly fill gaps of practitioners
and end users in pandemic management. It is important to keep in mind that gaps can occur at
different levels of pandemic management, operational, tactical, strategic, or even legal or political.
For each of these areas, different types of solutions may be helpful to meet existing requirements and
needs. In any case, the processes used must be considered when discussing gaps and solutions
dedicated to those gaps (see also Rainer et al. 2021). In addition to validating the suitability of
solutions, it is necessary to have a comprehensive overview of solutions. To achieve this, all solutions
tested in the STAMINA project will be systematically documented together with the results of the
validations in an already existing platform, the Portfolio of Solutions (PoS). The Portfolio of
Solutions is an online knowledge management system that stores information about gaps in crisis
management, solutions, and experiences in using these solutions, and makes this information
available to end users through a web interface (Ignjatović et al. 2019). The PoS is used and adapted
for the purposes of STAMINA to describe pandemic management solutions as well as to match the
characteristics of these solutions with specific management gaps via a specific taxonomy of pandemic
management capabilities. This type of matching between solutions and gaps is very useful for end
users to identify solutions that are potentially suitable for their needs before their suitability is
investigated in the context of the trials described earlier.
4.1. Acknowledgment
The research leading to these results has received funding from the European Union`s Horizon 2020
research and innovation programme under Grant Agreement n°883441.
5. References
CEN Workshop Agreement (2020) Systematic assessment of innovative solutions for crisis
management - Trial guidance methodology [pdf]. Retrieved from
https://ftp.cencenelec.eu/EN/ResearchInnovation/CWA/CWA17514_2020.pdf.
DRIVER+, (2019) Project DRIVER+. [online]. Available at: <https://www.driver-project.eu/> [Accessed 2 April 2021].
DRIVER+ Project Public Reports (2019). D934.10 - TAXONOMY OF CM FUNCTIONS FOR CLASSIFICATION
OF SOLUTIONS [pdf]. Retrieved from https://www.driver-project.eu/wp-content/uploads/2018/02/DRIVER-
_D934.10_Taxonomy-of-CM-functions-for-classification-of-solutions.pdf
Fonio, C., Widera, A. (Ed.), (2020) Trial Guidance Methodology Handbook. DRIVER+ (Driving Innovation in Crisis
Management for European Resilience), Brussels, 2020.
Ignjatović, D., Havlik, D., Neubauer, G., Turptil, S., Gonzales, F., Regeczi, D.: The Portfolio of Solutions. In:
Proceedings of the 27th Interdisciplinary Information Management Talks, Kutná Hora, Czech Republic, 4 – 6
September 2019, pp199-206
John Hopkins University, (2020). COVID 19 Dashboard. [online]. Available at: <
https://gisanddata.maps.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6/>
[Accessed 23 April 2021].
Rainer, K., Leidwein, A., Nemenz. P., Hoffmann, M., Fastl, C., Rathammer, K., Neubauer, G., Preinerstorfer, A.,
Aumayr, G., Scheuer, S.; Eisenberger, I., Hofer, A., Scholz, S.: Risk, Vulnerability and Resilience Analyses, for
Pandemic Management. In Proceedings of the 29th Interdisciplinary Information Management Talks, Kutná Hora,
Czech Republic, 1 – 3 September 2021
RKI, (2021) Neuartiges Coronavirus. [online]. Available at:
<https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Steckbrief.html> [Accessed 4 April 2020].
VISUAL CAPITALIST, (2020) Visualizing the History of Pandemics. [online] Available at: <
https://www.visualcapitalist.com/history-of-pandemics-deadliest/> [Accessed 29 May 2020].
Widera, A., Fonio, C., Lechtenberg, S., Hellingrath, B. (2018) Measuring Innovations in Crisis Management.Proceedings
of the 15th International Conference on Information Systems for Crisis Response and Management.
