ArticlePDF Available

Abstract and Figures

Understanding the current situation is critical in every natural disaster or crisis. Therefore, there is a need for accurate and up-to-date information about the scope, extent and impact of a disaster. The basis for this information is data that is available through a variety of sensors. Decision Support Systems (DSSs) support decision makers in disaster management, response, and recovery by providing early warnings, insights into the current situation and recommendations for mitigation actions. For this purpose, raw sensor data needs to be collected, analyzed, integrated, and its semantics need to be automatically understood by the system. This series of processes forms a generic sensor to decision chain. In this paper, we present solutions and technologies to integrate those steps seamlessly, also demonstrating how each step of the pipeline can be visualized.
Content may be subject to copyright.
DOI: 10.4018/IJISCRAM.2018100104

Volume 10 • Issue 4 • October-December 2018
This article published as an Open Access article distributed under the terms of the Creative Commons Attribution License
(http://creativecommons.org/licenses/by/4.0/) which permits unrestricted use, distribution, and production in any medium,
provided the author of the original work and original publication source are properly credited.
65

Understanding the current situation is critical in every natural disaster or crisis. Therefore, there is a
need for accurate and up-to-date information about the scope, extent and impact of a disaster. The basis
for this information is data that is available through a variety of sensors. Decision Support Systems
(DSSs) support decision makers in disaster management, response, and recovery by providing early
warnings, insights into the current situation and recommendations for mitigation actions. For this
purpose, raw sensor data needs to be collected, analyzed, integrated, and its semantics need to be
automatically understood by the system. This series of processes forms a generic sensor to decision
chain. In this paper, we present solutions and technologies to integrate those steps seamlessly, also
demonstrating how each step of the pipeline can be visualized.

Crisis Management, Data Aggregation, Data Integration, Decision Support, Early Warning, Internet of Things,
Ontologies, Reasoning, Semantic Modelling, Sensors, Visualization



Philipp Hertweck, Fraunhofer IOSB, Institute of Optronics, System Technologies and Image Exploitation, Karlsruhe, Germany
Jürgen Moßgraber, Fraunhofer IOSB, Institute of Optronics, System Technologies and Image Exploitation, Karlsruhe, Germany
Efstratios Kontopoulos, Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece
https://orcid.org/0000-0002-4435-2521
Panagiotis Mitzias, Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece
Tobias Hellmund, Fraunhofer IOSB, Institute of Optronics, System Technologies and Image Exploitation, Karlsruhe, Germany
Anastasios Karakostas, Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece
Désirée Hilbring, Fraunhofer IOSB, Institute of Optronics, System Technologies and Image Exploitation, Karlsruhe, Germany
Hylke van der Schaaf, Fraunhofer IOSB, Institute of Optronics, System Technologies and Image Exploitation, Karlsruhe, Germany
Stefanos Vrochidis, Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece
Jan-Wilhem Blume, Fraunhofer IOSB, Institute of Optronics, System Technologies and Image Exploitation, Karlsruhe, Germany
Ioannis Kompatsiaris, Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

Volume 10 • Issue 4 • October-December 2018
66

The World Meteorological Organization expects a global temperature increase of 3°C caused by climate
change (World Meteorological Organization, 2018). This increases the probability for the occurrence
of critical natural events, such as floods, dry periods (resulting in forest fires) or heatwaves. To face
these challenges, the United Nations have called to intensify the development of early warning systems
(UNISDR, 2005). In 2015, the call was renewed, now also including chained disasters (UNISDR,
2015). Through the support of early warning and risk management systems, authorities aim to limit
the impact of natural disaster crises.
In this paper, we propose a generic approach to improve the quality of decision support and foster
early warning systems. Our method, called the “sensor to decision chain,” covers the steps from sensor
data acquisition1, semantic data analysis, data integration and eventual decision support. The chain
forms the basis of an integration framework supported by a variety of cutting-edge technologies.
Therefore, the main goal of the framework is to support authorities through a decision support system
that implements the sensor to decision chain.
The following subsections describe each step of the sensor to decision chain and present
appropriate technologies for its implementation. The rest of this paper is structured as follows:
Section “Background” discusses existing decision support workflows and respective implementations;
“Motivation” presents the beAWARE project, which serves as a practical example of our
implementation and offers the possibility to test our approach in three large-scale pilots; the general
approach is discussed in “The sensor to decision chain” section, followed by an thorough description of
each step. Finally, “Conclusion” summarizes our findings and discusses directions for further research.

Since the 1980s, computer systems provide decision support to human actors in complex situations.
A decision support system (DSS) is an interactive computer-based system, which supports decision
makers in solving unstructured problems by using data and models (Sprague, 1980). In their recent
survey on current decision support systems for natural hazard risk reduction, Newman et al. (2017)
assessed the capabilities and drawbacks of DSSs with the help of a classification system and found
that a key shortcoming of current approaches is the limitation to single hazard situations. Based on
our “sensor to decision chain,” we propose to overcome this limitation through a flexible framework
of sensors and semantic components, allowing the integration of various data sources. This exactly is
the main contribution of this work in comparison to other related works in the literature. We present
below a subset of representative examples from existing approaches.
Fang et al. (2014) propose a DSS for environmental monitoring that integrates various
technologies, like Internet of Things (IoT), Cloud Computing and Geographic Information Systems
(GIS). The authors pointed out the importance of data acquisition and data fusion and proposed a
layered architecture, which we also adopted to some degree in our sensor to decision chain. Similarly,
di Pietro et al. (2017) discuss a DSS for crisis management based on an architecture structured along
functional blocks, covering different aspects in crisis management like monitoring natural phenomena
or predicting damage scenarios.
However, neither of the aforementioned approaches considers adding a semantic integration
layer for integrating the various data sources. Nevertheless, semantic technologies play a vital role
in modern DSSs and their deployment has indeed been discussed in recent works. Indicatively,
Wanner et al. (2014) present an ontology-structured knowledge base, which helps deduce information
relevant to the specific user that is communicated in the language of their preference. Moßgraber et
al. (2015) demonstrate another usage of semantic technologies, where an ontology is used to improve
the understanding of the use case domain, as well as to structure and visualize information of the
current situation. Finally, Burel et al. (2017) encapsulated a layer of semantics into a deep learning
model for automatically classifying information from social media posts.

Volume 10 • Issue 4 • October-December 2018
67
The deployment of semantics is also adopted in early warning systems, which constitute a
specific type of DSS. Moßgraber (2017) provided an overview of current architectural designs and
technologies for such systems. The operation of such DSSs is complex and challenging. Therefore, in
order to facilitate the development of new DSSs, Moßgraber developed, based on the aforementioned
overview, a framework for the architecture of next generation early warning systems. This framework
includes semantic technologies and workflow automation. Moreover, Poslad et al. (2015) presented
an Internet of Things (IoT) early warning system for environmental crisis management, where the
deployment of semantics facilitated sensor and data source plug-and-play, offering simpler, richer,
and more dynamic metadata-driven data analysis and easier service interoperability and orchestration.
Finally, research and development has been invested in crowdsourcing as well. For instance,
the Finnish Meteorological Institute (FMI) offers a way to report weather measurements through
a mobile application. Furthermore, the Ushahidi platform was developed as a response to riots in
Kenya after the 2007 elections, in order to report and document such incidents (Okolloh, 2009).
Since then, the system has been extended to be applicable to other events as well. A student project
from the University of Bremen developed a mobile application that collects reports during a crisis
and offers means of communication with the involved people (Frommberger, 2013). Finally, the
i-REACT project (http://www.i-react.eu/) is going one-step further by integrating messages from
citizens as well as first responders into a decision support system. The focus of all those projects is
the collection of reports from citizens. Yet, integration into a bigger system, offering sophisticated
analysis capabilities, is not foreseen.

This section outlines the need for an integrated platform for data acquisition, analysis, evaluation,
and visualization for DSSs with the help of project beAWARE (Enhancing decision support and
management services in extreme weather climate events – http://beaware-project.eu/). The main goal
of the beAWARE project is to provide an integrated decision support solution, covering all phases
of an extreme weather event. Next to situational awareness, command and control aspects were
considered as well. To provide sophisticated situational awareness capabilities, all phases need to
be considered, from forecasting and early warning before the crisis, as well as informing authorities
together with workforce management while the event is taking place.
During the pre-warning phase, when a critical situation is predicted before it comes in effect, the
extent of the disaster should be estimated through forecasts and with the help of available knowledge.
This information can be used to dispatch early warnings, allocate first responder forces and prepare for
the event, in order to reduce its impact as much as possible. Once the disaster occurs, it is important
to get accurate information about scope, geographical distribution, affected people and assets as
quickly as possible. In a natural disaster event, it is important to know what happened in previous
(comparable) events, what is happening in this moment and what can happen in the context of the
event. Situational awareness requires the collection of available information in (near) real-time, as well
as background knowledge and experiences of past events. This information supports the command
and control of the available workforce and other resources to mitigate the effect of the critical event.
The sensor to decision chain provides a generic approach to facilitate decision support and early
warning by drawing a picture of the ongoing situation through available sensors and their raw readings.
Key challenges include the collection of data from heterogeneous sources (such as environmental
data, social media, first responders and people in danger), data analysis and integration, as well as
deducing and extracting important information and presenting it to the responsible persons.

To ensure the usability of the development, the work relies on the use cases and the feedback of the
end-users in the beAWARE project. This ensures realistic scenarios of extreme weather events (flood,
fire and heatwave) and heterogeneous data availability. The sensors collecting these data are included

Volume 10 • Issue 4 • October-December 2018
68
in the first step of the sensor to decision chain. Later in the paper, it is argued that the chosen approach
is not limited to these scenarios and sensors, but can easily be extended to other events as well. The
following subsections introduce the use cases and end-users.
Flood
The flood use case is located in Vicenza, a city in northern east Italy, crossed by the Bacchiglone
River. In this area, the Italian Alto Adriatico Water Authority (AWAA) has deployed nearly 50
weather stations, measuring air humidity, pressure and temperature, precipitation and wind, as
well as water-level sensors in the river. These sensors are all connected to the Internet and this
allows automatically importing the latest measurements into the system. These observations
are extremely valuable since they provide reliable real-time information about the situation.
In comparison to other extreme weather events, a flood is easier to predict, as precipitation in
the river basin can result in a higher water level in the lower parts of the river. To substantiate
this prediction, the Finnish Meteorological Institute (FMI) provides the necessary weather
data; especially precipitation forecasts are considered. Additionally, through expert knowledge,
a prediction model for the water level in the river has been developed. The exceeding of
thresholds in these forecasts is the main indicator for an upcoming event, which results in
issuing early warnings.
Fire
Similar to the flood use case, the current and predicted weather data, which is periodically
updated, are important indicators for fire hazards. A high temperature combined with a low
humidity and little precipitation increases the risk of fire. These conditions do not necessarily
lead to a critical event, but should draw increased attention. In this case, the fire brigade is
set on standby. There are several possibilities to detect a fire. The most efficient and reliable
way is to constantly record the area of interest by the means of static cameras and analyzing
the data (near real-time) by applying video analysis software. Nevertheless, static cameras and
sensors in general are typically expensive in acquisition and operation. Therefore, it is usually
not possible to monitor the whole area at risk and other data sources need to be considered.
The pilot region for the beAWARE use case is a forest area near Valencia in Spain, where, like
in most rural areas, unfortunately no static cameras are deployed. To overcome this, the pilot
is supported by the usage of drones, which are capable of monitoring a larger area. By flying
over the region of interest, pictures and videos are recorded and analyzed to detect critical
events, especially starting fires.
Heatwave
In this use case, the weather situation and forecast are factors as well. In contrast to the fire use case,
where a low humidity increases the risk of fire, high humidity increases the severity of a heatwave. To
mitigate the impact of a heatwave, it is common to offer citizens the possibility to visit public shelters
that are cooled down by air conditioning. For the authorities, the condition and status of these places
(e.g. available space, problems with sanitary facilities etc.) is of interest. Collecting this data with
technical sensors can be very challenging. Therefore, the people themselves can be considered as an
additional data source: by analyzing data from social media (e.g. Twitter) or by using a dedicated
mobile application from which people can send multimodal reports (text, audio, images, videos)
directly to authorities, one can harvest further helpful information. Citizen involvement is not limited
to this use case and can help in other scenarios as well.
The above three use cases describe the variety of sensor data and data sources in the project.
By overcoming the challenges of heterogeneity, one can offer a detailed description of an ongoing
disaster. Furthermore, the importance of each sensor type highly depends on the specific use case.

Volume 10 • Issue 4 • October-December 2018
69

As already stated, there is a broad variety of sensors available today. Sensors are accessible via
Internet of Things (IoT), Machine-2-Machine (M2M) standards, from social media or from a mobile
application (human sensors) (Meissen, 2014). To make use of the available data, it needs to be collected
and analyzed. This is a prerequisite for gaining a better understanding of a crisis, for finding critical
events like people in need of rescue or for coordinating countermeasures.
To approach this problem Moßgraber et al. (2018) presented a generic approach for decision
support and early warning in climate-related crises by applying a pipeline to get from raw sensor data
to a model about the ongoing situation (see Figure 1). The key contribution of the paper at hand is to
provide tools, models and technologies to execute and visualize this pipeline. The parts of this method
cover the steps from sensor data acquisition, data integration and aggregation, semantic modeling and
data analysis to provide decision support and early warning capabilities. The challenge of integrating
a variety of heterogeneous sensor data to understand the current situation often occurs in the context
of crisis management. The presented approach offers the needed flexibility to be applicable for a
broad variety of situations. We show this by applying it to three different large-scale use cases within
the beAWARE project.
As a first step, the data needs to be collected from the sensors. It then needs to be aggregated,
fused and analyzed to understand the semantics. A knowledge base (KB), formalized as an ontology,
provides a global schema to support the semantic integration from heterogeneous sources (Noy, 2004).
The ontology describes the concepts of interest and their interrelationships, covering various domains
like sensor metadata, climate change and crisis management. Once the schema is established, it can
be populated with results from various analysis components processing the raw sensor data. Since the
KB knows not only the plain data but also its underlying semantics, a DSS can recommend actions
or provide information about the situation.
In the following sections, each individual step is presented in detail. After a generic description,
applicable techniques and technologies are presented. Then we will show how the aforementioned
use cases can be implemented with our approach, including visualizations adjusted to specific
stakeholders, who interact with the sensor to decision chain.

To allow well-grounded decision support and early warning, reliable up-to-date information is
crucial. Usually it is not possible for decision makers to visit the affected area of the crisis in-
situ, and it is not possible to overview all relevant aspects without technical support. Therefore,
the situation needs to be captured by sensors, to be considered in the decision-making process.
In the use case section, necessary information and their possible sources have been presented.
This section will present three generic types of sensors, their characteristics and the existing
challenges. These categories are namely static sensors in the context of IoT, social media, and
mobile app technology. Depending on the use case, available resources and infrastructure, the
appropriate sensors to be used need to be selected.
Figure 1. The sensor to decision chain

Volume 10 • Issue 4 • October-December 2018
70

Spatially distributed static sensors can observe a specific area. There are various types of sensors
like weather stations, river level sensors as well as video cameras, offering reliable information
about the physical phenomena in the covered region. However, they are expensive in installation and
operation. Additionally, they usually have a fixed position, which limits the coverage to the deployed
area. Still, the upcoming Internet of Things (IoT) technologies lead to a growing market of new and
cheap sensors. These devices can be used to deploy additional sensors in the observed area. Despite
their lower measurement accuracy when compared to traditional sensors, the correlations of their
outputs can lead to valuable insights.

Another important data source in a crisis is the public itself. Social media services like Twitter
offer new possibilities to retrieve information (e.g. (Terpstra et al., 2012)). A lot of research has
been focused on collecting and analyzing social media messages. While collecting this data is
comparatively easy, the automated analysis can be very challenging. The content of the message
needs to be understood by applying text analysis. Due to the ambiguity of natural language, this
process is rather complex. Even more challenging is the individual textual style applied in social
media, e.g. the use of abbreviations, hashtags or emojis, which prevents the application of traditional
text analysis and content extraction mechanisms.
Besides textual analysis of individual messages, research has been also conducted on clustering
and classifying similar messages determining ongoing events (Angaramo & Rossi, 2018). Furthermore,
researchers try to derive the sentiment of the message (Schulz et al., 2013) to infer the emotional
state of the author.
To utilize this information, the authors and their electronic devices can be seen as a sensor
in the sensor to decision chain. It allows insights into incidents, thoughts and feelings that are not
directly communicated to the authorities, but are openly available and can be very helpful to get a
better picture of the ongoing situation. Still, social media data often has no geo information attached.
Thus, if the position of a message is missing (which is the common case) it can only be inferred
by applying text analysis. These methods are not reliable through well-known challenges in natural
language processing techniques. Additionally, an exact geographic location needs to be mentioned
in the text (e.g. the name of a well-known building), which makes assigning social media data to a
concrete position rather challenging.

Another communication possibility for the public is a mobile application for user-optimized provision
of information. This is a more direct way in comparison to social media: on social media, the recipients
of the messages are not clear, and people might not be aware that their published information is useful
for decision-making. Thus, we are currently working on incorporating data from a dedicated mobile
application into the sensor to decision chain (see Figure 2).
The variety of sensors embedded in modern smartphones can provide precise information. People
can choose the appropriate input modality depending on the current situation and their personal
preferences. This can be done by allowing text input as well as utilizing the smartphone camera
to take pictures/videos or the microphone to record audio. The real-world context can be added by
utilizing the GPS data of the mobile device.

The outcome of the first step of the sensor to decision pipeline is raw data. Integrating this data is
difficult, because of the multiple ways accessing this data and the different formats in which the

Volume 10 • Issue 4 • October-December 2018
71
measurements and metadata are available. To allow decision support and early warning, the data
needs to be harmonized and aggregated in the next step. Just as the sensor selection is dependent
on the use cases, the integration step is dependent on the selected sensors. This section will present
methods to integrate different types of sensors.
Figure 2. Sending reports from a mobile application

Volume 10 • Issue 4 • October-December 2018
72

The most prominent method to integrate data from various sources is to rely on standards. One
example is the OGC SensorThings API (Liang et al., 2016), which offers the possibility to manage
time series based sensor data as well as sensor metadata. Besides an underlying data model, an
API has been specified, based on the representation state transfer (REST) paradigm, which is very
common in the context of web applications (Fielding & Richard, 2002). The REST interface allows
easy access from different applications and programming languages. The SensorThings API integrates
selection and filter operators from the OASIS Open Data Protocol (Pizzo et al., 2014), which offers
the possibility to query data.
Next to this REST-based API, which allows CRUD (create, read, update, delete) operations, a
Message Queue Telemetry Transport (MQTT) extension was specified. This allows the notification
of new and changed entities through a listener/subscriber pattern. Due to these characteristics,
the SensorThings API standard offers an appropriate solution to integrate time series data (like
sensor measurements) into a decision support system. In the presented use case scenarios, we
use the Fraunhofer IOSB open source implementation of the OGC SensorThings API to integrate
measurements from weather stations, river gauges or weather models. To this moment, both current
measurements as well as forecasts are stored on the FROST server (van der Schaaf, 2016).
To get an understanding of the current situation and the provided results in the decision support
step, transparency of each chain link is crucial. To do so, the data provided by the sensors can be
visualized in an interactive graph. Even though this data can be analyzed automatically, it is important
to offer a possibility to get a detailed view of single measurements. Experts might have to validate
recommendations of the decision support module. In addition, correlations between two measured
values can be discovered through visual inspection. Figure 3 shows an example for the correlation
between precipitation and water level.
At this point, we need to keep in mind that the data volume might exceed the volume that can
be handled by subsequent modules. A data aggregation step reduces the data amount for further
processing and displaying. Examples for aggregation functions are the calculation of average, minimum
or maximum values. Another possibility is to monitor the measurements and pass single events to
the following steps e.g. the exceeding of a threshold.

In the background section, we pointed out that there are various mobile applications allowing
citizens to provide data to authorities in a crowd-sourcing manner and we pointed out that all of these
approaches lack a deep integration into a DSS. To facilitate this, we propose to regard data coming
from mobile applications as additional sensor data running through our sensor to decision chain.
By this approach, we achieve a deep integration through applying analysis components to this data.
Semantically integrating these results allows an automated utilization for decision support as well.
The semantic model section demonstrates how this aspect can be integrated in the overall model.

Due to the variety and amount of available sensor data, a tight integration is only possible by applying
advanced analysis methods. Especially visual content (images, videos) provided by static cameras,
citizens or via social media is only helpful when useful information can be extracted in a machine
interpretable format. This can be achieved through image and video analysis tools. Audio content
firstly needs to be transcribed to written text in order to extract information by applying text analysis
and Natural Language Processing (NLP) methodologies.
Certain analysis processes tend to be demanding in processing power and time, vastly depending
on the incoming media characteristics (e.g. length, resolution of pictures or videos) and the system’s
hardware setup. As a result, a large volume of incoming resources can prevent a real-time approach.

Volume 10 • Issue 4 • October-December 2018
73
Thus, from a system architecture point of view, the various analysis components operate independently
and provide results asynchronously. For instance, a citizen’s submission of data via the Mobile App
might contain textual and visual attachments. This results in the creation of an incident report in
the main pipeline, which later on will be enriched with findings from the text and image analysis
components. Therefore, all analysis components need to monitor all reported data and decide about
which data artefact they can provide information.
For the implementation of the beAWARE platform, a standalone web application called beAWARE
Bus Logger (see Figure 4) has been developed to track, log and visualize these asynchronous
communications between sensors, analysis components, the semantic model (see next section) and
other system modules. This greatly facilitates the establishment and debugging of communication
protocols, allowing visual interpretation of the system and providing advanced testing capabilities.
However, the Logger is an excellent tool for decision makers to visualize the progress of information
exchange during an ongoing crisis.

This section describes the semantic model of our pipeline, which is formalized as an ontology (Gruber,
1993), offering a unified representation of all relevant domain-specific information in a formal way.
The ontology is a lightweight model for crisis management in the context of climate-related natural
disasters and plays a two-fold role. First, it serves as common uniform model for semantically
integrating heterogeneous information from the diverse sources and sensors. Ontologies are an
Figure 3. Visualization of precipitation and water level

Volume 10 • Issue 4 • October-December 2018
74
excellent fit for addressing this issue of semantic heterogeneity, and for establishing interoperability
through a process called “semantic integration” or, less frequently, “semantic fusion” (Wache et al.,
2001). The second role played by the semantic model is serving as the backbone of the decision
support system deployed within the context of the beAWARE project (see later subsection).
Contrary to other existing ontologies for crisis management that only focus on specific aspects
of the occurring natural disasters, the beAWARE ontology is an all-around model integrating all
pertinent aspects, including associated conditions and climate parameters, results of the analyzed
data (e.g. text, audio, images, videos), as well as workforce management. Nevertheless, parts of the
ontology are inspired from existing models for representing similar notions. MOAC (Management
of a Crisis) (Ortmann et al., 2011) constituted the basis of our representation for disaster impacts,
SoKNOS (Service-Oriented Architectures Supporting Networks of Public Security) (Babitski et al.,
2011) was adopted to categorize damages and resources, and the PESCaDO ontologies (Rospocher
& Serafini, 2012) were extended to represent environmental and meteorological conditions.
The key notions of the beAWARE ontology are presented in (Kontopoulos et al., 2018), while
the ontology itself is publicly available (beAWARE, 2018a). Figure 5 displays an overview of the
main concepts and their relationships, based on the Grafoo notation for ontology visualization (Falco
et al., 2014).
In order to assist the unfamiliarized users and to encourage further ontology reuse, the beAWARE
model embodies extensive definitions and representative examples, via the use of SKOS properties
skos:definition and skos:example, respectively (Miles & Bechhofer, 2009).
The following subsections briefly present the various representational aspects of the ontology.

To understand the crisis and to provide decision support, a basic understanding of the underlying
phenomena is needed. The involved concepts are visualized in Figure 6. In our presented case of
climate-related crises, a Natural Disaster is the main concept. Figure 7 shows a possible instantiation
for the UK heatwave that occurred in between the 17th and 22nd of June 2017 (BBC, 2017). This
particular disaster is characterized by a Natural Disaster Type, in this case “heatwave”. By using this
modelling approach, other types of natural disasters like floods, forest fires, storms or earthquakes can
be categorized as well. Disasters can interfere among one another (modeled by the leads to relation):
e.g., a heatwave may lead to forest fires or a storm may lead to floods. Each Natural Disaster Type
is characterized by measurable parameters, the types of which are modeled as Climate Parameter
Figure 4. The beAWARE Bus Logger interface

Volume 10 • Issue 4 • October-December 2018
75
Types. Their concrete manifestations are instances of a Climate Parameter, which are described by
a measured value in a specific unit at a specific time. Since those measurements are conducted at a
specific location, the Climate Parameter concept has a relation to the Location concept. Furthermore,
to describe the effects of natural disasters, the Impact concept is introduced (analogous split into
Impact Type and Impact). To limit the extent of an impact, counteractions might be needed; we refer
to such problems that can be tackled by an action as Incident. In the next sections, we can see that
the Incident is relevant in all the three domains of our ontology.
Figure 5. Main concepts of the beAWARE ontology
Figure 6. Model of natural disasters together with impacts and risks
Figure 7. Instantiation of a concrete natural disaster

Volume 10 • Issue 4 • October-December 2018
76

Information about the ongoing situation is fed into the system by sensors and it is automatically
analyzed by the appropriate analysis components. These results are semantically integrated into the
beAWARE ontology, as shown in Figure 8. The raw data is represented via Media Item. An instance
of a Task analyzes a Media Item and produces a Dataset. Analyzing tasks might be the processing of
a video or image analysis component, where the Dataset contains the relevant output, which is mainly
a set of Detections. Each Detection is described by a confidence degree, denoting the probability
of the detected element by the analysis component. Objects involved in an Incident are Vulnerable
Objects (e.g. assets, persons, infrastructure, buildings, etc.). They can be distinguished by extending
the Vulnerable Objects concept by corresponding sub-concepts, which are not displayed. Additional
properties, like detection risk or severity level can be added to the detection. The relation of a media
item to an incident can be given directly or it can contain a location, from where the incident can be
inferred. The relations from Datasets and Detection to Incident can be inferred from the Media Item,
and can be inserted manually or can be asserted as a result from the analysis.

Managing a crisis is only possible through the coordinated use of available forces. Therefore, first
responder assignments are also part of our semantic model, as shown in Figure 9. A first Responder
might be assigned to a Mission, which is characterized by several properties like status or priority. A
mission is related to an Incident and therefore mitigates an Impact. To get an overview of locations
of the available forces, the current location of a first responder is modelled as well.
Figure 8. Representing sensors and analysis
Figure 9. Rescue teams and assignments

Volume 10 • Issue 4 • October-December 2018
77

An ontology covering several domains and topics can be extremely complex and sophisticated.
Therefore, visualization plays an important role in (a) helping end users to understand the model’s
inherent structure, enabling them to work efficiently with the ontology, and, (b) giving a more thorough
overview of an ongoing crisis and its potential impacts, thus, improving the quality of decision support.
Besides the visualization of the structure, the contained data (e.g. specific instances, measurements
and incidents) needs to be displayed as well. Therefore, our implementation of the sensor to decision
chain also contains a module to visualize the ontology (structure and instances), offering an additional
tool that supports decision makers by providing a clearer picture of the underlying model, data and
situation. In this context, we implemented an interactive visualization that allows browsing and
analyzing the complete semantic content.
The fully automated generation of graphs of an ontology is difficult, since the importance of
individual parts is highly dependent on the use case. While displaying all concepts and their relations
might lead to an unmanageable amount of information, we decided not to adopt an automated
generation approach. A full manual solution, on the other hand, would involve the use of an external
tool to generate a visualization and uploading the result. This is very time-consuming and error-prone,
especially when the underlying ontology is changing. Hence, we are implementing an integrated
solution. By following this approach, there is the guarantee that the visualization matches the currently
used ontology and therefore avoids inconsistencies, which may prove too confusing for users. Thus, a
tool was integrated into the DSS to allow end users to compose images of the ontology, including both
concepts as well as instances. When creating an image, the current concept or instance is automatically
added. Related concepts and images can be added for each item in the picture, by selecting them
from the recommended list, which is automatically populated with all related elements existing in
the underlying ontology. This allows the automatic naming of the relations (drawn by arrows) and
ensures that the picture is aligned with the ontology. This accordance is verified every time the image
is shown. Entities that no longer exist are removed automatically. Applying relations dynamically
to the image (see Figure 10) ensures that relations added to the ontology at a later point in time are
automatically added to the image, without further interaction of the user. Depending on the use case,
different relationships are of interest each time. It is possible to attach multiple ontology images to
a concept or instance. This allows the visualization of different aspects.
Figure 11 shows the visualization of an instance. The instance itself is described through text
and image. On the right side, the concept of that instance together with all relations is shown. At
the bottom, there is the ontology visualization, showing the relationship of that instance with other
concepts. Since these concepts and relationships on the right-hand side are linked to the target entity,
the ontology can be easily browsed and analyzed.

The last part of the sensor to decision chain is decision support and early warning. All previous
modules work towards this step by collecting, analyzing, integrating and modeling the input data.
Decision support and early warning aims for the generation of an accurate model of the current crisis
that contains past information (experiences of previous events), current real-time information as well
as predictions and forecasts. The main goal is to capitalize on this model for supporting end users in
decision making for crisis response.
One aspect of decision support and early warning is the monitoring of available data to detect
critical situations in an early stage. This can be achieved by continuously evaluating predefined
metrics, which can directly be derived from sensor values (e.g. current water level) or from more
complex relations (e.g. used capacity of rescue forces). In addition, they can refer to a single point
(e.g. current water level at specific latitude/longitude coordinates) or a region (e.g. rescue forces in
this area). Next to metrics, which can be grouped geographically, the data model can be observed

Volume 10 • Issue 4 • October-December 2018
78
regarding critical events that require an instant action by a rescue team (e.g. a static camera captures
an incident with injured persons).
Decision support and early warning are very use-case specific. Therefore, in this subsection we
demonstrate only a proposed generic approach based on a flexible query mechanism for providing
authorities and human operators with decision support and early warning capabilities. The concrete
implementation each time needs to be materialized when applying the sensor to decision chain
methodology to specific use cases.
Therefore, starting with the risk assessment phase (i.e. before a disaster actually occurs), the
Crisis Classification (CRCL) system, which is the main component responsible for this task fuses
and analyses information acquired from heterogeneous data sources, in order to support authorities
and local stakeholders during the risk assessment as well as during the decision-making process. To
achieve this, the system has been equipped with functionalities and capabilities to collect multiple
types of data and information related with the crisis during the emergency phase. Specifically, sensing
data from weather stations, as well as aggregated data from other components, are available to CRCL
Figure 10. Creating a visualization of the ontology

Volume 10 • Issue 4 • October-December 2018
79
for assessing the risk and for classifying the impending crisis. Thus, a proposed holistic multimodal
fusion approach considers the analysis results from multimedia analysis, including image, video and
audio analysis, multilingual text analysis, mobile applications for citizens and first responders as
well as social media.
Entering the phase when the disaster is ongoing, in order to retrieve information from the semantic
model, integrating all the relevant knowledge, SPARQL can be used to query the information and
derive inferences. SPARQL is a set of specifications for querying and manipulating ontology models,
standardized by the W3C (W3C, 2012). The expressive power of SPARQL allows not only the retrieval
of explicitly asserted data, but also inferring new information via calculations or semantic reasoning.
For example, incidents can be spatially grouped, so that events happening very close to each other are
visible as a single event. Another example is the automated rating of events based on their severity
(see Figure 12). This can be done by utilizing the analysis results, especially when people are involved.
In addition, it is possible to dynamically calculate an incident’s certainty, severity and potential
impact, based on the available information. For instance, Figure 13 displays a query for retrieving
fire or flood incidents involving at least one human, which can then be assigned with a high priority.
The inference results can optionally be appended to the ontology in order to retrieve them in
further queries. In knowledge engineering, queries which can be answered by the use of an ontology
are typically called competency questions (CQs). In the context of our use case, a list of CQs was
created, which assisted in formulating a list of initial requirements for the decision-making process.
Each CQ was formalized as a SPARQL query to be answered by the semantic model. The full list of
CQs, together with the corresponding queries, can be found online (beAWARE, 2018b) and some
indicative sample queries are:
Figure 11. Visualization of an instance

Volume 10 • Issue 4 • October-December 2018
80
What are the locations affected by a natural disaster?
What are the impacts caused by a natural disaster?
What are the vulnerable objects that suffer the greatest risks?
Which rescue unit is assigned the most severe incident?
Those CQs need to be executed and evaluated and the execution of the respective SPARQL
queries must be explicitly triggered. This can be done at periodic intervals (e.g. every minute) or
on explicit request by the end-user (e.g. because the user is currently analyzing a given situation).
To reduce unnecessary evaluation that would consequently increase the response time, it
is also possible to estimate the importance of data that is added to the ontology. Based on this,
trigger rules for the CQs can be created. This possibility has not been evaluated yet but will be
part of our future research.
As already mentioned, our sensor to decision chain will be evaluated in three pilot use cases,
based on the outcomes of which, decision support and early warning capabilities will be extended in
the future. The use of SPARQL query capabilities allow an easy integration of further CQs.

At the point when writing this article, two out of three use-case tests have been executed successfully.
Each pilot was split into two sections executing the same scenario: one time using legacy tools and
one time using the beAWARE platform. In this platform, the sensor to decision chain, described in
this article together with the presented technologies was deployed. The double execution of the pilot
enabled a direct comparison of crisis management between the currently used tools and the support
of the beAWARE platform.
In November 2018, a heat wave was simulated in Thessaloniki, Greece. Due to high temperatures,
the authorities decided to warn the citizens and they provided recommended actions, which included
visiting air-conditioned places. The main task of the authorities was to monitor the capacity of places
of relief (public buildings with air condition), as well as the traffic situation, in order to facilitate the
Figure 12. SPARQL query that retrieves all high severity incidents
Figure 13. SPARQL query that retrieves incidents involving at least one human

Volume 10 • Issue 4 • October-December 2018
81
way to those places. The situation on the street as well as in the buildings was reported by citizens
either through the mobile application or through Twitter. It has been shown that using the integrated
and processed data increased the overview of the authorities to manage the situation in comparison
to the legacy tools, like e-mail, phone or radio. A detailed evaluation of the pilot can be found in the
publically available deliverable (Lombardo et al., 2018).
The second use-case test took place in Vicenza, Italy. A flooding of the city center was simulated:
due to heavy precipitation, the water level of the Bacchiglione River increased. Since the forecasts
of the water level prediction model exceeded the normal thresholds, the emergency protocol was
activated by the authorities, which triggered precautionary measure in the city. These tasks were
arranged, organized and monitored with support of the beAWARE platform. Weather forecasts, water
level measurements and predictions together with reports coming from the mobile application and
Twitter were integrated and shown to the authorities to help them better understand the situation.
A full description of the steps executed during the pilot can be found in the publically available
deliverable (Muhic et al., 2019). A detailed evaluation is currently ongoing and can later be found
on the beAWARE project website. The pilot execution and debriefing session afterwards showed
that the situational awareness for the responsible decision maker was higher than when using legacy
tools alone.
The successful execution of the two pilots (a third pilot is underway) proved that the methodology
described with the sensor to decision chain is fully capable to be applied in crises and it helps to
increase the situational awareness of the decision maker.

This paper presented the sensor to decision chain, which is being applied to three large-scale use-case
tests within the beAWARE EU-funded project. A methodology was presented utilizing various sensors
and data sources to implement sophisticated decision support and early warning capabilities. Different
sensor categories, their integration and analysis capabilities were discussed. A semantic model was
shown to allow a common understanding and integration of various data sources. Based on this model,
reasoning algorithms are applied to support decision support and early warning. The practicability
was proven by the successful execution of two pilot use cases. The preparations of the pilot use cases
have shown that all end user requirements can be realized by the presented methodology. It turned
out that the semantic model can satisfy its role as the central integration point. On the one hand,
it was possible to integrate new use-case specific sensors and, on the other hand, use-case specific
decision support capabilities were implemented by formalizing the according SPARQL-queries.
Through applying the sensor to decision chain, analysis and integration techniques are available for
all integrated data. It turned out that the ontology covered all the needed aspects, with the exception
of slight extensions being adopted to fully support all aspects of the use cases. This, however, does
not limit the applicability of our approach. The sensor to decision chain describes a methodology
along with respective technologies to facilitate decision support and therefore it is not directly visible
by the decision maker. In any case, visualizing each step, making the chain transparent and decisions
comprehensible is a key challenge to increase the acceptance by all involved users.
In future work, it should be evaluated what kind of additional data can be integrated into the
sensor step. It should be examined if static data sources (like topographic information, building
development and points of interest) can be used to improve decision support capabilities. A first
attempt has already been conducted in including external sematic data sources. In addition, adoptions
to the semantic model might be considered to represent the new aspects. Further research needs to
be conducted on reasoning techniques applied to the semantic data in addition to the described query
mechanisms. This will allow more advanced decision support and early warning capabilities, e.g.
generating automated warnings or reports of the current situation. Finally, the practical evaluation
of the beAWARE DSS, which is based on the sensor to decision chain methodology, is still ongoing.

Volume 10 • Issue 4 • October-December 2018
82
First results are available and proved the applicability of our approach, while the final results will
soon be available.

The research leading to these results has received funding from the European Union’s Horizon 2020
Research and Innovation Programme, under Grant Agreement no 700475 “beAWARE: Enhancing
decision support and management services in extreme weather climate events”.

Volume 10 • Issue 4 • October-December 2018
83

W3C SPARQL Working Group. (2012) SPARQL 1.1 overview. W3C recommendation.
Angaramo, F., & Rossi, C. (2018) Online clustering and classification for real-time event detection in Twitter.
In Proceedings of the 15th ISCRAM Conference, Rochester, NY. Academic Press.
Babitski, G., Bergweiler, S., Grebner, O., Oberle, D., Paulheim, H., & Probst, F. (2011). SoKNOS–using semantic
technologies in disaster management software. In Proceedings of the Extended Semantic Web Conference (pp.
183-197). Springer. doi:10.1007/978-3-642-21064-8_13
bAWARE. (2018a) The beAWARE Crisis Management Ontology v1.0. Retrieved from http://w3id.org/beaware_
ontology
BBC. (2017) Hottest June day since summer of 1976 in heatwave. Retrieved from https://www.bbc.com/news/
uk-40353118
beAWARE. (2018b) Competency Questions. Retrieved from https://github.com/beAWARE-project/ontology/
blob/master/CQs.md
Burel, G., Saif, H., & Alani, H. (2017). Semantic wide and deep learning for detecting crisis-information categories
on social media. In Proceedings of the International Semantic Web Conference (pp. 138–155). Academic Press.
doi:10.1007/978-3-319-68288-4_9
di Pietro, A., Lavalle, L., La Porta, L., Pollino, M., Tofani, A., & Rosato, V. (2017). Design of DSS for Supporting
Preparedness to and Management of Anomalous Situations. In R. Setola, V. Rosato, E. Kyriakides, & E. Rome
(Eds.), Managing the Complexity of Critical Infrastructures, Studies in Systems, Decision and Control (pp.
195–232). Cham: Springer. doi:10.1007/978-3-319-51043-9_9
Falco, R., Gangemi, A., Peroni, S., Shotton, D., & Vitali, F. (2014, May). Modelling OWL ontologies
with Graffoo. In Proceedings of the European Semantic Web Conference (pp. 320-325). Cham:
Springer.
Fang, S., Xu, L. D., Zhu, Y., Ahati, J., Pei, H., Yan, J., & Liu, Z. (2014). An Integrated System for Regional
Environmental Monitoring and Management Based on the Internet of Things. IEEE Transactions on Industrial
Informatics, 10(2).
Fielding, R., & Richard, T. (2002). Principled design of the modern web architecture. ACM Transactions on
Internet Technology, 2(2), 115–150. doi:10.1145/514183.514185
Frommberger, L., & Schmid, F. (2013). Mobile4d: Crowdsourced disaster alerting and reporting. In Proceedings
of the Sixth International Conference on Information and Communications Technologies and Development:
Notes (Vol. 2, pp. 29-32). Academic Press.
Gruber, T. R. (1993). A Translation Approach to Portable Ontology Specification. Knowledge Acquisition, 5(2),
199–220. doi:10.1006/knac.1993.1008
Kontopoulos, E., Mitzias, P., Moßgraber, J., Hertweck, P., van der Schaaf, H., Hilbring, D., . . . Kompatsiaris, I.
(2018). Ontology-based Representation of Crisis Management Procedures for Climate Events. In Proceedings
of the 1st International Workshop on Intelligent Crisis Management Technologies for Climate Events (ICMT
2018), Rochester NY, May 20. Academic Press.
Liang, S., Huang, C., & Khalafbeigi, T. (2016) OGC SensorThings API Part 1: Sensing, Version 1.0. Retrieved
from http://docs.opengeospatial.org/is/15-078r6/15-078r6.html
Lombardo, F., Norbiato, D., Ferri, M., Vourvachis, I., Meliadis, M., Karakostas, A., . . . Koren, I.
(2018). D2.4 Evaluation report of the 1st proto-type, beAWARE H2020 project deliverable. Retrieved
from https://beaware-project.eu/wp-content/uploads/2019/01/D2.4_-beAWARE_Evaluation-report-
of-P1_v0.5.pdf
Meissen, U., & Fuchs-Kittowski, F. (2014). Towards a reference architecture of crowdsourcing integration in
early warning systems. In Proceedings of the 11th International ISCRAM Conference, University Park, PA.
Academic Press.

Volume 10 • Issue 4 • October-December 2018
84
Miles, A., & Bechhofer, S. (2009). SKOS simple knowledge organization system reference. W3C recommendation.
Moßgraber, J. (2017). Ein Rahmenwerk für die Architektur von Frühwarnsystemen (Vol. 29). Karlsruher Schriften
zur Anthropomatik.
Moßgraber, J., Hilbring, D., van der Schaaf, H., Hertweck, P., Kontopoulos, E., Mitzias, P., & Kompatsiaris, I.
etal. (2018) The sensor to decision chain in crisis management. In Proceedings of the 15th ISCRAM Conference,
Rochester, NY. Academic Press.
Moßgraber, J., Schenk, M., & Hilbring, D. (2015) Modelling of an Ontology for a Communication Platform. In
Proceedings of the Ninth International Conference on Advances in Semantic Processing, Nice, France.
Muhic, S., Meliadis, M., Vourvachis, I., Nerantzis, L., Spyroglou, O., Lintrup, K., . . . Beliver, J.
(2019) D2.5 Pilot Use Cases Setup for the Second Prototype, beAWARE H2020 project deliverable.
Retrieved from: https://beaware-project.eu/wp-content/uploads/2019/03/D2.5_beAWARE_
pilotusecasessetupforthe2prototype_v0.5.pdf
Newman, J., Maier, H., Riddell, G., Zecchin, Z., Daniell, J., Schaefer, A., & Newland, C. etal. (2017).
Review of literature on decision support systems for natural hazard risk reduction: Current status
and future research directions. Environmental Modelling & Software, 96, 378–409. doi:10.1016/j.
envsoft.2017.06.042
Noy, N. F. (2004). Semantic integration: A survey of ontology-based approaches. SIGMOD Record, 33(4),
65–70. doi:10.1145/1041410.1041421
Okolloh, O. (2009). Ushahidi, or ‘testimony’: Web 2.0 tools for crowdsourcing crisis information. Participatory
learning and action, 59(1), 65-70.
Ortmann, J., Limbu, M., Wang, D., & Kauppinen, T. (2011). Crowdsourcing Linked Open Data for Disaster
Management. In Proceedings of Terra Cognita, the 10th International Semantic Web Conference, Germany
(pp. 11-22). Academic Press.
Pizzo, M., Handl, R., & Zurmuehl, M. (2014). Odata version 4.0 part 1: Protocol: Oasis standard.
Poslad, S., Middleton, S. E., Chaves, F., Tao, R., Necmioglu, O., & Bügel, U. (2015). A semantic IoT early warning
system for natural environment crisis management. IEEE Transactions on Emerging Topics in Computing, 3(2),
246–257. doi:10.1109/TETC.2015.2432742
Rospocher, M., & Serafini, L. (2012). An ontological framework for decision support. In Proceedings of the
Joint International Semantic Technology Conference (pp. 239-254). Springer.
Schulz, A., Thanh, T. D., Paulheim, H., & Schweizer, I. (2013). A fine-grained sentiment analysis approach for
detecting crisis related microposts. In Proceedings of the 10th International Conference on Information Systems
for Crisis Response and Management. Academic Press.
Sprague, R. (1980). A Framework for the Development of Decision Support Systems. MIS Quarterly, 4(4).
Terpstra, T., De Vries, A., Stronkman, R., & Paradies, G. L. (2012). Towards a realtime Twitter analysis during
crises for operational crisis management. In Proceedings of the 9th International ISCRAM Conference, Vancouver,
Canada. Academic Press. doi:10.1201/b13715-221
UNISDR. (2005). United Nations Office for Disaster Risk Reduction (2005), Hyogo Framework for Action.
Geneva, Switzerland: HFA.
UNISDR. (2015). United Nations Office for Disaster Risk Reduction (2015). Geneva, Switzerland: Sendai
Framework for Disaster Risk Reduction.
Van der Schaaf, H. (2016). FROST-Server, Fraunhofer IOSB, Retrieved from https://github.com/FraunhoferIOSB/
FROST-Server
Wache, H., Voegele, T., Visser, U., Stuckenschmidt, H., Schuster, G., Neumann, H., & Hübner, S. (2001)
Ontology-Based Integration of Information-A Survey of Existing Approaches. In: OIS@ IJCAI.

Volume 10 • Issue 4 • October-December 2018
85
Wanner, L., Bosch, H., Bouayad-Agha, N., Casamayor, G., Ertl, T., Hilbring, D., & Vrochidis, S. etal. (2014).
Getting the environmental information across: From the Web to the user. Expert Systems: The Journal for
Knowledge Engineering, 32(3), 405–432. doi:10.1111/exsy.12100
World Meteorological Organisation. (2018). WMO Greenhouse Gas Bulletin: The State of Greenhouse Gases in
the Atmosphere Based on Global Observations through 2017. Retrieved from https://library.wmo.int/doc_num.
php?explnum_id=5455

1 In the context of this paper, a sensor is a device that delivers data about an ongoing crisis. This explicitly
contains sensors delivering unstructured information, such as pictures, recordings, videos or even data
from social media.

Volume 10 • Issue 4 • October-December 2018
86
Philipp Hertweck received his M.Sc. in Computer Science from the Karlsruhe Institute of Technology, Germany.
Now he is working at Fraunhofer IOSB in Karlsruhe as research associate. His research interests include the
design and implementation of distributed information systems.
Ing. Jürgen Moßgraber (M) holds a degree in Computer Science and received a PhD from the Karlsruhe
Institute of Technology, Germany. He is the head of the research group “Architecture and Information
Systems” at the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB.
His research interests include the design and implementation of web-based information systems with
state-of-the-art technologies. He has particular experience with the design of distributed systems handling
large-scale databases and modern architectures for task-oriented systems with a service- and event-
based approach.
Efstratios Kontopoulos received his PhD in Artificial Intelligence from the Aristotle University of
Thessaloniki in 2011. He also holds a BSc in Applied Mathematics from the same University (2003)
and an MSc (Dist) in Computer Science from the University of Essex, UK (2004). He participated in
several national and international research projects and published a number of international journal
and conference proceedings papers. He currently works as a Research Associate at the Information
Technologies Institute (ITI) of the Center for Research and Technology, Hellas (CERTH), where he
serves as the lead researcher for H2020 project SUITCEYES, and as a task leader in various other
H2020 projects. His research interests include Knowledge Representation and Reasoning, Semantic
Technologies and Rule-based Systems.
Tobias Hellmund received his M.Sc. in Economics and Engineering at Clausthal University of Technology. After
two years within car manufacturing, he switched to Fraunhofer IOSB in Karlsruhe where he works as research
associate in Semantic Web Technologies.
Anastasios Karakostas (m) received the Degree in Computer Science and the PhD degree in
Computer Science Aristotle University of Thessaloniki Greece. He is a Researcher with ITI- CERTH.
Currently, he is deputy coordinator and scientific manager of the H2020 DRS project beAWARE,
deputy coordinator and technical manager of the H2020 DRS project aqua3S and Risk Manager and
task leader in the H2020 DRS project INGENIOUS. He has also participated in numerous European
and national research projects and is the author of more than 60 publications in refereed journals
and international conference. His research interests include decision support systems, semantic
multimedia analysis, ontologies and semantic information modeling and reasoning. He is member
of the EU Community of Users for Secure, Safe and Resilient Societies and he is the chair of the
International Workshop on Intelligent Crisis Management Technologies for Climate Events (ICMT)
in ISCRAM 2018 and 2019.
Désirée Hilbring (f) holds a PhD in geodesy engineering (2005) from the University of Karlsruhe, Germany. She
is head of the research group “Geospatial Data Management” at Fraunhofer IOSB. She gained experiences with
modern service oriented architectures and the semantic web in several European and BMBF projects. Her recent
work focused on Social Media Analysis. Furthermore she manages the development of Water Information Systems
supporting the European Water Framework Directive (WRRL).
Hylke van der Schaaf holds a degree in Bioprocess technology from Wageningen Agricultural
University and a PhD in Biotechnology from Wageningen University. Since he joined Fraunhofer IOSB
in 2011 he has been involved in several EC projects and has worked in the fields of sensor data
management, geospatial information systems and the Internet of Things and has published several
papers. He is the main author of FROST-Server, the first complete, open-source implementation of
the SensorThings API.
Stefanos Vrochidis received the Diploma degree in Electrical Engineering from Aristotle University of
Thessaloniki, Greece, the MSc degree in Radio Frequency Communication Systems from University of
Southampton and the PhD degree in Electronic Engineering from Queen Mary University of London. Currently,
he is a Senior Researcher with the Information Technologies Institute. His research interests include semantic
multimedia analysis, indexing and retrieval, search engine and human interactions as well as digital TV
learning and environmental applications. Dr. Vrochidis has successfully participated in many European and
National projects and he has been involved as a co-author in more than forty-five related scientific journal,
conference and book chapter publications.
Jan Blume earned a Master of Science degree in Physics from Heidelberg University in 2017. Since 2018 he
works as a researcher at the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation. His
research interests include the visualization and analyzation of multimodal data.

Volume 10 • Issue 4 • October-December 2018
87
Ioannis (Yiannis) Kompatsiaris is a Senior Researcher (Researcher B’) with the Information Technologies
Institute / Centre for Research and Technology Hellas, Thessaloniki, Greece. His research interests
include semantic multimedia analysis, indexing and retrieval, social media analysis, knowledge structures,
reasoning and personalization for multimedia applications. He received his Ph.D. degree in 3-D model-
based image sequence coding from the Aristotle University of Thessaloniki in 2001. He is the co-author
of 57 papers in refereed journals, 30 book chapters, 7 patents and more than 170 papers in international
conferences. He has been the co-organizer of various international conferences and workshops and has
served as a regular reviewer for a number of journals and conferences. He is a Senior Member of IEEE
and member of ACM.
... The following two examples show the wide range of usage of semantic technologies. Semantic integration [4] [5] can be applied in the context of crisis response to support decision support [6]. Another example shows the implementation of semantic data to protect cultural heritage [7]. ...
Conference Paper
Full-text available
Recently, the usage of triplestores has increased in complex computer systems. Traditionally, they are used for representing static knowledge. In the last years, systems started using semantic triplestores in highly dynamic scenarios, e.g., in the context of civil protection. In these use cases, performance characteristics are more and more important. There are various aspects influencing the query performance. We have noticed that already the query structure has a significant impact on the execution time. SPARQL Protocol And RDF Query Language (SPARQL) is a widely used standard for querying triplestores. In this work, we have developed SPARQL query patterns and evaluated their performance characteristics. For this, a literature review was done to select a suitable benchmark. As a result, we provide eight recommendations for formulating SPARQL queries. These can be easily used by everybody without a deeper knowledge about the implementation of the triplestore, which contains the desired data.
Conference Paper
Full-text available
In every disaster and crisis, incident time is the enemy, and getting accurate information about the scope, extent, and impact of the disaster is critical to creating and orchestrating an effective disaster response and recovery effort. Decision Support Systems (DSSs) for disaster and crisis situations need to solve the problem of facilitating the broad variety of sensors available today. This includes the research domain of the Internet of Things (IoT) and data coming from social media. All this data needs to be aggregated and fused, the semantics of the data needs to be understood and the results must be presented to the decision makers in an accessible way. Furthermore, the interaction and integration with existing risk and crisis management systems are necessary for a better analysis of the situation and faster reaction times. This paper provides an insight into the sensor to decision chain and proposes solutions and technologies for each step.
Conference Paper
Full-text available
One of the most critical challenges faced by authorities during the management of a climate-related crisis is the overwhelming flow of heterogeneous information coming from humans and deployed sensors (e.g. cameras, temperature measurements, etc.), which has to be processed in order to filter meaningful items and provide crisis decision support. Towards addressing this challenge, ontologies can provide a semantically unified representation of the domain, along with superior capabilities in querying and information retrieval. Nevertheless, the recently proposed ontologies only cover subsets of the relevant concepts. This paper proposes a more "all-around" lightweight ontology for climate crisis management, which greatly facilitates decision support and merges several pertinent aspects: representation of a crisis, climate parameters that may cause climate crises, sensor analysis, crisis incidents and related impacts, first responder unit allocations. The ontology could constitute the backbone of the decision support systems for crisis management. Keywords Crisis management, ontology, semantic integration, decision support, description logics. Kontopoulos et al. Ontology-based Representation of Crisis Management WiPe/CoRe Paper-Track Name
Conference Paper
Full-text available
When crises hit, many flog to social media to share or consume information related to the event. Social media posts during crises tend to provide valuable reports on affected people, donation offers, help requests, advice provision, etc. Automatically identifying the category of information (e.g., reports on affected individuals, donations and volunteers) contained in these posts is vital for their efficient handling and consumption by effected communities and concerned organisations. In this paper, we introduce Sem-CNN; a wide and deep Convolutional Neural Network (CNN) model designed for identifying the category of information contained in crisis-related social media content. Unlike previous models, which mainly rely on the lexical representations of words in the text, the proposed model integrates an additional layer of semantics that represents the named entities in the text, into a wide and deep CNN network. Results show that the Sem-CNN model consistently outperforms the baselines which consist of statistical and non-semantic deep learning models.
Article
Full-text available
Natural hazard risk is largely projected to increase in the future, placing growing responsibility on decision makers to proactively reduce risk. Consequently, decision support systems (DSSs) for natural hazard risk reduction (NHRR) are becoming increasingly important. In order to provide directions for future research in this growing area, a comprehensive classification system for the review of NHRR-DSSs is introduced, including scoping, problem formulation, the analysis framework, user and organisational interaction with the system, user engagement, monitoring and evaluation. A review of 101 papers based on this classification system indicates that most effort has been placed on identifying areas of risk and assessing economic consequences resulting from direct losses. However, less effort has been placed on testing risk-reduction options and considering future changes to risk. Furthermore, there was limited evidence within the reviewed papers on the success of DSSs in practice and whether stakeholders participated in DSS development and use.
Chapter
Full-text available
Decision Support Systems (DSS) are complex technological tools, which enable an accurate and complete scenario awareness, by integrating data from both “external” (physical) situation and current behaviour and state of functioning of the technological systems. The aim is to produce a scenario analysis and to guess identify educated the most efficient strategies to cope with possible crises. In the domain of Critical Infrastructures (CI) Protection, DSS can be used to support strategy elaboration from CI operators, to improve emergency managers capabilities, to improve quality and efficiency of preparedness actions. For these reasons, the EU project CIPRNet, among others, has realised a new DSS designed to help operators to deal with the complex task of managing multi-sectorial CI crises, due to natural events, where many different CI might be involved, either directly or via cascading effects produced by (inter-)dependency mechanisms. This DSS, called CIPCast, is able to produce a real-time operational risk forecast of CI in a given area; other than usable in a real-time mode, CIPCast could also be used as scenario builder, by using event simulators enabling the simulation of synthetic events whose impacts on CI could be emulated. A major improvement of CIPCast is its capability of measuring societal consequences related to the unavailability of primary services such as those delivered by CI.
Conference Paper
Full-text available
Safety and security in football events has been a heavily debated topic in the media for years. Especially communication processes between the police authorities, private security services, town councils, supporters and other spectators have often been neglected. Communication processes in the context of football matches can encounter technical limitations. Traditional communication channels are often characterized by a one-way communication structure, delayed forwarding of information or limited quality (e.g., stadium announcements). To address these problems, a new communication platform is explored. The semantic basis for this platform is formed by an ontology. The usage and benefits of the ontology are manifold: 1) The design process created a better understanding among the stakeholders. 2) The actors, roles and their relations are used for information filtering and access restrictions. 3) The relations of the ontology are used for structuring the information and navigating through it. 4) Heterogeneous information is fused into the platform from existing systems by annotating the data with concepts from the ontology. In this paper, the design of the ontology based on research of already existing ontologies is presented.
Conference Paper
Crowdsourcing has the potential to become a crucial information source in disaster management. In order to become effective as an integrated part of disaster management systems it is important to set the general architectural foundations for such integrations beyond prototypical experiments. This paper discusses general architectural principles of the application of crowdsourc ing in Early Warning Systems (EWS). An integrated architecture is proposed to use classical sensor data and crowdsourcing in an EWS solution. Therefore, typical components of crowdsourcing applications are identified and mapped to monitoring subsystems of EWS’s . Three main stru ctural variants of applying crowdsourcing in early warning systems along the example of a prototypical extension of two existing large -scale hydro -meteorological warning systems are presented .
Article
The World Wide Web has succeeded in large part because its software architecture has been designed to meet the needs of an Internet-scale distributed hypermedia application. The modern Web architecture emphasizes scalability of component interactions, generality of interfaces, independent deployment of components, and intermediary components to reduce interaction latency, enforce security, and encapsulate legacy systems. In this article we introduce the Representational State Transfer (REST) architectural style, developed as an abstract model of the Web architecture and used to guide our redesign and definition of the Hypertext Transfer Protocol and Uniform Resource Identifiers. We describe the software engineering principles guiding REST and the interaction constraints chosen to retain those principles, contrasting them to the constraints of other architectural styles. We then compare the abstract model to the currently deployed Web architecture in order to elicit mismatches between the existing protocols and the applications they are intended to support.
Conference Paper
In this paper we introduce Graffoo, i.e., a graphical notation to develop OWL ontologies by means of yEd, a free editor for diagrams.