Content uploaded by Jürgen Moßgraber
Author content
All content in this area was uploaded by Jürgen Moßgraber on Dec 13, 2018
Content may be subject to copyright.
2nd International Conference
Citizen Observatories for natural hazards and Water Management
Venice, 27-30 November 2018
Applying Semantic Web Technologies for Decision Support in
Climate-Related Crisis Management
Efstratios Kontopoulos1, Panagiotis Mitzias1, Stamatia Dasiopoulou2, Jürgen Moßgraber3,
Simon Mille2, Philipp Hertweck3, Tobias Hellmund3, Anastasios Karakostas1, Stefanos
Vrochidis1, Leo Wanner2,4, & Ioannis Kompatsiaris1
1 CERTH-ITI, Information Technologies Institute, Thessaloniki, Greece – e-mail: skontopo@iti.gr,
pmitzias@iti.gr, akarakos@iti.gr, stefanos@iti.gr, ikom@iti.gr
2 Information and Communication Technologies Department, Pompeu Fabra University, Barcelona, Spain –
e-mail: stamatia.dasiopoulou@upf.edu, leo.wanner@upf.edu
3 Fraunhofer IOSB, Karlsruhe, Germany – e-mail: juergen.mossgraber@iosb.fraunhofer.de,
philipp.hertweck@iosb.fraunhofer.de, tobias.hellmund@iosb.fraunhofer.de
4 Catalan Institute for Research and Advanced Studies
KEY POINTS
During climate-related crises vast volumes of heterogeneous multimodal information are generated.
Meaningfully processing and communicating this information for efficient decision support is a key challenge.
The paper describes applying Semantic Web technologies for decision support during such crises.
We are proposing the application of these technologies in the whole “sensor to decision chain”.
This approach is being tested within the beAWARE EU project, with contributions by domain experts.
1 INTRODUCTION
The efficient management of climate-related crises poses several challenges to authorities, the most
important of which arguably concern the exchange of vast volumes of heterogeneous, multimodal
information coming from citizens (e.g. through social media posts), machines (e.g. deployed sensors), and
other stakeholders (e.g. weather forecasting services). State-of-the-art Semantic Web technologies provide
an excellent means towards alleviating the burden of processing, integrating, and meaningfully making use
of all this information, as they provide the required infrastructure for ensuring enhanced data integration and
information interoperability across different stakeholders (Sikos, 2015).
There have been several recent attempts of deploying Semantic Web technologies for climate-related
crisis management. Approaches typically either (a) propose common semantic representation models, or, (b)
deliver crisis management systems based on Semantic Web technologies. The most prominent approaches
belonging to the former group include the works by Limbu (2012), Babitski et al. (2011), Liu et al. (2014),
Lauras et al. (2015), and Mescherin et al. (2013). As for the latter group, Pandey & Bansal (2017) developed
a system based on semantic technologies for monitoring social media for earthquake reports and weather
alerts, and for notifying the public in case of an emergency. Moreover, Burel et al. (2017) propose the
encapsulation of a layer of semantics into a deep learning model for automatically classifying information
from social media posts. Poslad et al. (2015) developed an IoT early warning system for environmental crisis
management, where the use of semantics facilitated sensor and data source plug-and-play, simpler, richer,
and more dynamic metadata-driven data analysis and easier service interoperability and orchestration.
The main drawback of the above approaches is their narrow focus on specific parts of the pipeline of
processes from sensor data capturing, analysis, semantic representation and fusion, to reporting and decision
making. In contrast, we recently proposed the “sensor to decision chain”, namely, a holistic framework for
facilitating decision support by data integration via sensors and semantic data analysis (Moßgraber et al.,
2018). In this paper, we focus on the application of Semantic Web technologies in all the phases of this
framework, capitalizing thus on the significant benefits brought forth by these technologies. By using
Semantic Web technologies we aim to support crisis management systems in the domain of situational
awareness. Situational awareness refers to being able to accurately determine what has happened so far
during a crisis, what is happening now, and what will come next, all in order to plan and coordinate the most
effective response possible with the resources available. The framework is being tested within the
E. Kontopoulos et al. – Applying Semantic Web Technologies for Decision Support
beAWARE EU-funded project (http://beaware-project.eu/).
2 DEPLOYED SEMANTIC WEB TECHNOLOGIES
Figure 1 illustrates the steps of our Semantic Web technologies-enabled “sensor to decision chain”
framework towards managing a natural disaster crisis: Data coming from artificial and human sensors (1) is
fed to respective analysis components (2), and the analysis results are semantically integrated into a semantic
knowledge base (KB). The KB performs semantic reasoning (3) and forwards its outputs to a reporting
module, which provides authorities and decision makers with the appropriate information in natural language
(4) for facilitating decision support during the crisis.
Figure 1. Application of Semantic Web technologies in the sensor to decision chain.
2.1 The Semantic knowledge base
The backbone of the deployed technologies is a semantic knowledge base (KB), which is formalized as
an ontology (Fensel, 2001) that semantically integrates all the pertinent information: (a) natural disasters and
respective climate parameters; (b) analyzed sensor data; (c) rescue unit assignments (Kontopoulos et al.,
2018). For example, a video analysis algorithm may detect that a street is flooded and that several cars are
partially submerged. A corresponding flood incident is created in the ontology and is linked to the affected
objects, in this case the cars. Based on that, the system generates automated suggestions to the decision
maker; e.g. to check for people who might get trapped in a submerged car.
2.2 Semantic integration and semantic reasoning
System components perform analysis upon various resources (e.g. audio, text, images, videos and social
media posts) and submit their results to the KB. These analysis results are semantically integrated within the
ontology schema, and can, from now on, be treated by the authorities as homogeneous information, although
they originate from different sources. Figure 2 displays an indicative example of analyzed sensor data,
namely an image analysis instance, where a potentially injured person is detected in the flood.
Figure 2. Representation of analyzed data in the ontology.
A semantic reasoning mechanism integrated in the framework further facilitates decision support. This
mechanism consists of a SPARQL-based ruleset
1
, and is capable of inferring underlying knowledge (e.g.
establish implied interconnections of detected entities, incidents, etc.) from the semantically fused data
generated by the analysis components. The newly inferred knowledge is appended back into the ontology.
An indicative task handled by the reasoning mechanism involves the spatial clustering of incidents during
a crisis. Incoming information items (e.g. from social media) can refer to the same incident, and thus need to
be clustered based on their location. Our reasoning mechanism classifies all recorded incidents into groups
within a certain user-defined radius, protecting the end user from information overload. This process can be
enriched with other semantic criteria, such as temporal information and incident importance.
1
SPARQL (Harris et al., 2013) is a semantic query language for ontologies in the Semantic Web.
E. Kontopoulos et al. – Applying Semantic Web Technologies for Decision Support
Additional semantic reasoning examples include: (a) the dynamic (re)calculation of incident severities,
e.g. incidents of high risk involving human beings should be classified as “severe” in order to attract the
most attention by authorities and decision makers (see Figure 3); (b) the monitoring of safe locations and
relief spots. Regarding the latter, during a crisis, citizens could be notified about the existence of such
locations, safe detours etc. The reasoner is responsible for inferring the availability of these spots and for
determining optimal alternatives in case of low availability. For example, if a bridge is reported to have
collapsed during a flood, the closest safest river passage should be calculated and announced to the citizens.
Figure 3. SPARQL rule for calculating incident severity.
2.3 Reporting
The various customized alerts and knowledge gained through semantic integration and reasoning about
the unfolding crisis is communicated to authorities and decision makers via a verbalization framework that
translates the information contained in the ontology to multilingual natural language descriptions. The
translation extends our previous work (Mille & Dasiopoulou, 2017) and is realized in two steps. First the
semantic ontological representations are mapped to abstract linguistic predicate-argument (predArg)
representations that serve as language-independent lexicalization templates. Then, the predArg structures are
mapped to sentences through a sequence of processing tasks that is grounded in the Meaning-Text Theory
(Mel’cuk, 1988) and consists in: the mapping from the abstract predArg meanings onto lexical units of the
target language, the syntacticization of predicate-argument graphs, the introduction of function words, and
finally the linearization and retrieval of surface forms.
Figure 4. Predicate-argument, deep-syntactic and surface-syntactic structures produced during the generation of
the sentence “The Bacchigilone has overflowed at Angeli bridge”.
Figure 4 illustrates some of the intermediate structures, which are generated as part of the English
verbalization pipeline, given a KB with appropriate assertions that encapsulate the overflowing of the
Bacchiglione river at Angeli bridge. Angeli bridge, denoting a location, is associated with the preposition at
in the deep-syntactic structures and, being a member of the class “Bridge”, no determiner is introduced; on
the other hand, Bacchiglione, as a member of the class “River”, is assigned a definite determiner (i.e. the).
As first argument of the predicate overflow, Bacchiglione becomes the subject of the corresponding active
sentence. The relations of the surface-syntactic structure are used to determine the order and the
morphological agreements (e.g. has) between the words. If Bacchiglione was to be pronominalized, the
pronoun it – as opposed to he/she – would be selected.
3 CONCLUSIONS AND FUTURE WORK
In this paper, we described the application of key Semantic Web technologies for facilitating decision
support during climate-related crises. Contrary to other related approaches, we proposed the application of
these technologies in the whole “sensor-to-decision chain”: the representation of all pertinent aspects is
implemented through a semantic KB; the analysis of information coming from sensors and social media is
realized through semantic integration mechanisms; semantic reasoning processes facilitate decision support
E. Kontopoulos et al. – Applying Semantic Web Technologies for Decision Support
for authorities, while communication to decision makers is achieved through a semantic verbalization
framework that translates the KB encoded information to multilingual natural language reports. Based on the
above aggregation of multimodal information, our work contributes to the disaster management procedures
(such as crisis classification) in all the phases of a crisis. Moreover, it provides the foundations for the
decision support services to the authorities and can be integrated in relevant disaster management systems.
4 ACKNOWLEDGMENTS
This work has been partially funded by the European Commission under contracts H2020-700475
beAWARE and H2020-776019 EOPEN.
REFERENCES
Babitski, G., Probst, F., Hoffmann, J. and Oberle, D. (2009). Ontology Design for Information Integration in Disaster
Management. GI Jahrestagung, 154, 3120-3134.
Burel, G., Saif, H. and Alani, H. (2017, October). Semantic Wide and Deep Learning for Detecting Crisis-Information
Categories on Social Media. In International Semantic Web Conference (pp. 138-155). Springer, Cham.
Fensel, D. (2001), Ontologies. In Ontologies (pp. 11-18). Springer, Berlin, Heidelberg.
Harris, S., Seaborne, A. and Prud’hommeaux, E. (2013), SPARQL 1.1 query language, W3C recommendation, 21(10).
Kontopoulos, E. Mitzias, P. Moßgraber, J. Hertweck, P. van der Schaaf, H. Hilbring, D. Lombardo, F. Norbiato, D.
Ferri, M. Karakostas, A. Vrochidis, S. and Kompatsiaris, I. (2018), Ontology-based Representation of Crisis
Management Procedures for Climate Events, 1st Int. Workshop on Intelligent Crisis Management Technologies for
climate events (ICMT), collocated with ISCRAM 2018, to be presented.
Lauras, M., Truptil, S. and Bénaben, F. (2015). Towards a better management of complex emergencies through crisis
management meta‐modelling. Disasters, 39(4), 687-714.
Limbu, M. (2012). Management of a Crisis (MOAC) Vocabulary Specification. Available online:
http://www.observedchange.com/moac/ns/, last accessed: Apr’18.
Liu, Y., Chen, S., & Wang, Y. (2014). SOFERS: Scenario Ontology for Emergency Response System. JNW, 9(9),
2529-2535.
Mel’cuk, I. (1988), Dependency Syntax: Theory and Practice. SUNY Press, Albany.
Mescherin, S. A., Kirillov, I. and Klimenko, S. (2013, October). Ontology of emergency shared situation awareness and
crisis interoperability. In Cyberworlds (CW), 2013 International Conference on (pp. 159-162). IEEE.
Mille, S. and Dasiopoulou, S. (2017), FORGe at E2E 2017, E2E NLG challenge. Technical Report 17-12, Dept. of
Engineering and Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain
(http://www.macs.hw.ac.uk/InteractionLab/E2E/final_papers/E2E-FORGe.pdf).
Moßgraber, J. Hilbring, D. van der Schaaf, H. Hertweck, P. Kontopoulos, E. Mitzias, P. Karakostas, A. Vrochidis, S.
and Kompatsiaris, I. (2018), The sensor to decision chain in crisis management, 15th Int. Conf. on Information
Systems for Crisis Response (ISCRAM), to be presented.
Pandey, Y. and Bansal, S. K. (2017). A Semantic Safety Check System for Emergency Management. Open Journal of
Semantic Web (OJSW), 4(1), 35-50.
Poslad, S., Middleton, S. E., Chaves, F., Tao, R., Necmioglu, O. and Bügel, U. (2015). A semantic IoT early warning
system for natural environment crisis management. IEEE Trans. on Emerging Topics in Computing, 3(2), 246-257.
Sikos, L. (2015). Mastering structured data on the Semantic Web: From HTML5 microdata to linked open data.
Apress. ISBN: 9781484210499.