Conference PaperPDF Available

Applying Semantic Web Technologies for Decision Support in Climate-Related Crisis Management

Authors:

Abstract and Figures

During climate-related crises vast volumes of heterogeneous multimodal information are generated. Meaningfully processing and communicating this information for efficient decision support is a key challenge. The paper describes applying Semantic Web technologies for decision support during such crises. We are proposing the application of these technologies in the whole “sensor to decision chain”. This approach is being tested within the beAWARE EU project, with contributions by domain experts.
Content may be subject to copyright.
2nd International Conference
Citizen Observatories for natural hazards and Water Management
Venice, 27-30 November 2018
Applying Semantic Web Technologies for Decision Support in
Climate-Related Crisis Management
Efstratios Kontopoulos1, Panagiotis Mitzias1, Stamatia Dasiopoulou2, Jürgen Moßgraber3,
Simon Mille2, Philipp Hertweck3, Tobias Hellmund3, Anastasios Karakostas1, Stefanos
Vrochidis1, Leo Wanner2,4, & Ioannis Kompatsiaris1
1 CERTH-ITI, Information Technologies Institute, Thessaloniki, Greece e-mail: skontopo@iti.gr,
pmitzias@iti.gr, akarakos@iti.gr, stefanos@iti.gr, ikom@iti.gr
2 Information and Communication Technologies Department, Pompeu Fabra University, Barcelona, Spain
e-mail: stamatia.dasiopoulou@upf.edu, leo.wanner@upf.edu
3 Fraunhofer IOSB, Karlsruhe, Germany e-mail: juergen.mossgraber@iosb.fraunhofer.de,
philipp.hertweck@iosb.fraunhofer.de, tobias.hellmund@iosb.fraunhofer.de
4 Catalan Institute for Research and Advanced Studies
KEY POINTS
During climate-related crises vast volumes of heterogeneous multimodal information are generated.
Meaningfully processing and communicating this information for efficient decision support is a key challenge.
The paper describes applying Semantic Web technologies for decision support during such crises.
We are proposing the application of these technologies in the whole “sensor to decision chain.
This approach is being tested within the beAWARE EU project, with contributions by domain experts.
1 INTRODUCTION
The efficient management of climate-related crises poses several challenges to authorities, the most
important of which arguably concern the exchange of vast volumes of heterogeneous, multimodal
information coming from citizens (e.g. through social media posts), machines (e.g. deployed sensors), and
other stakeholders (e.g. weather forecasting services). State-of-the-art Semantic Web technologies provide
an excellent means towards alleviating the burden of processing, integrating, and meaningfully making use
of all this information, as they provide the required infrastructure for ensuring enhanced data integration and
information interoperability across different stakeholders (Sikos, 2015).
There have been several recent attempts of deploying Semantic Web technologies for climate-related
crisis management. Approaches typically either (a) propose common semantic representation models, or, (b)
deliver crisis management systems based on Semantic Web technologies. The most prominent approaches
belonging to the former group include the works by Limbu (2012), Babitski et al. (2011), Liu et al. (2014),
Lauras et al. (2015), and Mescherin et al. (2013). As for the latter group, Pandey & Bansal (2017) developed
a system based on semantic technologies for monitoring social media for earthquake reports and weather
alerts, and for notifying the public in case of an emergency. Moreover, Burel et al. (2017) propose the
encapsulation of a layer of semantics into a deep learning model for automatically classifying information
from social media posts. Poslad et al. (2015) developed an IoT early warning system for environmental crisis
management, where the use of semantics facilitated sensor and data source plug-and-play, simpler, richer,
and more dynamic metadata-driven data analysis and easier service interoperability and orchestration.
The main drawback of the above approaches is their narrow focus on specific parts of the pipeline of
processes from sensor data capturing, analysis, semantic representation and fusion, to reporting and decision
making. In contrast, we recently proposed the sensor to decision chain, namely, a holistic framework for
facilitating decision support by data integration via sensors and semantic data analysis (Moßgraber et al.,
2018). In this paper, we focus on the application of Semantic Web technologies in all the phases of this
framework, capitalizing thus on the significant benefits brought forth by these technologies. By using
Semantic Web technologies we aim to support crisis management systems in the domain of situational
awareness. Situational awareness refers to being able to accurately determine what has happened so far
during a crisis, what is happening now, and what will come next, all in order to plan and coordinate the most
effective response possible with the resources available. The framework is being tested within the
E. Kontopoulos et al. Applying Semantic Web Technologies for Decision Support
beAWARE EU-funded project (http://beaware-project.eu/).
2 DEPLOYED SEMANTIC WEB TECHNOLOGIES
Figure 1 illustrates the steps of our Semantic Web technologies-enabled sensor to decision chain
framework towards managing a natural disaster crisis: Data coming from artificial and human sensors (1) is
fed to respective analysis components (2), and the analysis results are semantically integrated into a semantic
knowledge base (KB). The KB performs semantic reasoning (3) and forwards its outputs to a reporting
module, which provides authorities and decision makers with the appropriate information in natural language
(4) for facilitating decision support during the crisis.
Figure 1. Application of Semantic Web technologies in the sensor to decision chain.
2.1 The Semantic knowledge base
The backbone of the deployed technologies is a semantic knowledge base (KB), which is formalized as
an ontology (Fensel, 2001) that semantically integrates all the pertinent information: (a) natural disasters and
respective climate parameters; (b) analyzed sensor data; (c) rescue unit assignments (Kontopoulos et al.,
2018). For example, a video analysis algorithm may detect that a street is flooded and that several cars are
partially submerged. A corresponding flood incident is created in the ontology and is linked to the affected
objects, in this case the cars. Based on that, the system generates automated suggestions to the decision
maker; e.g. to check for people who might get trapped in a submerged car.
2.2 Semantic integration and semantic reasoning
System components perform analysis upon various resources (e.g. audio, text, images, videos and social
media posts) and submit their results to the KB. These analysis results are semantically integrated within the
ontology schema, and can, from now on, be treated by the authorities as homogeneous information, although
they originate from different sources. Figure 2 displays an indicative example of analyzed sensor data,
namely an image analysis instance, where a potentially injured person is detected in the flood.
Figure 2. Representation of analyzed data in the ontology.
A semantic reasoning mechanism integrated in the framework further facilitates decision support. This
mechanism consists of a SPARQL-based ruleset
1
, and is capable of inferring underlying knowledge (e.g.
establish implied interconnections of detected entities, incidents, etc.) from the semantically fused data
generated by the analysis components. The newly inferred knowledge is appended back into the ontology.
An indicative task handled by the reasoning mechanism involves the spatial clustering of incidents during
a crisis. Incoming information items (e.g. from social media) can refer to the same incident, and thus need to
be clustered based on their location. Our reasoning mechanism classifies all recorded incidents into groups
within a certain user-defined radius, protecting the end user from information overload. This process can be
enriched with other semantic criteria, such as temporal information and incident importance.
1
SPARQL (Harris et al., 2013) is a semantic query language for ontologies in the Semantic Web.
E. Kontopoulos et al. Applying Semantic Web Technologies for Decision Support
Additional semantic reasoning examples include: (a) the dynamic (re)calculation of incident severities,
e.g. incidents of high risk involving human beings should be classified as severe in order to attract the
most attention by authorities and decision makers (see Figure 3); (b) the monitoring of safe locations and
relief spots. Regarding the latter, during a crisis, citizens could be notified about the existence of such
locations, safe detours etc. The reasoner is responsible for inferring the availability of these spots and for
determining optimal alternatives in case of low availability. For example, if a bridge is reported to have
collapsed during a flood, the closest safest river passage should be calculated and announced to the citizens.
Figure 3. SPARQL rule for calculating incident severity.
2.3 Reporting
The various customized alerts and knowledge gained through semantic integration and reasoning about
the unfolding crisis is communicated to authorities and decision makers via a verbalization framework that
translates the information contained in the ontology to multilingual natural language descriptions. The
translation extends our previous work (Mille & Dasiopoulou, 2017) and is realized in two steps. First the
semantic ontological representations are mapped to abstract linguistic predicate-argument (predArg)
representations that serve as language-independent lexicalization templates. Then, the predArg structures are
mapped to sentences through a sequence of processing tasks that is grounded in the Meaning-Text Theory
(Melcuk, 1988) and consists in: the mapping from the abstract predArg meanings onto lexical units of the
target language, the syntacticization of predicate-argument graphs, the introduction of function words, and
finally the linearization and retrieval of surface forms.
Figure 4. Predicate-argument, deep-syntactic and surface-syntactic structures produced during the generation of
the sentence “The Bacchigilone has overflowed at Angeli bridge”.
Figure 4 illustrates some of the intermediate structures, which are generated as part of the English
verbalization pipeline, given a KB with appropriate assertions that encapsulate the overflowing of the
Bacchiglione river at Angeli bridge. Angeli bridge, denoting a location, is associated with the preposition at
in the deep-syntactic structures and, being a member of the class Bridge”, no determiner is introduced; on
the other hand, Bacchiglione, as a member of the class River”, is assigned a definite determiner (i.e. the).
As first argument of the predicate overflow, Bacchiglione becomes the subject of the corresponding active
sentence. The relations of the surface-syntactic structure are used to determine the order and the
morphological agreements (e.g. has) between the words. If Bacchiglione was to be pronominalized, the
pronoun it as opposed to he/she would be selected.
3 CONCLUSIONS AND FUTURE WORK
In this paper, we described the application of key Semantic Web technologies for facilitating decision
support during climate-related crises. Contrary to other related approaches, we proposed the application of
these technologies in the whole “sensor-to-decision chain”: the representation of all pertinent aspects is
implemented through a semantic KB; the analysis of information coming from sensors and social media is
realized through semantic integration mechanisms; semantic reasoning processes facilitate decision support
E. Kontopoulos et al. Applying Semantic Web Technologies for Decision Support
for authorities, while communication to decision makers is achieved through a semantic verbalization
framework that translates the KB encoded information to multilingual natural language reports. Based on the
above aggregation of multimodal information, our work contributes to the disaster management procedures
(such as crisis classification) in all the phases of a crisis. Moreover, it provides the foundations for the
decision support services to the authorities and can be integrated in relevant disaster management systems.
4 ACKNOWLEDGMENTS
This work has been partially funded by the European Commission under contracts H2020-700475
beAWARE and H2020-776019 EOPEN.
REFERENCES
Babitski, G., Probst, F., Hoffmann, J. and Oberle, D. (2009). Ontology Design for Information Integration in Disaster
Management. GI Jahrestagung, 154, 3120-3134.
Burel, G., Saif, H. and Alani, H. (2017, October). Semantic Wide and Deep Learning for Detecting Crisis-Information
Categories on Social Media. In International Semantic Web Conference (pp. 138-155). Springer, Cham.
Fensel, D. (2001), Ontologies. In Ontologies (pp. 11-18). Springer, Berlin, Heidelberg.
Harris, S., Seaborne, A. and Prud’hommeaux, E. (2013), SPARQL 1.1 query language, W3C recommendation, 21(10).
Kontopoulos, E. Mitzias, P. Moßgraber, J. Hertweck, P. van der Schaaf, H. Hilbring, D. Lombardo, F. Norbiato, D.
Ferri, M. Karakostas, A. Vrochidis, S. and Kompatsiaris, I. (2018), Ontology-based Representation of Crisis
Management Procedures for Climate Events, 1st Int. Workshop on Intelligent Crisis Management Technologies for
climate events (ICMT), collocated with ISCRAM 2018, to be presented.
Lauras, M., Truptil, S. and Bénaben, F. (2015). Towards a better management of complex emergencies through crisis
management metamodelling. Disasters, 39(4), 687-714.
Limbu, M. (2012). Management of a Crisis (MOAC) Vocabulary Specification. Available online:
http://www.observedchange.com/moac/ns/, last accessed: Apr’18.
Liu, Y., Chen, S., & Wang, Y. (2014). SOFERS: Scenario Ontology for Emergency Response System. JNW, 9(9),
2529-2535.
Melcuk, I. (1988), Dependency Syntax: Theory and Practice. SUNY Press, Albany.
Mescherin, S. A., Kirillov, I. and Klimenko, S. (2013, October). Ontology of emergency shared situation awareness and
crisis interoperability. In Cyberworlds (CW), 2013 International Conference on (pp. 159-162). IEEE.
Mille, S. and Dasiopoulou, S. (2017), FORGe at E2E 2017, E2E NLG challenge. Technical Report 17-12, Dept. of
Engineering and Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain
(http://www.macs.hw.ac.uk/InteractionLab/E2E/final_papers/E2E-FORGe.pdf).
Moßgraber, J. Hilbring, D. van der Schaaf, H. Hertweck, P. Kontopoulos, E. Mitzias, P. Karakostas, A. Vrochidis, S.
and Kompatsiaris, I. (2018), The sensor to decision chain in crisis management, 15th Int. Conf. on Information
Systems for Crisis Response (ISCRAM), to be presented.
Pandey, Y. and Bansal, S. K. (2017). A Semantic Safety Check System for Emergency Management. Open Journal of
Semantic Web (OJSW), 4(1), 35-50.
Poslad, S., Middleton, S. E., Chaves, F., Tao, R., Necmioglu, O. and Bügel, U. (2015). A semantic IoT early warning
system for natural environment crisis management. IEEE Trans. on Emerging Topics in Computing, 3(2), 246-257.
Sikos, L. (2015). Mastering structured data on the Semantic Web: From HTML5 microdata to linked open data.
Apress. ISBN: 9781484210499.
... Our approach is to use semantic technologies to integrate information from different sources and then apply reasoning to draw conclusions from the gathered information focusing on the semantics of geospatial data. While Kontopoulos et al. presented the overall approach in [3] and [4], this paper concentrates on two parts of the approach: firstly, accessing of time series data, e.g. sensor observations and, secondly, exploiting geospatial knowledge. ...
Conference Paper
Full-text available
In crisis management, it is crucial to have up-to-date data available to assess an ongoing situation correctly. This information originates from different sources, such as human observations, various sensors or simulation algorithms with heterogeneous geographic scope. The aggregation of such data is conducted by decision support systems to disburden the end-users from automatable tasks. By using semantic technologies for the integration, these systems can benefit from the expressional power of semantic queries. Therefore, all data that is available in the system should also be accessible for these queries. In the following, we present an approach how sensor data can be accessed through semantic queries and how geospatial knowledge can be integrated in a Decision Support System.
Conference Paper
Full-text available
In every disaster and crisis, incident time is the enemy, and getting accurate information about the scope, extent, and impact of the disaster is critical to creating and orchestrating an effective disaster response and recovery effort. Decision Support Systems (DSSs) for disaster and crisis situations need to solve the problem of facilitating the broad variety of sensors available today. This includes the research domain of the Internet of Things (IoT) and data coming from social media. All this data needs to be aggregated and fused, the semantics of the data needs to be understood and the results must be presented to the decision makers in an accessible way. Furthermore, the interaction and integration with existing risk and crisis management systems are necessary for a better analysis of the situation and faster reaction times. This paper provides an insight into the sensor to decision chain and proposes solutions and technologies for each step.
Conference Paper
Full-text available
One of the most critical challenges faced by authorities during the management of a climate-related crisis is the overwhelming flow of heterogeneous information coming from humans and deployed sensors (e.g. cameras, temperature measurements, etc.), which has to be processed in order to filter meaningful items and provide crisis decision support. Towards addressing this challenge, ontologies can provide a semantically unified representation of the domain, along with superior capabilities in querying and information retrieval. Nevertheless, the recently proposed ontologies only cover subsets of the relevant concepts. This paper proposes a more "all-around" lightweight ontology for climate crisis management, which greatly facilitates decision support and merges several pertinent aspects: representation of a crisis, climate parameters that may cause climate crises, sensor analysis, crisis incidents and related impacts, first responder unit allocations. The ontology could constitute the backbone of the decision support systems for crisis management. Keywords Crisis management, ontology, semantic integration, decision support, description logics. Kontopoulos et al. Ontology-based Representation of Crisis Management WiPe/CoRe Paper-Track Name
Conference Paper
Full-text available
When crises hit, many flog to social media to share or consume information related to the event. Social media posts during crises tend to provide valuable reports on affected people, donation offers, help requests, advice provision, etc. Automatically identifying the category of information (e.g., reports on affected individuals, donations and volunteers) contained in these posts is vital for their efficient handling and consumption by effected communities and concerned organisations. In this paper, we introduce Sem-CNN; a wide and deep Convolutional Neural Network (CNN) model designed for identifying the category of information contained in crisis-related social media content. Unlike previous models, which mainly rely on the lexical representations of words in the text, the proposed model integrates an additional layer of semantics that represents the named entities in the text, into a wide and deep CNN network. Results show that the Sem-CNN model consistently outperforms the baselines which consist of statistical and non-semantic deep learning models.
Article
Full-text available
(N.B. This is available as an open access article from the publisher at http://ieeexplore.ieee.org/xpl/abstractAuthors.jsp?reload=true&arnumber=7109842) An early warning system (EWS) is a core type of data driven Internet of Things (IoTs) system used for environment disaster risk and effect management. The potential benefits of using a semantic-type EWS include easier sensor and data source plug-and-play, simpler, richer, and more dynamic metadata-driven data analysis and easier service interoperability and orchestration. The challenges faced during practical deployments of semantic EWSs are the need for scalable time-sensitive data exchange and processing (especially involving heterogeneous data sources) and the need for resilience to changing ICT resource constraints in crisis zones. We present a novel IoT EWS system framework that addresses these challenges, based upon a multisemantic representation model. We use lightweight semantics for metadata to enhance rich sensor data acquisition. We use heavyweight semantics for top level W3C Web Ontology Language ontology models describing multileveled knowledge-bases and semantically driven decision support and workflow orchestration. This approach is validated through determining both system related metrics and a case study involving an advanced prototype system of the semantic EWS, integrated with a deployed EWS infrastructure.
Article
with the ability to assist people to make reasonable decisions in an emergency, scenario-response paradigm has been widely regarded as one of the most effective emergency management models. In order to promote the processing capacity of scenario information for Emergency Response System (ERS), a new kind of scenario ontology is proposed in this paper. Following the well-known modeling primitives, the scenario ontology is constructed on the basis of PROTON (Proto Ontology), the reference layer ontology of FactForge that includes some central datasets of Linked Open Data (LOD). The event module, scenario module and mitigation module comprise the core level scenario ontology, defined with some core concepts and relations, which can be reused in the domain level scenario ontology. Considering earthquake is a representative emergency, we analyze the scenario information required by decision makers in some earthquakes, and design the scenario ontology of earthquake as an example in the domain level scenario ontologies. To show the validity of the scenario ontology, we implement the scenario ontology of earthquake with Protégé and develop a prototype system, which can retrieve some parts of scenario information from FactForge and provide a visualization interface for decision makers to browse the scenario instances involved in an earthquake
Book
A major limitation of conventional web sites is their unorganized and isolated contents, which is created mainly for human consumption. This limitation can be addressed by organizing and publishing data, using powerful formats that add structure and meaning to the content of web pages and link related data to one another. Computers can “understand” such data better, which can be useful for task automation. The web sites that provide semantics (meaning) to software agents form the Semantic Web, the Artificial Intelligence extension of the World Wide Web. In contrast to the conventional Web (the “Web of Documents”), the Semantic Web includes the “Web of Data”, which connects “things” (representing real-world humans and objects) rather than documents meaningless to computers. Mastering Structured Data on the Semantic Web explains the practical aspects and the theory behind the Semantic Web and how structured data, such as HTML5 Microdata and JSON-LD, can be used to improve your site’s performance on next-generation Search Engine Result Pages and be displayed on Google Knowledge Panels. You will learn how to represent arbitrary fields of human knowledge in a machine-interpretable form using the Resource Description Framework (RDF), the cornerstone of the Semantic Web. You will see how to store and manipulate RDF data in purpose-built graph databases such as triplestores and quadstores, that are exploited in Internet marketing, social media, and data mining, in the form of Big Data applications such as the Google Knowledge Graph, Wikidata, or Facebook’s Social Graph. With the constantly increasing user expectations in web services and applications, Semantic Web standards gain more popularity. This book will familiarize you with the leading controlled vocabularies and ontologies and explain how to represent your own concepts. After learning the principles of Linked Data, the five-star deployment scheme, and the Open Data concept, you will be able to create and interlink five-star Linked Open Data, and merge your RDF graphs to the LOD Cloud. The book also covers the most important tools for generating, storing, extracting, and visualizing RDF data, including, but not limited to, Protégé, TopBraid Composer, Sindice, Apache Marmotta, Callimachus, and Tabulator. You will learn to implement Apache Jena and Sesame in popular IDEs such as Eclipse and NetBeans, and use these APIs for rapid Semantic Web application development. Mastering Structured Data on the Semantic Web demonstrates how to represent and connect structured data to reach a wider audience, encourage data reuse, and provide content that can be automatically processed with full certainty. As a result, your web contents will be integral parts of the next revolution of the Web.
Article
Managing complex emergency situations is a challenging task, mainly due to the heterogeneity of the partners involved and the critical nature of such events. Whatever approach is adopted to support this objective, one unavoidable issue is knowledge management. In the context of our research project, gathering, formalising and exploiting all the knowledge and information about a given crisis situation is a critical requirement. This paper presents some research results concerning this specific topic: from a theoretical point of view, the generic dimensions of crisis characterisation are defined, while from a technical point of view, we describe a software solution able to collect that knowledge (based on meta-models and ontologies). This is used to confront the characteristics of the situation (context) with characteristics of the resources (relief system) in order to design a suitable response. Finally, an illustrative example concerning a crash between a tanker truck and a train is described. © 2015 The Author(s). Disasters © Overseas Development Institute, 2015.
Conference Paper
The main article focus is creating of machine and human readable vocabulary describing emergencies. Vocabulary tends to solve problem of low interoperability and technological and semantic incompatibility of departmental information systems. Vocabulary is developed in the form of OWL ontology. Proposed ontology could help to establish shared situation awareness between all participants of crisis management (emergency services, medical services, police, population exposed to risk). Article describes all stages for creating such ontology, from creating high level conceptual metamodel to development of formal ontology and it evaluation in real information system prototype. Ontology depicts all phases of emergency management, from mitigation and preparedness to recovery. Developed ontology is deployed to public access and could be freely used in information systems, it could serve as semantic foundation for information exchange within "system-of-systems".
Ontology Design for Information Integration in Disaster Management
  • G Babitski
  • F Probst
  • J Hoffmann
  • D Oberle
Babitski, G., Probst, F., Hoffmann, J. and Oberle, D. (2009). Ontology Design for Information Integration in Disaster Management. GI Jahrestagung, 154, 3120-3134.
Management of a Crisis (MOAC) Vocabulary Specification
  • M Limbu
Limbu, M. (2012). Management of a Crisis (MOAC) Vocabulary Specification. Available online: http://www.observedchange.com/moac/ns/, last accessed: Apr'18.