ArticlePDF Available

ForeSight -AI-based Smart Living Platform Approach

Authors:

Abstract and Figures

In the upcoming years, the internet of things (IoT) will enrich daily life. The combination of artificial intelligence (AI) and highly interoperable systems will bring context-sensitive multi-domain services to reality. This paper describes a concept for an AI-based smart living platform with open-HAB, a smart home middleware, and Web of Things (WoT) as key components of our approach. The platform concept considers different stakeholders, i.e. the housing industry, service providers, and tenants. These activities are part of the ForeSight project, an AI-driven, context-sensitive smart living platform .
Content may be subject to copyright.
Jochen Bauer*, Hilko Hoffmann, Ingo Zinnikus, Thomas Feld, Mathias Runge, Oliver Hinz,
Michael Hechtel, Christoph Konrad, Martin Holzwarth, Andreas Mayr, Sven Schneider, and
Jörg Franke
ForeSight - AI-based Smart Living Platform
Approach
https://doi.org/10.1515/cdbme-2020-3099
Abstract: In the upcoming years, the internet of things (IoT)
will enrich daily life. The combination of artificial intelligence
(AI) and highly interoperable systems will bring context-
sensitive multi-domain services to reality. This paper describes
a concept for an AI-based smart living platform with open-
HAB, a smart home middleware, and Web of Things (WoT) as
key components of our approach. The platform concept con-
siders different stakeholders, i.e. the housing industry, service
providers, and tenants. These activities are part of the Fore-
Sight project, an AI-driven, context-sensitive smart living plat-
form.
Keywords: smart home, smart living, web of things, interop-
erability
1 Motivation
The economic relevance of connected homes or buildings is
proven by several key figures [1, 2]. German Federal Minis-
ter Peter Altmaier described the market for smart living ap-
plications as a „mega-ecosystem of the future“, with „annual
growth rates of 25-30 %“ and expected revenues in Germany
of „tens of billions by 2025“ [3]. The size of the market is rel-
evant, i.e. if all 43 million households by 2030 were equipped
with smart home technology with an average value of 3,000
EUR, this would result in a market potential of 129 billion
EUR [4].
*Corresponding author: Jochen Bauer, Michael Hechtel,
Christoph Konrad, Martin Holzwarth, Andreas Mayr, Sven
Schneider, Jörg Franke, Institute for Factory Automation and
Production Systems, Fürther Straße 246b, 90429, Nuremberg,
Germany, e-mail: jochen.bauer@faps.fau.de
Hilko Hoffmann, Ingo Zinnikus, Deutsches Forschungszentrum
für Künstliche Intelligenz GmbH, Stuhlsatzenhausweg 3, 66123
Saarbrücken, Germany
Thomas Feld, Strategion GmbH, Albert-Einstein-Straße 1, 49076
Osnabrück, Germany
Mathias Runge, IoT connctd GmbH, Hardenbergstraße 32,
10623 Berlin, Germany
Oliver Hinz, Faculty of Economics and Business Administration,
Goethe University Frankfurt, Theodor-W.-Adorno-Platz 4, 60323
Frankfurt am Main, Germany
There are several definitions of the terms smart home and
smart living, that range from cell phone controlled lights to
full-fledged agent-controlled systems of all devices in several
buildings [5]. Common relevant domains for smart home are
energy, health, and home automation [6]. We define the term
smart home as a group of more than three devices which are
able to be controlled by another device and offer useful infor-
mation to the user. In addition, smart living integrates use cases
that are not limited to the location of the local home network.
Most smart home systems offer their users a central ac-
cess point in form of a user interface, for example, a smart-
phone app or an installed display. Usually, available system
devices are supplied by the manufacturer and an integration
of foreign components, i.e. from different ecosystems, is nor-
mally unwanted. The open-source middleware openHAB of-
fers an abstraction layer for devices and services. The core of
the openHAB middleware uses the layers 5 to 7 of the OSI
model. There are several addons or bindings, which address
other layers of the OSI model as well, e.g. the network binding
works on layer 4. Moreover, there are vendor- and protocol-
specific bindings, called to communicate with devices and ser-
vices. This is the lowest level of interoperable systems, so-
called technical interoperability. One step further is to agree
on specific syntax or file formats, like CSV or XML for data
storage or data exchange, known as syntactic interoperability.
Regarding the IoT, it is useful to strive for the next higher level
of interoperability, the semantic interoperability. At this level,
machines are able to detect the meaning from the data transfer,
i.e. it is possible for the machine to understand what task needs
to be completed to fulfill the specific use case. As a result, the
machine can develop its own optimization strategies [7]. The
availability and interchangeability of information about who
is allowed to do what kind of actions lead to the next level of
interoperability, the organizational or pragmatic interoperabil-
ity. Such a degree of interoperability has not been achieved in
the smart living sector yet. Besides, there are more levels of
interoperability, which are not considered in this paper. To get
an impression of requirements it is helpful to analyze system
approaches related to the smart living reference architecture
model [8].
To include semantic information into systems, ontologies
can be used. These are semantic orders of terms related to a do-
DE GRUYTER Current Directions in Biomedical Engineering 2020;6(3): 20203099
Open Access. © 2020 Jochen Bauer et al., published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 License.
2J. Bauer et al., ForeSight
main and their relationships [9]. One of these ontologies is the
WoT [10], which uses so-called Thing Descriptions [11] and
represents a standard recommended by the W3C to connect
(IoT) objects to the web. There are several research projects
which are using WoT-based approaches for the smart living
domain. In addition to symbolic AI methods like using on-
tologies, data-driven machine learning (ML) tools are gaining
more and more attention. Last but not least, these days lots of
innovative US located companies are offering their cloud plat-
forms for storing all this data, so a possibility for a holistic
cloud abstraction layer, which follows German and European
privacy and security requirements as well as improved data
sovereignty, will be helpful for European companies. Sum-
ming up, both ontology- and ML-based approaches are rele-
vant in the smart living domain and need to be chosen wisely
to solve upcoming challenges as well as an overall integrated
identity and access management.
In the following section, we will identify current chal-
lenges, which need to be addressed to utilize the WoT with
openHAB in the context of an AI-based smart living platform.
Afterwards, we provide a concept for an approach with respect
to the mentioned challenges, which will be discussed at the end
of the paper.
2 Challenges
Recently, middleware applications are beginning to ensure in-
teroperability [12]. The capabilities of visual-based configura-
tion tools for smart home environments are improving, plug
and play, comfortable device discovery, and installation pro-
cedures are more and more available. Remote access is possi-
ble by an additional cloud service, which is usually provided
by the system vendor or are run on premise. More and more
systems offer comfortable tools to configure the devices of
the smart home and the corresponding UI. Summing up, the
costs for a smart home are much lower than five years ago,
but there is still a need for improving the interoperability be-
tween the systems to increase the benefits for the tenants. Fur-
thermore, systems are offering application programming inter-
faces (API) access to third party applications like IFTTT, and
so the user is able to automate procedures on his own without
having much expertise in software development. Anyway, the
possibility to bring existing business models to an IoT-based
ecosystem needs to be improved. In a first step, a promising
platform approach needs to offer interoperability through all
layers of the ecosystem. In a second step, tools for configur-
ing business models into digital services, which will use the
platform, need to be created.
Ontologies should be considered by almost any smart liv-
ing platform project due to their ability to be extended . These
approaches should be combined with ML and an appropriate
strategy to store data. Nevertheless, it needs to be considered
which side effects are occurring due to the lack of existing
tools for the work with AI, i.e. relevant and existing ontolo-
gies. Smart home systems neither consider the complete life-
cycle of a building nor relevant processes of the housing indus-
try’s business cases. These stakeholders and their specific re-
quirements need to addressed as well; otherwise, the targeted
economic impact of the ForeSight platform will be unlikely.
Increasing the availability of semantic information is a neces-
sary first step for enabling further improvements of the current
smart living ecosystems.
3 Approach
In our ForeSight approach, smart home devices can be con-
trolled by openHAB or other smart home middleware systems
like digitalSTROM. WoT will be used as a mechanism to store
vendor-independent data models. On top of WoT, providers are
offering their services, i.e. smart home wide infotainment ser-
vices. To minimize the effort to offer such services, these com-
panies need to have access to tools, which are helping to create
these services from companies’ business models, i.e. create a
service like an IFTTT action with the help of a UI-based con-
figuration dialogue.
In ForeSight, AI plays an important role to achieve dif-
ferent goals, i.e. for offering so-called base services for en-
ergy management, object identification, localization as well as
identity and access management. Sometimes it is necessary to
use a cloud-based infrastructure to perform calculations, oc-
casionally it is important that the relevant data will not leave
the local network, so AI-based operations need to be done
on embedded or edge devices to ensure tenants’ privacy re-
gards. Privacy and security issues requirements are key factors
for a successful platform concept. Therefore, we extended the
current openHAB software to take these requirements into ac-
count [8]. Furthermore, data storage concepts need to address
these requirements as well, i.e. the current Gaia-X initiative,
which is driven by Germany and Europe to address the chal-
lenge of an hyperscaler-independent abstraction layer with a
focus on data sovereignty [14].
ForeSight uses a layer-based architecture to enable hor-
izontal and vertical interoperability, thus reducing almost all
vendor- or company-specific dependencies (see Fig. 1). Hori-
zontal interoperability ensures the cross-domain and manufac-
turer compatibility of devices. In contrast, vertical interoper-
ability provides interchangeability between multiple services,
J. Bauer et al., ForeSight 3
e.g. use cases where services are used by clients, which are
probably tenants from different energy suppliers. In ForeSight
we strive to create service engineering tools, so that existing
businesses will be able to master the digital transformation of
their business models effortlessly.
Due to the mentioned challenges above, IoT devices or
groups of such devices need to be as smart as possible. In
the concept of thinking objects (TOs), several strands of re-
search are combined: smart environments, i.e. the physical in-
frastructure (sensors, actuators, and networks), ambient intel-
ligence which refers to a „digital environment that proactively,
but sensibly, supports people in their daily lives“ [13], and AI,
in particular agent systems and a variety of ML techniques.
TOs represent singular physical as well as software-defined
devices, temporary or stable aggregations, and combinations
of physical and virtual devices. TOs aggregate and abstract
sensor data of devices. Abstraction and aggregation are also a
prerequisite for the provision of value-added services to users
instead of (raw) data and information. TOs, which represent
sensors and actuators in a room or the lighting in a build-
ing, can be combined for coordinated activities, e.g. to create
a pleasant atmosphere or to guide the residents through the
building. The coordination can be controlled by a predefined
behavior or a self-organized, goal-directed activity. To achieve
this, TOs need to tackle several challenges. They need to in-
tegrate themselves in highly connected environments, but they
should offer their services in barely connected and automated
surroundings, too. A seamless connection can be achieved by
means of syntactic and semantic interoperability. Conceptu-
ally, TOs are an extension of multi-agent systems, which pro-
vide a flexible paradigm for an integral framework, relating
the different properties such as reactivity (acting in a timely
fashion), autonomy, proactiveness (taking the initiative), and
communicative social behavior [15]. Agents perceive the en-
vironment through sensors, acquire new correlations based on
the perceptions, and possess reasoning abilities to select and
execute actions. Devices and software entities need to be co-
ordinated in an intelligent environment. A wide range of tech-
niques for behavioral modeling, rule-, planning-, and logic-
based approaches can be used for coordinating actions. A crit-
ical factor is the collection of contextual information such as
location, time, temperature, and, more challenging, the inten-
tion of the user. TOs must learn to intervene only when neces-
sary or when they are asked to do so. Activity learning [16] is
the key to reliable context detection. Accordingly, devices in
the ForeSight ecosystem act as TOs.
After describing the layer-based ForeSight architecture
and the idea of TOs, we focus on the connection of the middle-
ware, here openHAB, as one of ForeSight’s middleware sys-
tems and WoT to add semantic information. An openHAB ad-
don has been developed to transform openHAB things into the
WebThing object model utilizing the Mozilla WebThings Java
framework. The WebThings are hosted on a WebThing server
to which the addon can connect to by using an API. Status
changes of the items are communicated using websockets and
REST API calls. Services can also use this interface to inter-
act with the objects through the web. Each thing provides a
thing description which can be used by humans and machines
to interpret the semantics, and thus capabilities of the object
(see Fig. 2) are known to everyone. The combination of ag-
gregating the data of all devices and the existing semantic data
history in the cloud makes it possible to create a digital twin
of the smart living ecosystem with all corresponding benefits.
4 Discussion
Due to the utilization of the WoT architecture defined by the
W3C, the smart home objects will be made available in a data
model that was designed with cross-domain interoperability in
mind. Therefore, the services of the ForeSight platform will
be able to consume relevant information easily, and appli-
cation development for horizontally and vertically integrated
software will have a low barrier of entry for business partners.
Additionally, the ForeSight consortium consists of multiple in-
dustry partners who are experienced in running IoT platforms.
Therefore, it seems promising to find a team of companies
which will run the platform in the future and thus eliminate
potential questions regarding the future of the platform after
the research project has ended.
Author Statement
Research funding: This work was supported by the BMWi
(ForeSight). Conflict of interest: Authors state no conflict of
interest.
References
[1] IDCWorldwide. Quarterly Smart
Home Device Tracker. (2019).
https://www.idc.com/getdoc.jsp?containerId=prUS44971219
(last accessed 2020-07-02)
[2] MarketsandMarkets. Smart Home Market
by Product Global Forecast to 2024. (2019).
https://www.marketsandmarkets.com/Market-Reports/smart-
homes-and-assisted-living-advanced-technologie-and-global-
market-121.html (last accessed 2020-07-02)
[3] Altmaier P. Zukunftsmarkt Smart Living. Symposium Smarte
Wohnung von GdW, ANGA,Wirtschaftsinitiative Smart Liv-
ing. Berlin (2019). https://anga.de/smarte-wohnung (last
accessed 2020-07-02)
[4] ZVEI. Zukunftsmarkt Smart Living. Smart Home Haushalte in
Deutschland. Frankfurt; 2017.
4J. Bauer et al., ForeSight
Fig. 1: ForeSight’s layer-based architecture to enable horizontal and vertical interoperability for reducing almost all vendor- or company-
specific dependencies, here shown by the example of a smart shading use case.
Fig. 2: ForeSight platform approach considering different smart
home middleware systems, here openHAB, which interacts with a
WoT technology stack. On top of these technologies companies
create and connect their services as providers.
[5] M. Schiefer. Smart Home Definition and Security Threats. In:
2015 Ninth International Conference on IT Security Incident
Management & IT Forensics; 2015, p. 114–118.
[6] Bauer J, Kettschau A-K, Michl M, Bürner J, Franke J. Die
intelligente Wohnung als Baustein im Internet der Dinge:
Potenzialanalyse und Konzept einer domänenübergreifenden
Lösung. In: Weidner R RT, editor. Erste Transdisziplinäre
Konferenz zum Thema Technische Unterstützungssys-
teme, die die Menschen wirklich wollen. Hamburg; 2014,
p. 298–307.
[7] Noura M, Atiquzzaman M, Gaedke M. Interoperability in In-
ternet of Things: Taxonomies and Open Challenges. Mobile
Netw Appl 2019;24(3):796–809
[8] Bauer J, Hechtel M, Konrad C, Holzwarth M, Hoffmann H,
Feld T et al. ForeSight - An AI-driven Smart Living Platform,
Approach to Add Access Control to openHAB. In: Jmaiel M,
Mokhtari M, Abdulrazak B, Aloulou H, Kallel S, editors. The
impact of digital technologies on public health in developed
and developing countries: 18th International Conference,
ICOST 2020, Hammamet, Tunisia, June 24-26, 2020, Pro-
ceedings. Cham: Springer; 2020, p. 432–440
[9] Gruber TR. A translation approach to portable ontology spec-
ifications. Knowledge Acquisition 1993;5(2):199–220.
[10] Kovatsch M, Matsukura R. Web of Things (WoT) Architec-
ture. (2020) https://www.w3.org/TR/wot-architecture/ (last
accessed 2020-07-02)
[11] Charpenay V, Käbisch S, Kosch H. Introducing Thing De-
scriptions and Interactions: An Ontology for the Web of
Things. In: SR+SWIT@ISWC; 2016, p. 55–66
[12] Broring A, Schmid S, Schindhelm C-K, Khelil A, Kabisch S,
Kramer D et al. Enabling IoT Ecosystems through Platform
Interoperability. IEEE Softw. 2017;34(1):54–61.
[13] Augusto JC. Ambient Intelligence: The Confluence of Ubiq-
uitous/Pervasive Computing and Artificial Intelligence. In:
Schuster AJ, editor. Intelligent Computing Everywhere. Lon-
don: Springer-Verlag London Limited; 2007, p. 213–234.
[14] Hoppe G, et al. GAIA-X: A Pitch Towards
Europe Statusbericht zu Anwenderökosys-
temen und -anforderungen. BMWi. (2020)
https://www.bmwi.de/Redaktion/DE/Publikationen/Digitale-
Welt/gaia-x-a-pitch-towards-europe.pdf (last accessed 2020-
07-01)
[15] Wooldridge M, Jennings NR. Intelligent agents: the-
ory and practice. The Knowledge Engineering Review
1995;10(2):115–52.
[16] Cook DJ. Activity learning: Discovering, recognizing, and
predicting human behavior from sensor data. Hoboken, NJ:
Wiley; 2015
... One of the fastest growing data ecosystem is the "smart living mega ecosystem" [4] by extending the smart home concept beyond simple home automation. On a technological level, the cross-actor exchange of data and its utilization in AI technologies are of key importance [5]. Speech recognition, object identification or energy management are exemplary application fields of AI in the smart living domain. ...
... Following the approach of [38] for qualitative content analysis, a fivestep evaluation concept was applied to the raw data in order to further transform the transcripts to answer the research questions and derive the relevant information to enhance our model. The evaluation concept consists of the following phases: (1) Determination of research questions and analysis perspective, (2) Development of a category system, (3) Search texts for relevant information and assign them to corresponding categories, (4) Aggregation of information over different interviews, (5) Analyzing the results to answer RQ2. Based on the concepts derived from the literature review, a category system was designed to encode the Page 2019 transcribed interviews. ...
... Bauer J, et. al. [14] The author has created a concept that enables the AI platform to work in smart living applications. The services allow each of the individual and business users to be adequately secured and reliable. ...
Article
Full-text available
Traffic management system is a key parameterfor highly efficient transportation system rather than depending on the speed of the vehicles and how smart it works. Police may not have all the dynamic data, which changes in a fraction of seconds, such as traffic due to accidents. The idea is to propose a system that can identify a truck in a road network and make decisions on clearing the traffic signals by getting results from the individual junction in a network. The system uses Narrow Band Internet of Things (NB-IoT) which is faster and energy-saving, and it sends signals to the particular traffic system based on the previous or the next traffic node. The Artificial Intelligence (AI) model will be biased independently. The system sends a message using radio waves from unused bandwidth. The main aim of the system is to make the passengers spend less time on traffic signals and save time as well as energy of the transportation system. It works based on getting traffic density on an individual traffic signal and the system decides which message should be open and which to close.
Chapter
With the emerging Internet of Things (IoT) techniques in smart home applications, artificial intelligence (AI), and highly interoperable IoT s ystems enable the development of context-sensitive multi-domain services in smart homes [1]. However, while such systems create enormous challenges regarding security and privacy, the IoT practitioners may overlook certain security and privacy concerns such as European Union (EU) General Data Protection Regulation (GDPR). This paper describes the necessities to consider privacy- and security-related challenges for smart living platforms. Core elements of this contribution are a user survey to detect key aspects to fulfill users’ expectations and an in-detail description of a Gaia-X-compatible software technology stack for the smart living domain. The concept will be applied to a smart kitchen use case.
Article
Full-text available
Today, the Internet of Things (IoT) comprises vertically oriented platforms for things. Developers who want to use them need to negotiate access individually and adapt to the platform-specific API and information models. Having to perform these actions for each platform often outweighs the possible gains from adapting applications to multiple platforms. This fragmentation of the IoT and the missing interoperability result in high entry barriers for developers and prevent the emergence of broadly accepted IoT ecosystems. The BIG IoT (Bridging the Interoperability Gap of the IoT) project aims to ignite an IoT ecosystem as part of the European Platforms Initiative. As part of the project, researchers have devised an IoT ecosystem architecture. It employs five interoperability patterns that enable cross-platform interoperability and can help establish successful IoT ecosystems.
Conference Paper
Full-text available
Im SmartHome-Umfeld existieren zahlreiche Insellösungen, etwa für das Energiemanagement. Die Domäne Ambient Assisted Living gewinnt angesichts des demografischen Wandels zusehends an Bedeutung. Des Weiteren gibt es am Markt tragbare Sensorik, beispielsweise Armbänder zur Aktivitätsmessung. Es fehlen hingegen tragfähige Geschäftsmodelle. Ursächlich sind zwei Faktoren: Erstens ist die Bedienung der Systeme zu komplex, zweitens ist der wahrgenommene Nutzen pro Inselsystem zu gering. Das im Artikel dargestellte Lösungskonzept beinhaltet Entwürfe für eine nutzerzentriert entwickelte Oberfläche und zeigt Ideen zur Anbindung von Subsystemen (Hausautomation, Energiemanagement, Gesundheit, Kommunikationswerkzeuge). Die Webanwendung verbirgt die Komplexität der Subsysteme und bettet sich unaufdringlich in die Lebenswelt des Menschen ein.
Article
Full-text available
We elaborate on recent developments in the area of Ambient Intelli-gence. Our work includes a description of possible applications, a description of a general architecture that can help to define such systems and the computational process that can link perception through sensors with actuation after decision mak-ing.
Book
Defines the notion of an activity model learned from sensor data and presents key algorithms that form the core of the field Activity Learning: Discovering, Recognizing and Predicting Human Behavior from Sensor Data provides an in-depth look at computational approaches to activity learning from sensor data. Each chapter is constructed to provide practical, step-by-step information on how to analyze and process sensor data. The book discusses techniques for activity learning that include the following: Discovering activity patterns that emerge from behavior-based sensor data Recognizing occurrences of predefined or discovered activities in real time Predicting the occurrences of activities The techniques covered can be applied to numerous fields, including security, telecommunications, healthcare, smart grids, and home automation. An online companion site enables readers to experiment with the techniques described in the book, and to adapt or enhance the techniques for their own use. With an emphasis on computational approaches, Activity Learning: Discovering, Recognizing, and Predicting Human Behavior from Sensor Data provides graduate students and researchers with an algorithmic perspective to activity learning.
Conference Paper
The home of the future should be a smart one, to support us in our daily life. Up to now only a few security incidents in that area are known. Depending on different security analyses, this fact is rather a result of the low spread of Smart Home products than the success of such systems security. Given that Smart Homes become more and more popular, we will consider current incidents and analyses to estimate potential security threats in the future. The definitions of a Smart Home drift widely apart. Thus we first need to define Smart Home for ourselves and additionally provide a way to categorize the big mass of products into smaller groups.
Article
The concept of an agent has become important in both Artificial Intelligence (AI) and mainstream computer science. Our aim in this paper is to point the reader at what we perceive to be the most important theoretical and practical issues associated with the design and construction of intelligent agents. For convenience, we divide these issues into three areas (though as the reader will see, the divisions are at times somewhat arbitrary). Agent theory is concerned with the question of what an agent is, and the use of mathematical formalisms for representing and reasoning about the properties of agents. Agent architectures can be thought of as software engineering models of agents; researchers in this area are primarily concerned with the problem of designing software or hardware systems that will satisfy the properties specified by agent theorists. Finally, agent languages are software systems for programming and experimenting with agents; these languages may embody principl...
Quarterly Smart Home Device Tracker
  • Idcworldwide
IDCWorldwide. Quarterly Smart Home Device Tracker. (2019).
Smart Home Market by Product Global Forecast to 2024
  • Marketsandmarkets
MarketsandMarkets. Smart Home Market by Product Global Forecast to 2024. (2019).
Symposium Smarte Wohnung von GdW, ANGA,Wirtschaftsinitiative Smart Living
  • P Altmaier
  • Zukunftsmarkt Smart
  • Living
Altmaier P. Zukunftsmarkt Smart Living. Symposium Smarte Wohnung von GdW, ANGA,Wirtschaftsinitiative Smart Living. Berlin (2019). https://anga.de/smarte-wohnung (last accessed 2020-07-02)