Content uploaded by Jochen Bauer
Author content
All content in this area was uploaded by Jochen Bauer on Oct 14, 2019
Content may be subject to copyright.
ForeSight - Platform Approach
for Enabling AI-based Services
for Smart Living
Jochen Bauer1(B
), Hilko Hoffmann2, Thomas Feld3, Mathias Runge4,
Oliver Hinz5, Andreas Mayr1,KristinaF¨orster1, Franz Teske1,
Franziska Sch¨afer1, Christoph Konrad1,andJ¨org Franke1
1Institute for Factory Automation and Production Systems,
Friedrich-Alexander-University Erlangen-N¨urnberg,
Egerlandstraße 7-9, 91058 Erlangen, Germany
jochen.bauer@faps.fau.de
2Deutsches Forschungszentrum f¨ur K¨unstliche Intelligenz GmbH,
Stuhlsatzenhausweg 3, 66123 Saarbr¨ucken, Germany
3Strategion GmbH, Albert-Einstein-Straße 1, 49076 Osnabr¨uck, Germany
4IoT connctd GmbH, Hardenbergstraße 32, 10623 Berlin, Germany
5Faculty of Economics and Business Administration, Goethe University Frankfurt,
Theodor-W.-Adorno-Platz 4, 60323 Frankfurt am Main, Germany
Abstract. In future, smart home and smart living applications will
enrich daily life. These applications are aware of their context, use arti-
ficial intelligence (AI) and are therefore able to recognize common use
cases reliably and adapt these use cases individually with the current
user in mind. This paper describes a concept for such an AI-based plat-
form. The presented platform approach considers different stakeholders,
e.g. the housing industry, service providers and tenants.
Keywords: Artificial intelligence ·Ecosystem ·Platform ·Smart
home ·Smart living
1 Motivation
The term “Smart Living” comprises several areas that are separated today:
energy management, health and home automation [1]. Furthermore, smart home
is a core element in a connected world. There is need for intelligent applications,
which fulfil cross-domain use cases. Smart buildings, which include smart homes
and commercial buildings, will take an important role to enable smart grid [2]
and smart city related approaches. Such approaches can only be realised with
intelligent, situation-adaptive control opportunities and building related services
[3]. This leads to more comfort, better assistance and increased safety and secu-
rity as well as improved resource efficiency and reduced overall costs. The reason
Supported by German Federal Ministry for Economic Affairs and Energy.
c
The Author(s) 2019
J. Pag´an et al. (Eds.): ICOST 2019, LNCS 11862, pp. 204–211, 2019.
https://doi.org/10.1007/978-3-030-32785-9_19
ForeSight 205
for such advanced opportunities is the intense usage of AI. This paper describes
a concept for a platform approach to enable such an intense AI usage in smart
living.
The economic relevance of connected homes or buildings is proven by sev-
eral key figures [4,5]. In Germany there are currently approximately 19 million
residential buildings and around 42 million households. The residential build-
ings have a share of approximately 98% of the total building stock. 57% of all
Germans and 72% of single households lived for rent in 2015. [6] The 23 mil-
lion rental housing units are managed by about 68,000 companies. Due to the
situation described above, it is demanded at government level that the topic
of digitisation and connectivity for the housing industry will be addressed. For
other European countries, the situation seems not exactly the same, but similar.
2 State of the Art
There are currently various stand-alone smart home systems on the market.
Some of these are based on wired or wireless technologies, e.g. KNX [7], Home-
matic [8, S. 231], Z-Wave [8, S. 216], free@home or digitalSTROM [9]. Moreover,
most of the systems offer cloud-based remote access possibilities as an addi-
tional option for controlling the building from abroad. Of course, the biggest
value proposition for the tenant is if he is able to add various devices from dif-
ferent vendors and consume third party services in his system. There are various
middleware systems, which follow this idea like the openHAB system [10]or
ioBroker.
Middleware systems usually provide an abstraction layer for devices. Thus,
these systems are able to ensure data transfer between such systems and therefore
achieve interoperability. In smart homes and for assistance systems in general,
privacy and security issues [11] play an important role in Europe. This means
that each data record should be extended with the information, who is allowed
to do what actions at what time with it.
According to the paragraph before, it becomes obvious that available seman-
tic information is useful to include domain knowledge in suitable data structures.
For this purpose the concept of ontologies exists. Ontologies are semantic orders
of domain related terms and their relationships [12]. Due to ontologies it is pos-
sible to start semantic search operations, combine two knowledge bases or test
data for inconsistencies [13]. In the domain of smart living the SAREF ontology
[14], BRICK [15] and Web of Things [16] are ontologies or approaches which
should be investigated. The world of ontologies and knowledge representations
lost some importance in recent years because solutions for numerous challenges
were delivered by another part of AI, so-called machine learning (ML). In ML,
algorithms and statistical methods are used to find solutions for optimization
problems, e.g. more and more pre-trained neural networks offer robust high-
quality answers to related questions [21]. In the context of smart living, speech
recognition [22], object identification or energy management are relevant exam-
ples, where ML is used successfully.
206 J. Bauer et al.
3 Challenges
Requirements are rapidly increasing in the smart building context, i.e. for achiev-
ing the energy transition flexible control and monitoring mechanisms need to
be offered. Another challenge for German society is to assist elderly people and
enable them to live in their preferred environment as long as possible. To achieve
this, AI-based smart living technologies can help to extend this period of time.
Thus, it is necessary to increase the amount of connected devices and enable
energy consumption measurement and localisation technologies to create indi-
vidual adaptable use cases. Currently, systems lack this functionality and do
not offer such individually profile-based approaches. Future smart home systems
need to have the specific user in mind, detect users’ behaviour and make the
correct conclusions. Smart speaker can help here to locate and identify persons.
Moreover voice user interfaces (VUI) can bring smart home technology to people
who are not used to smartphones. The first numbers of sold smart speakers show
that VUIs are a game changer. Anyway, there are lots of people who are still
reluctant, because they do not know what happens to their data. Summing up,
promising smart home platforms should offer a VUI and address privacy issues
as well.
Existing middleware systems are slowly beginning to ensure interoperabil-
ity [18,19]. In addition to automating tasks, smart home systems begin to offer
authentication options based on OAuth. This is pretty common for social media
websites, i.e. signing in with your Google or Facebook account to another web-
site [20]. On the one hand, this is quite comfortable for the user but on the other
hand it will be even more advanced to sign in with your preferred service and
avoid sharing data to lots of different servers abroad. Current platforms and eco
systems lack organizational interoperability. This interoperability level should
be persuaded, because only then it will be possible to create a flexible eco sys-
tem with different partners, who are able to include their specific services and
corresponding business models. The ideal place to store semantic information is
in an ontology, e.g. the SAREF ontology. Ontologies can be extended [17], hence
this should be considered by almost any platform project in the smart living
domain.
Furthermore, concepts are needed which enable the transition from hard-
coded if-else-statements towards context aware, dynamic and situation adap-
tive, self-learning approaches, which consider future trends as well. These new
approaches need to add to current best practice approaches, e.g. plug-and-play
mechanisms like the OSGI-based openHAB system. Changing the perspective
from system to device, it is possible to speak from “Thinking Objects”.
4 Approach
The following ForeSight platform approach was initiated from members of the
German Smart Living Business Initiative, a network of smart home and smart
building experts. The goal of the ForeSight platform is to create a significant
ForeSight 207
contribution to the further development of smart buildings and homes by pro-
viding cross-domain AI-based solutions in combination with established building
automation technologies.
In the past, there have been various platform approaches, most of these were
not able to conquer the market. There are technical, legal and economic chal-
lenges to deal with. In a first step core partners were allowed to found ForeSight.
In this six months lasting project a useful platform concept needs to be created
and relevant partners have to be identified, so that the value added chain is
completed. In addition to that, established research institutes and organisations
are added to the consortium to develop the concept further and further.
Due to literature research and experience, possible partners were detected.
Afterwards, the team decided to start the working phase by founding a matrix
based organisation, composed of several working groups (WG) (see Fig.1). The
figure shows WGs with domain specific focus and groups with cross-sectoral
topics. The WG coordinators need to collect relevant data of their WG and
interconnect with other WGs.
InstallaƟon,
commissioning and
operaƟon WG 4
Service
engineering
WG 9
Tes Ɵng and evaluaƟon
WG 5
AI and ML
WG 7
IT infrastructure, data, security and privacy (WG 6)
Interoperability (WG 8)
Use cases,
smart service
concepts and
smart service
architecture
Smart data and
AI algorithms
OperaƟonal
concepts,
intelligent services
and processes
Intelligent
environments and
laboratories
Comfort,
assistance
WG 1
Energy
management
WG 2
Access control
WG 3
Fig. 1. The consortium created working groups to start an efficient process to push
the ForeSight concept creation.
The created ForeSight platform concept should be evaluated and verified in
laboratories and real world environments. Therefore, possible user stories and
use cases need to be identified and implemented. There are several user story
catalogues but usually there is no direct combination with AI, which is one of
the core elements of this platform approach. Therefore, a new template for user
stories was created and given to the members of the different WGs. This template
considers relations to AI services, which are necessary for the specific use case.
With reference to the challenges section, ForeSight wants to reach a high level
of interoperability, i.e. that service providers with different business models can
replace their services without much effort. The resident focuses on one use case
and the platform takes care of necessary processes in the background. To iden-
tify users is one of ForeSight’s basic features and the key to enable user specific
208 J. Bauer et al.
use case adaptions. Horizontal interoperability ensures that the manufacturers
and cross-domain interchangeability of devices works. Vertical interoperability
considers that interoperability between two services is available, i.e. energy man-
agement use cases are interchangeable, although tenants are clients from different
energy providers. Following the thinking objects approach it is necessary to offer
the most appropriate AI method in relation to a client’s use case. It makes sense
to offer three subsystems: first, the AI method platform module; second, an IoT
platform module to make sure that all commands can be transported to sev-
eral vendor-independent devices and third, service-related apps which can be
built and ran on the IoT platform module and are allowed to connect to the
AI module. Several AI base services are necessary and will be made available in
ForeSight:
– Service for activity recognition of tenants
– Service for object identification
– Service for predictive maintenance related to Thinking Objects
– Service for self-configuration of Thinking Objects
– Service for position detection of tenants and Thinking Objects
– Service for identity and access management
– Service for optimized energy management in a group of Thinking Objects
– Service for technical-based health analysis of a building
– Service for privacy and security issues in a group of Thinking Objects
The AI platform connects with ForeSight service apps and the IoT platform
component. This service-related app offers the AI module data and a preferred
use case. Now the AI method finder needs to check the quality of the data, pick
the most appropriate AI tool and start the function. Afterwards, the answer
of the AI based platform module will be sent back. To generate high quality
answers for accessing apps, several tasks need to be addressed (see Fig. 2).
To get an impression of the time schedule, an excerpt of the ForeSight
roadmap is shown below.
– 2019, 2020: creation of the ForeSight platform concept and identification of
all project partners
– 2020, 2021: implementation of reference architecture and AI-based context
sensitive services
– 2021, 2022: realization of service layer to achieve interoperability on business
model layer
– 2023, ... : ForeSight entering the market for third party service providers
In addition to the time schedule we defined some milestones and evaluation
steps:
– 2019: Partners are able to contribute to all steps of the value added chain.
– 2020: Partners are willing and able to run the platform from 2020 to 2030.
ForeSight 209
ML systemsInteroperability
IoT, Middleware-
systems
Learning on
generaliseddata
Behavioural
modelling
Agent-based
approaches
Compared to
if-else-relaƟons
SemanƟc
descripƟon
Mapping of
datamodels
PotenƟal
statements
Quality of
statements
Trends
Basic services TesƟng environments
SemanƟc building
laboratory
ComputaƟonal
infrastructures
Requirementsfor
AI and ML
Ontologies,
reasoning
Data Method development TesƟngImplementaƟon Use cases /
demonstrators
AI on embedded
systems
Example services for:
Comfort assistant
InstallaƟon, integraƟon
and planning
Energy
management
Building management,
maintenance
Housing industry
Safety
Intelligent digital
twins, ML
Self-integraƟon
Thinking objects
Agent systems
Generate, exchangeand extend plaƞormrelated know-how
Method improvement, update policies, business models, sustainability, proposalsfor standards
Fig. 2. The AI method platform offers different components to fulfil the needs of access-
ing service apps.
– 2021: Tests in labs and real world scenarios achieve satisfactory results and
robustness.
– 2022: 10 third-party companies connect to the ForeSight platform with their
own services.
– 2023: More than 80% of existing smart living systems are using a ForeSight
platform service.
5 Discussion
When designing a platform, one of the first questions which arises is who wants to
operate such a platform. The ForeSight consortium consists of several companies,
which are experienced in running platforms. A flexible service-based approach
for running the platform seems to outperform a monolithic approach, as first
interviews showed.
Numerous use cases from the ML area require high quality data in order to
train the ML models, e.g. neural networks. Such data is often not available and
must be generated, which usually takes an enormous amount of time. It will be
hard to consider such soft facts in the AI method finder function to answer the
service-based apps.
In the past, several critical factors for smart home platforms were identified,
i.e. IT security, privacy and economic beneficial business models for all stake-
holders. The appropriate treatment of these topics needs to be considered from
the very beginning to avoid major concept changes later, which results in high
costs.
210 J. Bauer et al.
References
1. Bauer, J., Kettschau, A., Michl, M., B¨urner, J., Franke, J.: Die intelligente Woh-
nung als Baustein im Internet der Dinge: Potenzialanalyse und Konzept einer
dom¨anen¨ubergreifenden L¨osung. In: Weidner, R., Redlich, T. (eds.) Erste Trans-
diziplin¨are Konferenz zum Thema Technische Unterst¨utzungssysteme, die die Men-
schen wirklich wollen., pp. 298–307. Hamburg (2014)
2. Kunold, I., Kuller, M., Bauer, J., Karaoglan, N.: A system concept of an energy
information system in flats using wireless technologies and smart metering devices.
In: Proceedings of the 6th IEEE International Conference on Intelligent Data
Acquisition and Advanced Computing Systems, pp. 812–816 (2011)
3. Bakakeu, J., Sch¨afer, F., Bauer, J., Michl, M.: Building cyber-physical systems -
a smart building use case. In: Song, H., Srinivasan, R., Sookoor, T., Jeschke, S.
(eds.) Smart Cities: Foundations, Principles, and Applications, pp. 605–640. Wiley,
Bonn (2017)
4. IDC Worldwide: Quarterly Smart Home Device Tracker, 29 March 2019. https://
www.idc.com/getdoc.jsp?containerId=prUS44971219
5. MarketsandMarkets: Smart Home Market by Product (Lighting Control, Security
& Access Control, HVAC, Entertainment, Smart Speaker, Home Healthcare,
Smart Kitchen, Home Appliances, and Smart Furniture), Software & Services,
and Region - Global Forecast to 2024. https://www.marketsandmarkets.com/
Market-Reports/smart-homes-and- assisted-living-advanced-technologie-and-
global-market-121.html
6. Statista: ¨
Ubersicht Wohnen. https://de.statista.com/themen/51/wohnen/.
Accessed 16 June 2019
7. Kriesel, W., Sokollik, F., Helm, P.: KNX/EIB f¨ur die Geb¨audesystemtechnik in
Wohn- und Zweckbau. H¨uthig Verlag, Heidelberg (2014)
8. Heinle, S.: Heimautomation mit KNX, DALI, 1-Wire und Co. Rehinwerk Comput-
ing, Bonn (2016)
9. Dickmann, G.: DigitalSTROM R
: a centralized PLC topology for home automation
and energy management (2011)
10. Spiller, M.: Smart Home mit openHAB 2. Rheinwerk Computing, Bonn (2018)
11. Zibuschka, J., Nofer, M., Zimmermann, C., Hinz, O.: Users’ preferences concerning
privacy properties of assistant systems on the internet of things. In: American
Conference on Information Systems (AMCIS 2019) (2019, forthcoming)
12. Gruber, T.R.: A translation approach to portable ontology specification. Knowl.
Acquis. 5, 199–220 (1993)
13. Sabou, M.: An introduction to semantic web technologies. Semantic Web Technolo-
gies for Intelligent Engineering Applications, pp. 53–81. Springer, Cham (2016).
https://doi.org/10.1007/978-3-319- 41490-4 3
14. Daniele, L., den Hartog, F., Roes, J.: Created in close interaction with the industry:
the Smart Appliances REFerence (SAREF) ontology. In: Cuel, R., Young, R. (eds.)
FOMI 2015. LNBIP, vol. 225, pp. 100–112. Springer, Cham (2015). https://doi.org/
10.1007/978-3-319-21545- 7 9
15. Balaji, B., et al.: Brick: towards a unified metadata schema for buildings. In: ACM
Proceedings of the 3rd ACM International Conference on Systems for Energy-
Efficient Built Environments, New York, pp. 41–50 (2016)
16. Guinard, D., Trifa, V.: Towards the web of things: web mashups for embedded
devices. In: Workshop on Mashups, Enterprise Mashups and Lightweight Compo-
sition on the Web (MEM 2009), In Proceedings of WWW (International World
Wide Web Conferences) Madrid (2009)
ForeSight 211
17. Daniele, L., Solanki, M., den Hartog, F., Roes, J.: Interoperability for smart appli-
ances in the IoT world. In: Groth, P., et al. (eds.) ISWC 2016. LNCS, vol. 9982,
pp. 21–29. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46547-0 3
18. Kreuzer, K.: Semantic modelling of smart buildings with the open source software
openHAB smart public building website. http://www.hft-stuttgart.de/Forschung/
icity/Handlungsfelder/Explorative-Projekte-i-city/Exploratives-Pro jekt-SPUB/
de/. Accessed 16 June 2019
19. Broering, A., et al.: Enabling IoT ecosystems through platform interoperability.
IEEE Softw. 34(1), 54–61 (2017)
20. Rossnagel, H., Zibuschka, J., Muntermann, J., Hinz, O.: Users’ willingness-to-pay
for web identity managment. Eur. J. Inf. Syst. (EJIS) 23, 36–50 (2014)
21. Bauer, J., et al.: Camera-based fall detection system with the service robot sanbot
ELF. In: Uckelmann, D., (ed.) Smart Public Building 2018 Conference Proceedings,
Stuttgart, DE, pp. 15–28 (2018)
22. Pan, Y., Shen, P., Shen, L.: Speech emotion recognition using support vector
machine. Int. J. Smart Home 6(2), 101–108 (2012)
Open Access This chapter is licensed under the terms of the Creative Commons
Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/),
which permits use, sharing, adaptation, distribution and reproduction in any medium
or format, as long as you give appropriate credit to the original author(s) and the
source, provide a link to the Creative Commons license and indicate if changes were
made.
The images or other third party material in this chapter are included in the
chapter’s Creative Commons license, unless indicated otherwise in a credit line to the
material. If material is not included in the chapter’s Creative Commons license and
your intended use is not permitted by statutory regulation or exceeds the permitted
use, you will need to obtain permission directly from the copyright holder.
