Conference PaperPDF Available

Augmented Coaching Ecosystem for Non-obtrusive Adaptive Personalized Elderly Care on the basis of Cloud-Fog-Dew computing paradigm


Abstract and Figures

The concept of the augmented coaching ecosystem for non-obtrusive adaptive personalized elderly care is proposed on the basis of the integration of new and available ICT approaches. They include multimodal user interface (MMUI), augmented reality (AR), machine learning (ML), Internet of Things (IoT), and machine-to-machine (M2M) interactions. The ecosystem is based on the Cloud-Fog-Dew computing paradigm services, providing a full symbiosis by integrating the whole range from low level sensors up to high level services using integration efficiency inherent in synergistic use of applied technologies. Inside of this ecosystem, all of them are encapsulated in the following network layers: Dew, Fog, and Cloud computing layer. Instead of the "spaghetti connections", "mosaic of buttons", "puzzles of output data", etc., the proposed ecosystem provides the strict division in the following dataflow channels: consumer interaction channel, machine interaction channel, and caregiver interaction channel. This concept allows to decrease the physical, cognitive, and mental load on elderly care stakeholders by decreasing the secondary human-to-human (H2H), human-to-machine (H2M), and machine-to-human (M2H) interactions in favor of M2M interactions and distributed Dew Computing services environment. It allows to apply this non-obtrusive augmented reality ecosystem for effective personalized elderly care to preserve their physical, cognitive, mental and social well-being.
Content may be subject to copyright.
Augmented Coaching Ecosystem for Non-
obtrusive Adaptive Personalized Elderly Care on
the Basis of Cloud-Fog-Dew Computing
Yu.Gordienko1*, S.Stirenko1, O.Alienin1, K.Skala2, Z.Soyat2, A.Rojbi3, J.R.López Benito4, E.Artetxe
González4, U.Lushchyk5, L.Sajn6, A.Llorente Coto7, G.Jervan8
1 National Technical University of Ukraine "Igor SIkorsky Kyiv Polytechic Institute" (NTUU KPI), Kyiv, Ukraine
2 Ruder Boskovic Institute, Zagreb, Croatia
3 University of Paris 8, Paris, France
4 CreativiTIC Innova SL, Logroño, Spain
5 Medical Research Center "Veritas", Kyiv, Ukraine
6 University of Ljubljana, Ljubljana, Slovenia
7 Private Planet, London, United Kingdom
8 Tallinn University of Technology, Tallinn, Estonia
Abstract - The concept of the augmented coaching
ecosystem for non-obtrusive adaptive personalized elderly
care is proposed on the basis of the integration of new and
available ICT approaches. They include multimodal user
interface (MMUI), augmented reality (AR), machine
learning (ML), Internet of Things (IoT), and machine-to-
machine (M2M) interactions. The ecosystem is based on the
Cloud-Fog-Dew computing paradigm services, providing a
full symbiosis by integrating the whole range from low level
sensors up to high level services using integration efficiency
inherent in synergistic use of applied technologies. Inside of
this ecosystem, all of them are encapsulated in the following
network layers: Dew, Fog, and Cloud computing layer.
Instead of the "spaghetti connections", "mosaic of buttons",
"puzzles of output data", etc., the proposed ecosystem
provides the strict division in the following dataflow
channels: consumer interaction channel, machine
interaction channel, and caregiver interaction channel. This
concept allows to decrease the physical, cognitive, and
mental load on elderly care stakeholders by decreasing the
secondary human-to-human (H2H), human-to-machine
(H2M), and machine-to-human (M2H) interactions in favor
of M2M interactions and distributed Dew Computing
services environment. It allows to apply this non-obtrusive
augmented reality ecosystem for effective personalized
elderly care to preserve their physical, cognitive, mental and
social well-being.
A. Background
The advances in medicine and living standards in the
last century have resulted in a significant increase in the
number of elderly people in Europe and most other
developed countries in the world. Over the next decades,
the worldwide number of older people will further
increase dramatically. In Europe, this development is even
more pronounced: for example, in Portugal, Spain, Croatia
and other European countries, the old age dependency
ratio, which gives the quotient of people 65+ will reach
~30-36% with pan-European average value up to 29.6% in
2050 [1]. These demographic changes have drastic
structural, societal and economic implications, and
challenge elderly care stakeholders like policymakers,
families, businesses and healthcare providers alike. The
ever increasing percentage of old people in the most
advanced Western and Eastern countries is posing a great
challenge in social healthcare systems. The effort required
by formal caregivers for supporting older people can be
enormous, and this requires an increase in the efficiency
and effectiveness of today care. One way for achieving
such a goal is the use of information and communication
technologies (ICTs) for supporting and assisting people in
their own homes.
Older generations need to be included as active and
integral pillars of our society instead of being isolated in
the special elderly care facilities. They should remain
active members of the work force as long as possible,
since the traditional assumption that retirement equals the
worker’s final exit from the labor force does not hold true
any longer. The required transition of society can only be
successful if huge efforts are made on various levels to
foster independence of this age group, from more flexible
employment arrangements, remote services in care giving
(telecare), support of independent living (ambient assisted
living - AAL), access to information, access to
transportation (accessibility), to specific communication
services and devices as well as entrepreneur approaches in
educational offers like life-long learning (LLL).
ICT is believed to play a key role in all these fields.
However, ICT can successfully contribute to their
individual well being, and help to meet the challenges of
an aging society in general, if ICT could be non-
obtrusively adapted to the older adults’ knowledge, needs,
and abilities. Furthermore, the whole of our society can
gain enormous benefits by integrating the knowledge and
skills and high degree of experience the elderly can
provide to the coming generations, in all aspects of living,
from technological expertise in any field, to everyday
living experiences. Current ICTs range from systems for
reminding appointments and activities [2], for medical
assistance and tele-healthcare [3], to human-computer
interfaces for older persons or people with special needs
[2]. Usually, these ICTs incorporate application dependent
sensors, such as sensors, cameras or microphones. Many
studies [4] have demonstrated that people prefer non-
invasive sensors, such as microphones, over cameras and
wearable sensors, and this drove the scientific community
to develop systems and technologies based on non-
invasive approaches only.
ICTs have become an integral component of
everyone’s life, including older adults, to continue
education, obtain health information, communicate and
exchange experiences, as well as online banking/shopping
etc. Though recent research has shown that older adults
are receptive to using ICTs, a commonly held belief is still
prevalent that supports the idea that older adults are
unwilling to use ICTs due to bodily and cognitive decline
in working memory, attention, and spatial abilities [5,6].
The main problem is that despite the current progress
of elderly care facilities the vast majority of EU older
people wish to live independently at home as long as
possible; meeting their needs can be a major challenge [7].
The different providers often work under conditions of
poor coordination among ICT experts, elderly caregivers,
patients, and their families [8-9].
B. State of the Art (Similar Works)
ICTs are promising for the long-term care of elderly
people. As all European member states are facing an
increasing complexity of health and social care, good
practices in ICTs should be identified and evaluated.
Recently, several projects funded by DG CNECT were
related to Active and Healthy Ageing (AHA). They
provided: independent living and integrated services
BeyondSilos (, integrated care
coordination, patient empowerment and home support
CareWell (, set of standard
functional specifications for an ICT platform enabling the
delivery of integrated care to older patients SmartCare
( Some successful initiatives
were initiated in Europe and supported by EU, for
example, European Rosetta project [11], research network
for design of environments for ageing (GAL) [10],
assisted living environment for independent care and
health monitoring of the elderly (ENRICHME),
responsive engagement of the elderly promoting activity
and customized healthcare (REACH), digital environment
for cognitive inclusion (DECI), integrated intelligent
home environment for the provision of health, nutrition
and mobility services to the elderly (MOBISERV),
unobtrusive smart environments for independent living
(USEFIL), open architecture for accessible services
integration and standardization (OASIS) and others.
C. Unresolved Problems
These innovations can improve health outcomes,
quality of life and efficiency of care processes, while
supporting independent living. However, in the face of
new challenges some disruptive innovations should be
proposed and implemented, and the new
challenges/problems should be addressed. The potential
radically new solution should take into account the
following additional set of aspects/problems related to
quite different (1) targeted communities; (2) level of
functional (technical/computer/digital) literacy of the
targeted communities; (3) realistic time of massive
implementation of the proposed technologies for these
communities with people of various functional literacy;
(4) differences in national and geographical mentality as
to elderly care in Europe.
Targeted communities in the context of elderly care
consist of:
individuals self-directed elderly care, where
elders control both the objectives and means of
elderly care;
families, i.e. individuals inside family and/or
supported by family informal elderly care,
where elders control the means/tools, but not the
objectives of elderly care;
assisted elderly care non-formal elderly care,
where elders control the objectives but not the
means/tools of elderly care;
specialized elderly care facilities formal elderly
care, where elders have no or little control over
the objectives or means/tools of elderly care.
Level of functional/computer/digital literacy of the
targeted communities (in the order from the lowest to
highest): absolute computer illiteracy, digital phobic, basic
computer literacy, digital immigrants [12], intermediate
computer literacy, digital visitors [13], proficient
computer literacy, digital residents [13], digitally native
The proposed time of massive implementation of the
proposed technologies/environments depends on the
maturity of the available solutions and the
functional/computer/digital literacy level of the targeted
community: now (the current mature technologies can be
applied immediately), in the nearest future (the
perspective technologies can be mature in the nearest 2-3
years), in the much later future (the perspective
technologies can be mature at unknown time).
Differences in national and geographical mentality as
to elderly care in Europe were observed and reported
elsewhere [14,15]:
informal care is more common in South than in
North Europe;
informal care is more common in the "new"
member states in the "East" than in the "old"
member states in the "West";
informal care provision to someone outside the
household is comparatively rare in the
Mediterranean countries, elderly care to someone
in the home is more common in these countries
than in the EU-states on average;
the low proportion of people providing care
within households is explained by the rarity of
multigenerational households in Nordic Europe.
The proposed Augmented Coaching Ecosystem for
Non-obtrusive Adaptive Personalized Elderly Care (AGE-
Care) is focused on the provision of the virtual care,
support, and coaching to elderly people in the various
targeted communities and with different
functional/computer/digital literacy of the targeted
communities. It will be achieved by enhancement of
available ICT-enabled elderly care services, development
of new ones, and their application with the tight
coordination, monitoring, self-management and caregivers
involvement inside the proposed AGE-Care ecosystem.
A. General concept
The proposed AGE-Care ecosystem is assumed to be
based on the integration of the several new ICT
approaches and available ones, which should be enhanced
by the radically new ICT based technologies concepts
(shown in Figure 1) in favor of the elderly care
stakeholders. They include multimodal user interface
(MMUI), augmented reality (AR), machine learning
(ML), Internet of Things (IoT), Internet of Everything
(IoT), machine-to-machine (M2M) interactions, based on
the Cloud-Fog-Dew computing paradigm services,
providing a full symbiosis by integrating the whole range
from low level sensors up to high level services using
integration efficiency inherent in synergistic use of
applied technologies.
The AGE-Care ecosystem is assumed to penetrate any
organizational, national, mental, gender, and cultural
division lines, boundaries, and limits. It will use the most
appropriate available resources and elderly care,
healthcare, and social care services. The AGE-Care
ecosystem will be based on open standards, multi-vendor
interoperability, collaboration with ICT suppliers and
ICT-related service providers.
B. Main aims
The main aims of AGE-Care ecosystem are as follows:
to develop, test, and validate radically new ICT
based concept of non-obtrusive augmented reality
learning and coaching ecosystem for effective
personalized elderly care to improve and maintain
their independence, functional capacity, health
status as well as preserving their physical,
cognitive, mental and social well-being,
to develop and implement the synergetic user-
centered design of intuitive human-to-machine
(H2M) and machine-to-human (M2H) interactions
on the basis of information and communication
technologies (ICTs) including internet of things
(IoT), multimodal augmented reality (AR), and
predictive machine learning (ML) approaches,
to decrease the physical, cognitive, and mental
load on elderly care stakeholders by decreasing
the secondary human-to-human (H2H), human-to-
machine (H2M), and machine-to-human (M2H)
interactions in favor of machine-to-machine
(M2M) interactions and distributed Dew
Computing services environment,
to overcome cognitive, mental, institutional,
regional, and national barriers enabling delivery
of integrated elderly care on the European scale
by joining efforts across governmental, non-
governmental, and volunteer elderly care
organizations and individuals.
The following radically new ICT based main concepts
and approaches are planned to be used to reach these aims
(Figure 1):
multimodal user interface (MMUI) for the
more accessible and effective intuitive H2M/M2H
interaction on the basis combination of creative
"artistic" approaches;
augmented reality (AR) for non-obtrusive
H2M/M2H interactions,
machine learning (ML) for virtual decision
making and virtual guidance of users,
Internet of Things (IoT) + Internet of Everything
(IoT) + machine-to-machine (M2M) interactions
encapsulated inside Dew computing layer to
hide "behind the curtains" the mental and
cognitive overloads, and shift them from H2H to
M2M interaction zone.
C. Basic Principles
The proposed open AGE-Care ecosystem is based on
the several basic principles:
dominance of machine-to-machine (M2M)
interaction over human-to-human (H2H);
multimodal instead of single-modal interactions;
Figure 1. The integration concept of non-obtrusive augmented reality
learning and coaching ecosystem for effective personalized elderly care.
non-obtrusive augmented reality feedback instead
of obtrusive direct communication with numerous
high-tech sensors, actuators, devices, and gadgets;
virtual decision making and coaching by machine
learning instead of real human-related services,
short adaptive learning curve by selection of
specific and context-related virtual coaching
methods based on LLL principles instead of the
obsolete and awkward "user guide" and "context
help" approaches;
highly distributed service oriented local and
distance communication and service facilities.
A. Hierarhical Structure
This basic hierarchical structure of the AGE-Care
ecosystem is virtualized at different levels and visually
presented in Figure 2. In contrast to the current concept of
elderly care (Fig. 2a), the proposed concept (Fig. 2b) will
allow stakeholders:
to decrease significantly (and avoid in the most
situations) the level of H2H interactions by
emphasis on the M2M interactions for the basic
technological scenarios;
to avoid technological H2H interactions, but
emphasize emotional H2H interactions in favor of
emotional positive feedback from elderly people
due to involvement of augmented multimedia
channels like observed and even performed art,
music, dance, etc.;
to increase efficiency of H2M/M2H interactions
by introduction of multimodal communication
channels like audio, visual, tactile, odor, etc., so-
called Augmented Reality Human-to-IoT
(ARH2IoT) interactions;
to increase the acceptance level of the available
ICT technologies for elderly care by providing
their functional abilities through non-obtrusive
augmented reality pathways;
to eliminate the gap between the newest available
ICT technologies for elderly care and computer
literacy of the targeted communities by
context-related, problem-based, and personalized
virtual AR-related coaching;
to decrease the market entry threshold for the
future ICT technologies for elderly care by
providing the related open platform specifications
based on the best practices and lessons learned
during the project;
to provide more security and privacy by the
localization of the personal consumer data at the
lower scales of the AGE-Care ecosystem.
B. Workflows and Network Layers
Inside of AGE-Care ecosystem all workflows are
encapsulated in the following network layers:
Dew computing layer: the raw sensor data and
basic multimodal actuator actions are
concentrated, pre-processed, and resumed in the
smallest scale local network (Dew) at the level of
the IoT-controllers (individuals) and shared with
the upper Fog computing layer;
Fog computing layer: the resumed IoT-controller
data and advanced actuator actions are located in
the medium scale regional network unit (Fog) at
the level of the IoT-gateway (family/room/office)
and shared with the lower Dew computing layer
and upper Cloud computing layer;
Cloud computing layer: the accumulated IoT-
Figure 2. The current concept of elderly care (a, top
), and the proposed
concept of Augmented Coaching Ecosystem for Non-obtrusive
Adaptive Personalized Elderly Care (AGE-Care) (b, bottom).
gateway data are thoroughly analyzed by ML
methods to provide virtual decisions and coaching
advices in the highest scale global network
(Cloud) at the level of the global computing
centers (hospitals, healthcare authorities,
associations, corporations, etc.) and delivered to
the lower Fog and Dew Computing layer.
C. Communication Flows and Interactions
The typical communication flows inside the AGE-
Care ecosystem are schematically shown in Fig. 2b by
arrows, where the higher emphases (in contrast to the
current concept of elderly care) are placed on:
ARH2IoT interactions under Dew computing
layer: green arrows depict the main dataflows
from/to consumers by the familiar communication
channels and devices, but with context-sensitive
information provided by the multimodal
augmented reality;
M2M interactions mainly inside Dew
computing layer: light blue circle depicts the
undercover dataflows among sensors and
actuators, which are laid in the base of the
multimodal augmented reality in ARH2IoT
Cloud-Fog interactions between Cloud and
Fog computing layers: red arrows denote the
familiar dataflows between the global computing
centers and the medium scale network unit (Fog)
at the level of the IoT-gateway
Cloud-Dew interactions between Cloud and
Dew computing layers: blue arrow denotes the
dataflows between the global computing centers
and the IoT-controllers.
It will allow to decrease cognitive overload on the
stakeholders, because in the current concept of elderly
care (Fig. 2a) the stakeholders are overwhelmed by the
everyday increasing variety of the newest ICT
technologies, the related devices and unusual practices. In
the current paradigm of eHealth and elderly care, the
stakeholders have to go by the long, complicated, and
non-familiar learning curve to leverage the new ICT
technologies. In contrast to it, the AGE-Care ecosystem
proposes them to use the familiar information pathways
(devices like television and radio broadcasting, landline
phone communication), that seem to be the same old
things, but actually enhanced by newest AR and AI
technologies under the hood.
Instead of the "spaghetti connections" to the numerous
sensors, actuators, devices, and gadgets with sporadic
dataflows, "mosaic of buttons", and "puzzles of output
data" for each device/technology, etc. (Fig. 2a), the AGE-
Care ecosystem will provide the strict division in the
following dataflow channels (Fig. 2b):
consumer interaction channel by allowing
feedback data from all applied eHealth and elderly
care ICT technologies through augmented reality
pathway only at ARH2IoT layer;
machine interaction channel by integration of
all sensor/actuator technologies and isolation of
their raw data at Dew computing layer,
caregiver interaction channel by integration of
Dew, Fog, and Cloud computing layers.
In general, the AGE-Care ecosystem will decrease the
high cognitive load on customers, increase the efficiency
of caregivers, and provide a unified way for incorporation
of any future ICTs by division of dataflows into the above
mentioned consumer, machine, and caregiver channels.
This work will include the necessary formalization
procedures: standardization, definitions of customer and
stakeholder interfaces, identification of data models and
data processing tools, and privacy and security policies
and recommendations.
The necessary conditions for incorporation of the
available and future ICTs to the AGE-Care ecosystem are
mostly related with adaptation to the paradigms of:
multimodal augmented reality (AR) data output
for consumers;
Dew computing (and available M2M standards
inside it) for basic and automatic decision making;
multilayer interaction between Cloud, Fog, and
Dew computing for advanced (mostly automatic
and limited manual) decision making.
D. Some Implemented Combinations of Components
Several combinations of the new ICTs (which are
actually the components of the AGE-Care ecosystem) are
already implemented by authors and their detailed
explanation and related background can be found
elsewhere in the related publications, for example:
Frameworks for Integration of Workflows and
Distributed Computing Resources: gateway
approaches in science and education [16-18];
Dew (+ Fog + Cloud) computing + IoT + IoE:
the conceptual approach for organization of the
vertical hierarchical links between the scalable
distributed computing paradigms: Cloud
Computing, Fog Computing and Dew Computing,
which decrease the cost and improve the
performance, particularly for IoT and IoE [19];
AR + visual + tactile interaction modes: to
provide tactile metaphors in education to help
students in memorizing the learning terms by the
sense of touch in addition to the AR tools [20,21];
ML + visual + tactile interaction mode: to
produce the tactile map for people with visual
impairment and recognize text within the image
by advanced image processing and ML [22];
IoT for eHealth (wearable electronics) + ML +
AR + brain-computing interface + visual
interaction mode: to monitor, analyze, and
estimate the accumulated fatigue by various
gadgets and visualize the output data by AR
means [23-25].
The proposed integrated ecosystem provides the basis
for effective personalized elderly care by introduction of
multimodal personalized communication channels. It
allows end users to get cumulative effect from mixture of
ICTs like IoT/IoE, multimodal AR, and predictive ML
approaches. As a result, it could exclude obtrusive
H2M/M2H technological interactions by delivering them
to M2M interactions encapsulated in Dew Computing
layer, and enhancing the pleasant multimedia H2M/M2H
intuitive interactions. It hides "behind the curtains" the
mental and cognitive overloads by: shifting the most
portion of ICT-related interactions from H2H to M2M
zone; using AR pathways for delivering status information
and advices for elderly end users; increasing AR-readiness
of the available ICTs for AR-output of data for non-
obtrusive H2M/M2H interactions, and improving every-
day communication and service needs. It could be the
integral platform and paradigm for overcoming cognitive,
cultural, mental, gender/ethical, institutional, regional, and
national barriers and enabling the targeted delivery of
integrated elderly care on European and worldwide scale
by joining efforts across governmental, non-governmental,
and volunteer elderly care organizations and individuals.
In this way elimination of any kinds of “borders” between
people at European (and worldwide) scale by targeted
efforts can strengthen the relationships between the
different age categories of people and various elderly
communities despite their intrinsic or imposed differences.
The work was partially supported by Ukraine-France
Collaboration Project (Programme PHC DNIPRO)
LeAGUe project (, and Croatian
Centre of Research Excellence for Data Science and
Advanced Cooperative Systems.
[1] Vienna Institute of Demography (2016)
[2] Boll, S., Heuten, W., Meyer, E. M., & Meis, M. (2010).
Development of a multimodal reminder system for older persons
in their residential home. Informatics for health and Social Care,
35(3-4), 104-124.
[3] Lisetti, C., Nasoz, F., LeRouge, C., Ozyer, O., & Alvarez, K.
(2003). Developing multimodal intelligent affective interfaces for
tele-home health care. International Journal of Human-Computer
Studies, 59(1), 245-255.
[4] Ziefle, M., Rocker, C., & Holzinger, A. (2011). Medical
technology in smart homes: exploring the user's perspective on
privacy, intimacy and trust. In IEEE Proc. 35 Annual Computer
Software and Applications Conference Workshops (pp. 410-415).
[5] Czaja, S. J., & Lee, C. C. (2007). The impact of aging on access to
technology. Universal Access in the Information Society, 5(4),
[6] Smith, A. (2014). Older adults and technology use: Adoption is
increasing but many seniors remain isolated from digital life. Pew
Research Center (
[7] Dimitrova R. (2013). Growth in the intersection of eHealth and
active and healthy ageing. Technol Health Care; 21(2):169-72.
[8] Hernandez C, Alonso A, et al. Integrated care services: lessons
learned from the deployment of the NEXES project. Int J Integr
Care. 2015; 15.
[9] Frenk J. (2009). Reinventing primary health care: the need for
systems integration. Lancet. 374 (9684): 170-3.
[10] Haux R, Hein A, Kolb G, Kunemund H, Eichelberg M, Appell JE,
et al. (2014). Information and communication technologies for
promoting and sustaining quality of life, health and self-
sufficiency in ageing societies--outcomes of the Lower Saxony
Research Network Design of Environments for Ageing (GAL).
Inform Health Soc Care, 39(3-4):166-87.
[11] Meiland FJ, Hattink BJ, Overmars-Marx T, de Boer ME,
Jedlitschka A, Ebben PW, et al. (2014). Participation of end users
in the design of assistive technology for people with mild to severe
cognitive problems; the European Rosetta project.
IntPsychogeriatr., 26(5):769-79.
[12] Prensky, M. (2001). Digital natives, digital immigrants part 1. On
the horizon, 9(5), 1-6.
[13] White, D. S., & Le Cornu, A. (2011). Visitors and Residents: A
new typology for online engagement. First Monday, 16(9).
[14] Haberkern, K., & Szydlik, M. (2010). State care provision,
societal opinion and children's care of older parents in 11
European countries. Ageing and Society, 30(02), 299-323.
[15] Jacobs, M. T., van Groenou, M. I. B., Aartsen, M. J., & Deeg, D.
J. (2016). Diversity in older adults’ care networks: the added value
of individual beliefs and social network proximity. J.Gerontol. B,
Psychol. Sci. Soc. Sci, DOI: 10.1093/geronb/gbw012.
[16] Davidović, D., Lipić, T., & Skala, K. (2013), AdriaScience
gateway: Application specific gateway for advanced
meteorological predictions on croatian distributed computing
infrastructures. In Proc. IEEE 36th International Convention on
Information & Communication Technology Electronics &
Microelectronics (MIPRO), (pp. 217-221).
[17] Gordienko, Y., Bekenov, L., Baskova, O., Gatsenko, O.,
Zasimchuk, E., & Stirenko, S. (2015). IMP Science Gateway:
from the Portal to the Hub of Virtual Experimental Labs in e-
Science and Multiscale Courses in e-Learning. Concurrency and
Computation: Practice and Experience, 27(16), 4451-4464.
[18] Gordienko, Y., Stirenko, S., Gatsenko, O., & Bekenov, L. (2015).
Science gateway for distributed multiscale course management in
e-Science and e-LearningUse case for study and investigation of
functionalized nanomaterials. In Proc. IEEE 38th International
Convention on Information and Communication Technology,
Electronics and Microelectronics (MIPRO) pp. 178-183.
[19] Skala, K., Davidovic, D., Afgan, E., Sovic, I., & Sojat, Z. (2015).
Scalable distributed computing hierarchy: Cloud, fog and dew
computing. Open Journal of Cloud Comp. (OJCC), 2(1), 16-24.
[20] E. Artetxe González, F. Souvestre, J.R. López Benito (2016),
Augmented Reality Interface for E2LP: Assistance in Electronic
Laboratories through Augmented Reality, in Embedded
Engineering Education, Advances in Intelligent Systems and
Computing (Volume 421, Chapter 6), Springer.
[21] I. Kastelan, J.R. Lopez Benito, E. Artetxe Gonzalez, J. Piwinski,
M. Barak, M. Temerinac (2014), E2LP: A Unified Embedded
Engineering Learning Platform, Microprocessors and
Microsystems, Elsevier, Vol. 38, Issue 8, Part B, pp. 933-946.
[22] Nizar Bouhlel and Anis Rojbi (2014), New Tools for Automating
Tactile Geographic Map Translation, Proc. 16th Int. ACM
SIGACCESS Conf. on Computers & Accessibility, pp.313-314.
[23] Gordienko, N., Lodygensky, O., Fedak, G., & Gordienko, Y.
(2015). Synergy of volunteer measurements and volunteer
computing for effective data collecting, processing, simulating and
analyzing on a worldwide scale. In Proc. IEEE 38th International
Convention on Information and Communication Technology,
Electronics and Microelectronics (MIPRO) (pp. 193-198).
[24] Gordienko, N. (2016). Multi-Parametric Statistical Method for
Estimation of Accumulated Fatigue by Sensors in Ordinary
Gadgets, arXiv preprint arXiv:1605.04984.
[25] Stirenko, S., Gordienko, Yu., Shemsedinov, T., Alienin, O.,
Kochura, Yu., Gordienko, N., Rojbi, A., López Benito, J.R.,
Artetxe González, E. (2017) User-driven Intelligent Interface on
the Basis of Multimodal Augmented Reality and Brain-Computer
Interaction for People with Functional Disabilities, IEEE Ukraine
Conf. on Electrical and Computer Engineering (UKRCON-2017).
... At the same time, Gusev [22] proposed the dew computing architecture as the solution for IoT streaming devices by building processing and communication close to IoT devices. Gordienko et al. [23] presented an augmented coaching ecosystem for non-obtrusive adaptive personalized elderly care based on cloud, fog, and dew computing paradigm in the row. They have explored domains where dew computing can be adopted. ...
Full-text available
Real-time communication is a significant aspect of Internet of Things (IoT). IoT-enabled devices requires the immediate adoption of the highly distributed and heterogeneous framework of collateral merits. Moreover, cloud-based streaming services for IoT have disadvantages such as the inability to provide low latency, mobility support, location-awareness, and real-time data handling, which makes ubiquitous connectivity between the IoT device and server. At the same time, the concept of dew computing modifies the current mechanism of cloud-based services for IoT. It minimizes the response time of comprehensive data, which was collected by nearby resources. However, speedy advancement in IoT directs the evolving security aspects to address emerging challenges. To address the security issues, a mutual authentication architecture has been introduced for dew computing, which ensures secure and authorized session establishment without the requirement of a trusted server. The main objective of the proposed framework is to avoid bottleneck situations without compromising efficiency and security in real-time communication to IoT users through dew computing. To ensure the correctness of protocol, proof of security and simulation using AVISPA are presented. Analysis of performance and comparative study is also conducted to show the advantage in efficiency.
Deep learning neural networks (DNNs) were applied for recognition of historical graffiti, the letters (XI-XVIII centuries) carved on the plastered walls of St. Sophia cathedral (Kyiv, Ukraine). Recently, the graffiti dataset of these carved Glagolitic and Cyrillic letters (CGCL) was assembled for recognition and prediction by machine learning methods. In this work the dataset was updated with additional Latin, Greek, and other letters, a new structure was suggested for entire dataset and for each alphabet. The current version of the dataset includes more than 5000 images for 90 types of letters and other images as draws. Updated version of dataset was used for training and test on standards CNN networks and custom VGG-like architecture for two-letter and multi-letter classification, also was explored dependence of accuracy for classification problem on increasing numbers of trainable parameters. The new criterion was proposed new criterion for the search of the better DNN architecture that can be formulated like reaching the value of generalization (AUCLOSSY–AUCNODA) that is equal or lower than the correspondent standard deviation of AUCLOSSY and AUCNODA after data augmentation. Proposed criterion can be used as new method of optimization for custom DNN architectures, for making search the best numbers of parameters quicklier. Research result in this paper is optimization method of custom DNN architectures for Historical Graffiti Datasets, which has not enough data for standard models usage. The updated CGCL dataset was published to be available for the data science community. KeywordsDeep learningStone carving datasetConvolutional neural networkDeep learningData augmentationGeneralizationCorrespondence plot
Full-text available
Dew computing is complementing fog and cloud computing by offering the first layer of connection for any IoT device in the field. Typically, data are stored locally in the dew servers in cases when for instance Internet is not available. Therefore, dedicated authentication and key agreement protocols need to be developed in order to guarantee secure communication without the online presence of a trusted third party. First, a complete and clear presentation on the attack model and the required security features for dew computing scenarios are provided. Next, the relation with client-server security schemes is explained and two particular criteria are identified that need to be addressed in these schemes in order to serve as security scheme for dew computing. It is shown how a recently published client-server authentication scheme, satisfying these two criteria, can be extended with a key agreement feature, resulting in a very efficient authentication and key agreement scheme for a dew computing scenario. The obtained scheme outperforms from a security point of view the currently available alternatives and behaves in a similar line with respect to computational and communication efforts. More in particular, severe security vulnerabilities are demonstrated for a recently proposed dedicated dew computing authentication and key agreement protocol.
Full-text available
Smart city architecture development has become a significant research domain. Internet of Things (IoT) has become a principal element in the development of the smart paradigm. However, data security is a primary concern here as a massive amount of data collection is involved. In the field of data security, blockchain is a key technology today. In this paper, we have proposed a blockchain and dew computing-based architecture for a smart city model. The proposed model is a four-tier architecture containing physical nodes, dew, edge, and cloud. We have used the local blockchain application in the dew tier, whereas, in the cloud tier, we have used the global blockchain application. The proposed architecture reduces the latency ~ (25–30)% and power consumption ~ (35–41)% using two different blockchain applications. We have used Amazon Web Services (AWS) in our work while designing the global blockchain application. The CloudWatch monitoring tool has demonstrated the status of the global blockchain database.
Recently, the numerous deep learning (DL) approaches were successfully applied for analysis of the melanoma that is one of the most dangerous skin cancer types. This work is devoted to investigation of various DL neural networks (DNNs) applied to melanoma images to classify the skin state like in malignant or benign condition. The obtained results demonstrate the possibility to improve melanoma classification results due to application of testing time augmentation (TTA) for various DNNs with different accuracy and model size. The common tendency to increase of AUC mean values was observed for all investigated DNNs (EfficientNetB0, EfficientNetB6, DenseNet121, Mobile NetV2, InceptionV3, ResNet101, ResNet101V2, Inception ResNetV2, VGG16, and Xception). But it is unavoidably followed by decrease of inference times due to TTA-related time overheads. It can be significant especially for the fastest DNNs deployed on specialized devices based on tensor processing units (TPU) with the limited computational resources for Edge Computing layer.
Full-text available
Preventive care and telemedicine are expected to play an important role in reducing the impact of an increasingly aging global population while increasing the number of healthy years. Virtual coaching is a promising research area to support this process. This paper presents a user-centered virtual coach for older adults at home to promote active and healthy aging and independent living. It supports behavior change processes for improving on cognitive, physical, social interaction and nutrition areas using specific, measurable, achievable, relevant, and time-limited (SMART) goal plans, following the I-Change behavioral change model. Older adults select and personalize which goal plans to join from a catalog designed by domain experts. Intervention delivery adapts to user preferences and minimizes intrusiveness in the user’s daily living using a combination of a deterministic algorithm and incremental machine learning model. The home becomes an augmented reality environment, using a combination of projectors, cameras, microphones and support sensors, where common objects are used for projection and sensed. Older adults interact with this virtual coach in their home in a natural way using speech and body gestures on projected user interfaces with common objects at home. This paper presents the concept from the older adult and the caregiver perspectives. Then, it focuses on the older adult view, describing the tools and processes available to foster a positive behavior change process, including a discussion about the limitations of the current implementation.
Full-text available
The analysis of the current integration attempts of some modes and use cases of user-machine interaction is presented. The new concept of the user-driven intelligent interface is proposed on the basis of multimodal augmented reality and brain-computer interaction for various applications: in disabilities studies, education, home care, health care, etc. The several use cases of multimodal augmentation are presented. The perspectives of the better human comprehension by the immediate feedback through neurophysical channels by means of brain-computer interaction are outlined. It is shown that brain-computer interface (BCI) technology provides new strategies to overcome limits of the currently available user interfaces, especially for people with functional disabilities. The results of the previous studies of the low end consumer and open-source BCI-devices allow us to conclude that combination of machine learning (ML), multimodal interactions (visual, sound, tactile) with BCI will profit from the immediate feedback from the actual neurophysical reactions classified by ML methods. In general, BCI in combination with other modes of AR interaction can deliver much more information than these types of interaction themselves. Even in the current state the combined AR-BCI interfaces could provide the highly adaptable and personal services, especially for people with functional disabilities.
Full-text available
The paper considers the conceptual approach for organization of the vertical hierarchical links between the scalable distributed computing paradigms: Cloud Computing, Fog Computing and Dew Computing. In this paper, the Dew Computing is described and recognized as a new structural layer in the existing distributed computing hierarchy. In the existing computing hierarchy, the Dew computing is positioned as the ground level for the Cloud and Fog computing paradigms. Vertical, complementary, hierarchical division from Cloud to Dew Computing satisfies the needs of high-and low-end computing demands in everyday life and work. These new computing paradigms lower the cost and improve the performance, particularly for concepts and applications such as the Internet of Things (IoT) and the Internet of Everything (IoE). In addition, the Dew computing paradigm will require new programming models that will efficiently reduce the complexity and improve the productivity and usability of scalable distributed computing, following the principles of High-Productivity computing.
Full-text available
To identify barriers to deployment of four articulated Integrated Care Services supported by Information Technologies in three European sites. The four services covered the entire spectrum of severity of illness. The project targeted chronic patients with obstructive pulmonary disease, cardiac failure and/or type II diabetes mellitus. One health care sector in Spain (Barcelona) (n = 11.382); six municipalities in Norway (Trondheim) (n = 450); and one hospital in Greece (Athens) (n = 388). The four services were: (i) Home-based long-term maintenance of rehabilitation effects (n = 337); (ii) Enhanced Care for frail patients, n = 1340); (iii) Home Hospitalization and Early Discharge (n = 2404); and Support for remote diagnosis (forced spirometry testing) in primary care (Support) (n = 8139). Both randomized controlled trials and pragmatic study designs were combined. Two technological approaches were compared. The Model for Assessment of Telemedicine applications was adopted. The project demonstrated: (i) Sustainability of training effects over time in chronic patients with obstructive pulmonary disease (p < 0.01); (ii) Enhanced care and fewer hospitalizations in chronic respiratory patients (p < 0.05); (iii) Reduced in-hospital days for all types of patients (p < 0.001) in Home Hospitalization/Early Discharge; and (iv) Increased quality of testing (p < 0.01) for patients with respiratory symptoms in Support, with marked differences among sites. The four integrated care services showed high potential to enhance health outcomes with cost-containment. Change management, technological approach and legal issues were major factors modulating the success of the deployment. The project generated a business plan to foster service sustainability and health innovation. Deployment strategies require site-specific adaptations.
Full-text available
The paper concerns the hype idea of "Citizen Science" and the related paradigm shift: to go from the passive "volunteer computing" to other volunteer actions like "volunteer measurements" under guidance of scientists. They can be carried out by ordinary people with standard computing gadgets (smartphone, tablet, etc.) and the various standard sensors in them. Here the special attention is paid to the system of volunteer scientific measurements to study air showers caused by cosmic rays. The technical implementation is based on integration of data about registered night flashes (by radiometric software) in shielded camera chip, synchronized time and GPS-data in ordinary gadgets: to identify night "air showers" of elementary particles; to analyze the frequency and to map the distribution of "air showers" in the densely populated cities. The project currently includes the students of the National Technical University of Ukraine "KPI", which are compactly located in Kyiv city and contribute their volunteer measurements. The technology would be very effective for other applications also, especially if it will be automated (e.g., on the basis of XtremWeb or/and BOINC technologies for distributed computing) and used in some small area with many volunteers, e.g. in local communities (Corporative/Community Crowd Computing).
Full-text available
The current tendency of human learning and teaching is targeted to development and integration of digital technologies (like cloud solutions, mobile technology, learning analytics, big data, augmented reality, natural interaction technologies, etc.). Our Science Gateway ( in collaboration with High Performance Computing Center ( is aimed on the close cooperation among the main actors in learning and researching world (teachers, students, scientists, supporting personnel, volunteers, etc.) with industry and academia to propose the new frameworks and interoperability requirements for the building blocks of a digital ecosystem for learning (including informal learning) that develops and integrates the current and new tools and systems. It is the portal for management of distributed courses (workflows), tools, resources, and users, which is constructed on the basis of the Liferay framework and gUSE/WS-PGRADE technology. It is based on development of multi-level approach (as to methods/algorithms) for effective study and research through flexible selection and combination of unified modules ("gaming" with modules as with LEGO-bricks). It allows us to provide the flexible and adjustable framework with direct involvement in real-world and scientific use cases motivated by the educational aims of students and real scientific aims in labs.
The new method is proposed to monitor the level of currently accumulated fatigue and estimate it by the several statistical methods. The experimental software application was developed and used to get data from sensors (accelerometer, GPS, gyroscope, magnetometer, and camera), conducted experiments, collected data, calculated parameters of their distributions (mean, standard deviation, skewness, kurtosis), and analyzed them by statistical methods (moment analysis, cluster analysis, bootstrapping, periodogram and spectrogram analyses). The hypothesis 1 (physical activity can be estimated and classified by moment and cluster analysis) and hypothesis 2 (fatigue can be estimated by moment analysis, bootstrapping analysis, periodogram, and spectrogram) were proposed and proved. Several "fatigue metrics" were proposed: location, size, shape of clouds of points on bootstrapping plot. The most promising fatigue metrics is the distance from the "rest" state point to the "fatigue" state point (sum of 3 squared non-normal distribution of non-correlated acceleration values) on the skewness-kurtosis plot. These hypotheses were verified on several persons of various age, gender, fitness level and improved standard statistical methods in similar researches. The method can be used in practice for ordinary people in everyday situations (to estimate their fatigue, give tips about it and advice on context-related information).
Objectives: Policy reforms in long-term care require an increased share of informal caregivers in elderly care. This may be more feasible for older adults who (believe they) can organize the care themselves and have a local social network. This study describes care network types, how they vary in the share of informal caregivers, and examines associations with characteristics of community-dwelling older adults, including individual beliefs and network proximity. Method: Latent class analyses were applied to a subsample of older care receivers (N = 491) from the Longitudinal Aging Study Amsterdam, in order to identify homogeneous subgroups of people with similar care networks. Multinomial regression analysis explored associations between network type and care receiver characteristics. Results: Privately paid, coresidential, large informal, and publicly paid care network types were distinguished. Variation in informal care appeared mostly related to health, partner status, income, and proximity of children. Proximity of other potential informal caregivers did not affect the network type. Perceived control of care was highest in the privately paid network. Discussion: The results suggest that local (non-)kin could be mobilized more often in coresidential networks. Increasing informal or alternative care in publicly paid networks is less likely, due to limited social and financial resources.
This study presents an Augmented Reality Interface for engineering education. The interface, designed to use Augmented Reality to facilitate learning, is composed of both specific software and hardware elements and provides useful information and assistance in Electronic Laboratories. The document first presents the overall system and its objectives under the E2LP project. Then its components, their functioning and adaptation to educational purposes are discussed in detail. The study concludes with the approach of the scalability of the system and its future use in classrooms.
‘Science gateway’ (SG) ideology means a user-friendly intuitive interface between scientists (or scientific communities) and different software components + various distributed computing infrastructures (DCIs), where researchers can focus on their scientific goals and less on the peculiarities of software/DCI. G.V.Kurdyumov Institute for Metal Physics ‘IMP Science Gateway Portal’ ( is presented for complex workflow management and integration of distributed computing resources (like clusters, service grids, desktop grids, and clouds). It is created on the basis of Web Service – Parallel Grid Run-time and Application Development Environment (WS-PGRADE) and gUSE (grid and cloud User Support Environment) technologies, where WS-PGRADE is designed for science workflow operation and gUSE — for smooth integration of available resources for parallel and distributed computing in various heterogeneous DCIs. Some use cases (scientific workflows) are considered for molecular dynamics simulations of complex behavior of various nanostructures. The modular approach allows scientists to use SG portals as research hubs of various virtual experimental labs in the context of practical applications in material science, physics, and nanotechnologies. In addition, workflows and their components are proposed to be used as Lego-style construction units for learning modules of various scale by duration, complexity, targeted audience, and so on. These workflows can be used also in e-Learning infrastructures as constituent elements of learning hubs for the management of learning content, tools, resources, and users in the regular, vocational, lifelong, and informal learning. Copyright © 2015 John Wiley & Sons, Ltd.
The main idea behind this project is to provide a unified platform which will cover a complete process for embedded systems learning. A modular approach is considered for skills practice through supporting individualization in learning. This platform shall facilitate a novel development of universal approach in creative learning environment and knowledge management that encourage use of ICT. New learning model is challenging the education of engineers in embedded systems design through real-time experiments that stimulate curiosity with ultimate goal to support students to understand and construct their personal conceptual knowledge based on experiments. In addition to the technological approach, the use of cognitive theories on how people learn will help students to achieve a stronger and smarter adaptation of the subject. Applied methodology will be evaluated from the scientific point of view in parallel with the implementation in order to feedback results to the R&D.