Content uploaded by Silvia Ceccacci
Author content
All content in this area was uploaded by Silvia Ceccacci on Jan 11, 2018
Content may be subject to copyright.
An adaptive smart system to foster disabled and elderly
people in kitchen-related task
F. Gullà
Università Politecnica delle
Marche
Department DIISM
Via Brecce Bianche, Ancona
(Italy)
+39-71-2204880
f.gulla@univpm.it
S. Ceccacci
Università Politecnica delle
Marche
Department DIISM
Via Brecce Bianche, Ancona
(Italy)
+39-71-2204880
s.ceccacci@univpm.it
R. Menghi
Università Politecnica delle
Marche
Department DIISM
Via Brecce Bianche, Ancona
(Italy)
+39-71-2204880
r.menghi@univpm.it
M. Germani
Università Politecnica delle
Marche
Department DIISM
Via Brecce Bianche, Ancona
(Italy)
+39-71-2204880
m.germani@univpm.it
ABSTRACT
The present paper proposes a new adaptive smart kitchen
environment able to support different users with several
typologies of impairment (i.e., visual, motor, cognitive) in
performing cooking activities. This system is managed through
an adaptive user interface, which guides the user in food
preparation according to users’ capabilities and needs and permits
household appliances controlling. Results of a preliminary
usability evaluation has shown that the proposed solution is
accessible and highly usable for users with some limitations such
elderly with mild to moderate dementia and adult people with
moderate retinopathy and rheumatoid arthritis.
Keywords
Adaptive User Interface, Design for all, Human Computer
Interaction, Smart Environment
1. INTRODUCTION
Smart Environment is a significant challenge for current ICT
research: it represents one of the solutions which allow elderly
and users with impairment to remain in their home as long as
possible in a normal daily life environment. In this context, the
development of smart kitchen reveals a great interest, as
motivated by a critical number of studies [1,2,3]. One of the
strong motivation of such interest is due to the fact that,
supporting users in the main activities they perform in the kitchen
(i.e., preparing meals, using household appliances) means to
support them in performing daily life activities that most
determine their ability to live independently. In fact, these
activities cover two of the domains defined by IADLs [4]. To
sustain people independence in execution of kitchen activities, it
is necessary to provide technologies able to make the
environment universally accessible and usable. This means that
the environment should be able to support, as much as possible,
different users (i.e., normal people, disabled people, elderly,
children, etc.) in order to ensure to all of them the possibility to
achieve their goals, regardless of what are their physical or
cognitive abilities. The achievement of this object is a very big
challenge. As a matter of fact, technology is day by day the tool
to support and ensure a better liveability including the kitchen
activities, safety, comfort and well-being. In particular, it is
necessary to developed a system that allows to coordinate the
appliances and kitchen subsystems in order to optimize human-
machine and human environment interaction both from physical
and cognitive point of view. It should be able to support end-
users, providing the necessary information according to the goal
that user aims to achieve and support user’s skill in performing
such tasks. Moreover, it should be able to identify user behaviour,
detect any possible modifications and modify itself accordingly.
In the last several years, the research on smart environments
proposed new assistive systems that support and enhance the
abilities of its occupants in executing domestic tasks. Most of
these studies focused often only on services provided of smart
home and marginally consider the end user preferences and
applications. In particular, some solutions have been proposed
that aim to support the user during the meal preparation [], or
during appliances interaction [] or that aim to empower the user
about environmental energy consumption [6,7,10]. Moreover,
these studies do not address universally valid solutions: they
neglect the adaptive solutions definition focusing on solutions for
specific user profiles. To design product able to support different
users with several disabilities, it is necessary to adopt an Ability-
oriented Design perspective [WeA]. This approach encourages
the design of self-adaptive or user-adaptable system interfaces, to
provide the best possible match to user' abilities.
In this context, we propose a new adaptive smart kitchen
environment able to support normal people and well as users with
several typologies of impairment (i.e., visual, motor, cognitive) in
performing cooking activities. This smart system, in particular,
allows to manage the appliances and kitchen subsystems in order
to optimize and improve usability for different contexts of use.
The system consists of several devices (that is oven, dishwasher
and the fridge) furnished with smart functions. The network is
realized through wireless technology and it supplies
communication from/to the gateway and from/to any appliance or
device. The home automation platform is the 'core' of smart
kitchen that organizes and elaborates all the data collected by the
devices and operates events. It is a platform in a cloud service that
gives network storage service and can be accessed from logic
network applications. The system is managed through an adaptive
web-user interface, which gives information on the features and
usability of all kitchen available devices, allows to set and control
all household appliances in a simple and intuitive way and gives
information and alerts in case of the raise of uncommon situations
requiring warning, and in any how adapting itself to the end-
user’s capabilities and needs.
2. THE SYSTEM FUNCTIONALITIES
The interface system architecture is based on 4 main units,
continuously communicating between each other: the database,
the core, the interface and the doctor module. The interface
system architecture is showed in Figure 3.
A Database Management System (DBMS) is designed to achieve
a large set of structured data inputs and processes the amount of
data requested by numerous users; this unit is in charge to collect
the data arising from the different input of the system. In
particular, the information is structured in four semantic areas.
User Features Profile includes the User Personal Information and
provides the description of the user’s profile pattern, according to
its cognitive and physical structure, status and preferences. The
user’s profile pattern is outline according to the coding provided
by the International Classification of Functioning, Disability and
Health [24]. This information is collected through an on-line
analysis of the psycho-physical state of the user, according to a
medical protocol defined by specialized research institutes. The
User Features Profile it is populated by the information inserted
in the doctor module: personal data and ICF. User Use Profile
includes the User Model Context Information: previous
interaction’s history, user’s preferences and information needs.
Log Adaptation Actions represents the collection of all adaptation
actions performed by the interface. It receives the information
each time that the system performs an adaptation action: this
information is necessary to control the system’s adaptation degree
and simultaneously the user’s skills improvement in the process.
Context Data, upon which are gathered all relevant data for the
context user definition of the system. It shall record the
information derived from environmental sensors (spatial context)
associated with temporal coordinate. It will record the
information from the sensors and smart objects in the
environment.
The core Module represents overall adaptive system pivot: it is
composed of two adaptive mechanisms and a changes monitoring
system. Adaptable Engine shall make the system more suitable to
the user profile. Adaptability is based on features and preferences
that are known at a first interaction, and they are assumed to
remain static during a single session of interaction. This engine
takes as input the collected information in the User Profile
Features to adapt the graphical interface features, such as text,
size and type of font. The technical accessibility requirements and
the corresponding control points for compliance verification have
been defined on the basis of the Standards, Guidelines and
success criteria contained in the Recommendation that the World
Wide Web Consortium (W3C) - Web Accessibility Initiative
(WAI) published on 11 December 2008 and which contains the
Web Content Accessibility Guidelines 2.0 (WCAG 2.0).
Adaptive Engine shall make the system adaptive according to the
use profile. Adaptivity is based on change mechanisms which
include all dynamic features, such as interaction story based
preferences, information contents, icons, layout, etc. The working
systems take advantages of Bayesian Network adaptation
mechanism, by using software called Netica. Bayesian networks
[25] are one of the most complete and consistent knowledge tools
for the acquisition, representation and use in uncertainty
conditions, and the simulation software Netica, produced by
Norsys Software Corporation [26], is a comprehensive tool for
working with belief networks and influence diagrams. The
Adaptive Engine depends on information by User Features, User
Use and Context Data database, and apply changes on the
Interface level: Normal Interface or Wizard Interface. Change
monitoring system allows to generate two categories of alert:
User Preferences Changes takes into account the user preferences
changes, in terms of both graphics and content; User Abilities
Changes takes into account the interface changes (both graphical
and content), store them and send an alert when they frequently
change, both in a positive and negative way, going to update the
User Features Profile. The monitoring system, placed at this level,
is a tool to evaluate the psychophysical state and the disorder
progress or regression over time. This assessment will be
performed automatically by a machine. Finally, the interface
module allows enabling system interaction and communication
with user. Furthermore, the Adaptive Engine provides two
different output: the static output sets the interface type (normal
or wizard) and the initial appliances default, it is managed by
probability calculating associated at the User Use Profile finding;
the dynamic output refers to user action at the runtime. It manages
the dynamically appliances default and update the interface at
runtime. The interface is one of the most important modules of
the entire architecture; this enables the system interaction and
communication with the user.
The Interface structure can be summarized in the following two
aspects: graphic features, basic, i.e. standard features uniquely
related to a disorder (colour blindness, visual disturbances, etc.),
and advanced features that represent all dynamic features about
adapted interface items according to specific residual function
consequent to a specific disorder and they are designed on a single
user. Contents represent all interface items editable according to
user’s actions and the user acts on the interface with his own
preferences and needs. In particular, the needs related to specific
deficit, visual actions, cognitive (procedural memory) and motor,
according to user-defined profiles, it must be considered by the
features interface. In this case, the technology must be of a
"compensatory" and cooperate with the user in order to allow the
activities of daily living independently. For instance, considering
a kitchen, if the user decides to enter in the oven section, the
system will offer in evidence more used recipes.
To this end, the interface supports the following functional areas:
1) Meal preparation, 2) User interaction-appliance and 3)
Environmental Comfort. With the aim to support the end user to
set cooking program execution two different modes of
information presentation were assumed: using a common menu
(Normal Setting) and through setting driven (Step-by-step Setting
or Wizard mode) process.
Figure 3. The Interface System Architecture
The Interface structure can be summarized in the following two
aspects: graphic features, basic, i.e. standard features uniquely
related to a disorder (colour blindness, visual disturbances, etc.),
and advanced features that represent all dynamic features about
adapted interface items according to specific residual function
consequent to a specific disorder and they are designed on a single
user. Contents represent all interface items editable according to
user’s actions and the user acts on the interface with his own
preferences and needs. In particular, the needs related to specific
deficit, visual actions, cognitive (procedural memory) and motor,
according to user-defined profiles, it must be considered by the
features interface. In this case, the technology must be of a
"compensatory" and cooperate with the user in order to allow the
activities of daily living independently. For instance, considering
a kitchen, if the user decides to enter in the oven section, the
system will offer in evidence more used recipes.
Figure 4. Select Recipe menu
The Wizard mode is designed to accomplish the task and
minimizing the amount of information that the user should
understand and manage.
Consequently, this solution is suitable for users who have not
familiarity with technology and / or have some cognitive
dysfunction. On the other side, the Normal mode, is designed to
support user without cognitive dysfunction and characterized by
a good technology attitude. Consequently, on first use, the setting
menu will be presented in a “Wizard mode” to person with
cognitive dysfunction, while in "normal" mode for other profile.
Figure 4 show an example of recipe setting menu.
As one can observe, during interaction with the interface, the user
can change the information presentation mode by tapping on the
proper button. In detail, when the probability of default mode
information exceeds a certain threshold, the information mode
presented by default for that specific user is changed. Depending
on user preferences, acquired in the profile phase acquisition, the
system will provide a specific interactions mode. At any time, the
user may change the interaction mode, changing preferences to
his profile.
3. EXPERIMENTAL RESULTS
A preliminary usability evaluation has been carried out to assess
system in terms of usability for each user target category. This
first evaluation aimed to measure usability perceived by users
during the first use of smart system: each user is asked to use the
interface optimized, depending on his/her profile, according to
default specification. Therefore, normal interface was presented
to user with visual and dexterity problem, respectively adapted to
maximize accessibility, while users with cognitive disability
interacted with interface in wizard modality.
To recruit people, three users’ profiles are considered, according
to defined Personas:
- Profile 1: aged between 45 and 54 years, with moderate
visual impairment (retinopathy) and good attitude and
skill with technology;
- Profile 2: adults aged between 55 and 65 years with
moderate loss of dexterity (rheumatoid arthritis) and low
attitude and skill with technology;
- Profile 3: adults over 70 years with mild to moderate
cognitive impairment (dementia) which affect procedural
memory and scarce skill with technology.
Five users per profile have been involved in the tests.
The usability lab was equipped with the interconnected smart
appliances (i.e., oven, a dishwasher, a refrigerator), a table, a chair
and a tablet. Ingredients were in the refrigerator or on the
worktop, as well as the necessary tools, pots and dishes.
Usability tests have been based on a task analysis. Users were
asked to simulate the preparation of a baked omelette for two
people, and in particular to perform four tasks: 1) find the recipe
2) retrieve the necessary ingredients and simulate the preparation,
3) programming and start oven and 4) fill, set and start the
dishwasher.
To support data collection both direct observation and Video
Interaction Analysis (VIA) have been used. The following
objective parameters have been collected for each task: %
Completion without support (%C), Number of errors made by
user during the interaction and (E) and Number of support
requests (S).
Before starting test session, the interface has been shown to the
users and the various system functions were explained. At the end
of the session, users ware asked to answer to the SUS
questionnaire [25].
Table 1. Usability test results
By analysing the results, we can observe that the propose smart
interface can be considered very usable for all considered user
profile. With respect to the Profile 1 and 3, there were no
significant usability issues: the smart interface in normal modality
has proven to be suitable for people with vision problems and to
able to provide them support during preparation of meals
activities, household appliances setting. At the same time, smart
interface in wizard modality can effectively support people with
problems in procedural memory. For this category, only a minor
usability problem was observed during the execution of task 1: a
user at the beginning found difficult perform tapping in a correct
manner. However, it must point out that a total of 5 subjects (i.e.,
2 belonging to Profile 2 and 3 to Profile 3) had never used a tablet
before. Regard to Profile 2, we can observe that 2 users of 5 found
difficult to set the oven with interface in normal modality. This
can be due to the high number of operations that the user must
perform to set the oven as well as the low users’ technological
attitude and skill. One possible solution is to provide the interface
in wizard modality even for those users with poor skills in the use
of technologies.
4. CONCLUSIONS AND FUTURE
WORKS
This paper presents the design of an innovative home automation
system which aims to support users with several impairments
(visual, cognitive, motor related) in performing cooking activities
through a smart adaptive interface.
The results of a qualitative experimentation with final users has
shown that the proposed interface is accessible and highly usable
for users with some limitations such elderly with mild to moderate
dementia and adult persona with moderate retinopathy and
rheumatoid arthritis. In general, it emerged that the new smart
kitchen system is able to improve the usability of household
appliances, such as oven and dishwasher, as regards the
programming and controlling operations. At the same time, it
shows how the introduction of smart technologies in the kitchen
environment can support users also with scarce attitude to
technology.
The work is not definitive: the new smart kitchen has to be
evaluated with a greater number of users in order to verify this
first qualitative evaluation. Moreover, it will be necessary to carry
out a field study to assess the ability of adaptable features of
interface to improve the system usability, as to assess the
effectiveness of system to enhance the users' skills related to
cooking activities and to improve users’ independent living. An
additional effort must be focused on the assessment of
effectiveness of such system to support a doctor in monitoring
patients’ cognitive status.
5. ACKNOWLEDGMENTS
This work has been developed in the context of “D4All: Design
for all” project, National Technological Cluster funded by the
Italian Minister of University and Research.
6. REFERENCES
1. Hashimoto, Atsushi, et al. "Smart kitchen: A user
centric cooking support system." Proceedings of
IPMU. Vol. 8. 2008
2. Chi, Pei-yu, et al. "Enabling nutrition-aware cooking
in a smart kitchen." CHI'07 extended abstracts on
Human factors in computing systems. ACM, 2007.
3. Blasco, Rubén, et al. "A smart kitchen for ambient
assisted living." Sensors 14.1 (2014): 1629-1653
4. Katz, Sidney. "Assessing self‐maintenance: activities
of daily living, mobility, and instrumental activities of
daily living." Journal of the American Geriatrics
Society 31.12 (1983): 721-727.
5. Weiser, M., (1999). The computer for the 21st
century. SIGMOBILE Mobile Computing and
Communications Review 3 (3), 3–11
6. Cook, Diane; Das, Sajal (2005). Smart Environments:
Technology, Protocols and Applications. Wiley-
Interscience
7. Xinguo, Y., Bin, X., Weimin, H., Boonfong, C.,
Junfeng, D., (2007). A framework of context-aware
object recognition for smart home. In: Proceedings of
the 5th International Conference on Smart Homes and
Health Telematics, pp. 9–19.
8. Joon, L.S., Ilseok, K., Wook, K.M., (2007). A user
interface for controlling information appliances in
smart homes. In: Nguyen, N.T., et al. (Eds.), Agent
and Multi-Agent Systems: Technologies and
Applications, vol. 4496/2007, pp. 875–883 (Chapter
92).
9. Frey, J.; Neßelrath, R.; Stahl, C. An Open
Standardized Platform for Dual Reality Applications.
International Conference on Intelligent User Interfaces
(IUI), Palo Alto, CA, USA, 13–16 February 2011.
10. M. Ficocelli and G. Nejat, “The design of an
interactive assistive kitchen system”, Assistive
Technology Volume 24, Issue 4, pp. 246-258, 2012.
11. J. H. Chen, P. P. Y. Chi, H. H. Chu, C. C. H. Chen,
and P. Huang, “A smart kitchen for nutrition-aware
cooking”, IEEE Pervasive Computing Volume 9, Issue
4, December 2010, pp 58-65, doi:
10.1109/MPRV.2010.75.
12. Elaine Ostroff. (2011). Universal Design: An Evolving
Paradigm. Universal Design Handbook. W. F. E.
Preiser, K. H. Smith, Chapter1
13. Mace, R. (1985). “Universal Design, Barrier Free
Environments for Everyone,” Designers West,
November.
14. Wobbrock, J. et al., 2011. Ability-based design:
Concept, principles and examples. ACM
Transactions on Accessible Computing
(TACCESS), 3(3), pp. 1-36.
