Key Practices for Welfare Robots provision: Assessment Framework and Participation

Full text

Abstract—As aging population grows, robots used in welfare and
healthcare domain such as social robots, assistive robots and
service robots (hereafter Welfare Robots, WR) has been
attracted more attentions as one of the solutions to the welfare
resource shortages. However, at present, WR have not
necessarily implemented or embedded widely in society. Reasons
for hindering WR provision to our society include the absence of
clear assessment criteria and social implementation process
models. While some care institutions and private homes
promoted and integrated WR successfully, large part of our
society have struggled to identify ways of appropriating the WR
provision. Based on the investigation on seven successful
organizations on evaluation and implementation practices, the
research extracted key aspects for WR provision in society. The
extracted three key categories with seven aspects highlights
importance of standard assessment framework which includes
stakeholder involvements and concepts of care recipients’
autonomy. These critical aspects of WR provision contribute to
better WR design, development and implementation in society
under the new normal.
I. INTRODUCTION
As world aging population grows with rapid pace, in
healthcare and welfare domain, the increase number of care
recipients and the associated labor shortage have become
societal problems [1]. To solve the labor shortage, the
possibility of introducing welfare robots (WR) has attracted
attention [2]–[4]. The active utilization of WR has huge
potential to reduce unnecessary contact with caregivers while
maintaining and increasing the care quality. Considering the
emerging social condition caused by COVID-19, WR
utilization under the new normal will increase its importance
further in our society.
In our society, wide varieties of robots have been
developed, ranging from high-tech robots such as
entertainment robots equipped with sensors [5], [6], social and
communication robots [7], [8], [9], sensory robots [10], [11],
therapy robots [6], [12], [13], mobility robots [14], cleaning
robots, to simple functional robots such as dietary service and
smart toilets. They support physical, cognitive, medical and
psychological challenges. WR has been attracted more
attentions than ever as technologies for improving quality of
life (QoL) of care receivers and reducing the burden of
caregivers [2].
Mika Yasuoka is with Sustainable Digitalization, Roskilde University,
Universitetsvej 1 DK-4000 Roskilde, Denmark. (corresponding author to
provide phone: +45 46 74 28 89; e-mail: mikaj@ ruc.dk).
Yasuko Akutsu is with MT Health Care Research Inc., 1-17-6
Uchikanda,Chiyoda-ku,Tokyo, Japan. (e-mail: yasukoakutsu@hcdr.co.jp).
In spite of the urgent societal needs and high expectations,
WR have not widely been implemented in society on a full
scale [3]. For example, in Japan, where many WR have been
suggested and developed, the WR research and development
has not always linked to the implementation to the society,
according to the research conducted under the Ministry of
Health, Labour and Welfare of Japan [15]. There are various
reasons for hindering WR provision to our society. From
healthcare institutional point of view, some of the biggest
reasons are, a lack of guidance how to select appropriate WR
from extensive choices and a lack of understandings to care
receivers’ and caregivers’ needs [3]. From care receivers’ and
caregivers’ point of view, limited understandings on WR
possibilities and potentials hinder WR use. In addition, many
caregivers as well as receivers feel psychological resistance to
be taken care by robots [4], [16], [17]. Caregivers often feel
more comfortable to make their work done without any
complicated technological help [15], [18]. In addition, a lack
of acknowledged assessment frameworks, which cover all
contingencies and implementation process of WR make it
difficult to make appropriate choices among all possibilities
[19]. At present, although vast range of technology assessment
framework [20]–[24] and check lists [25] are suggested, for
verifying the basic guideline of safety, effectiveness and
usefulness, Baudin and Sundstrom’s literature review [23] on
Assistive and Welfare Technology concluded “in spite of a
growing interests, evidence-based decision models and other
guidance are lacking”. It is also important to note that even
after the appropriate local choice of WR, institutional WR
implementation process is still a challenge as new device
implementations create a lot of concern among care receivers,
caregivers and managers in healthcare institutions who are not
robot specialists.
In order to promote WR provision in society, this paper
aims at exploring practices of WR provision and extracting
key essence. More specifically, this research target at
clarifying assessment practices in choosing, and processing
WR in recipient daily life, based on interviews and
accompanied field investigations. The research chose
Denmark as a preliminarily target since WR has widely
prevailed in Danish healthcare and welfare domain since 2006
[5], [6], involving municipalities (primary welfare service
provider), care institutions, caregivers, and care recipients.
The survey in 2019 reported that surprising number of
healthcare technology cases (1,400) were implemented in the
Keiko Homma and Yoshio Matsumoto are with The National Institute of
Advanced Industrial Science and Technology (AIST), Kashiwa ll Campus,
University of Tokyo, 6-2-3-, Kashiwanoha, Kashiwa, Chiba, 277-0882,
Japan (e-mail: keiko.homma@aist.go.jp, yoshio.matsumoto@aist.go.jp).
Key Practices for Welfare Robots provision: Assessment Framework
and Participation
Mika Yasuoka1, Keiko Homma, Yasuko Akutsu, and Yoshio Matsumoto

year over all 98 municipalities in Denmark [5], with high
satisfaction rates both among recipients and caregivers [6].
The structure of this paper is as follows. First, Section 2
describes our research design. Section 3 and Section 4 show
comparative results and analysis respectively, and discussion
follows in Section 5. The paper concludes and expresses final
remarks in Section 6.
II. METHODS
The purpose of this study is to clarify how WR has been
chosen, assessed and implemented at home and in care
institutions. As a research method, we conducted in-depth
interviews [26] with those who are responsible in choosing,
testing and implementing WR.
A. Cases
The selected organizations are those with a strong record
on implementing many WR over multiple health care
institutions. First, as primary care providers, three
municipalities at different size among 98 organizations were
chosen. Next, two health care centers were chosen, which
closely work with local municipalities to provide service of
WR provision and technology support programs in each
municipality. In addition, one research institution, which
specializes welfare technology, and has in depth experiences
in evaluating numerous WR implementation projects. Lastly
an independent consultant firm, specializing WR
implementations at homes and care institutions globally was
chosen. The total seven organizations are our investigation
subjects, which is summarized in Table l with organizations’
name, interview style and interview duration.
TABLE I. INTERVIEW SUBJECTS
ID
Interview Subjects
Name
Style
Duration
A
Municipality A (large, 0.6
million population)
Face to Face
2 hours
B
Municipality B (middle,
0.2 population)
Face to Face
2 hours
C
Municipality C (small, 0.1
million population)
Online*
1.5 hours
D
Healthcare Center D
Online*
1.5 hours
E
Healthcare Center E
Online*
1 hour
F
University F
Face to Face
2 hours
G
Consulting Firm G
Face to Face
2 hours
* For, online interview, telecommunication tool, Zoom, was used.
B. Research Design
For the interviews, main three questions (See C. Main
Three Questions) were prepared. The interviews were
conducted online or face-to-face for one to two hours in spring
2020. While conducting interviews, interviewees were asked
to introduce WR implementation cases, together with
evaluation tools and methods if any. The interviews were
recorded using a voice recorder or Zoom’s recording function
after interviewees’ consents. After the interviews, evaluation
tools and digital documents introduced in the interviews were
collected or got a permission to copy and the interview data
were transcribed and analyzed.
C. Main Three Questions
Questions of the in-depth interview are roughly divided
into three categories. The first is about WR assessment criteria
and process. The questions in this category are for example,
"From what kind of viewpoints, do you choose and assess
WR?" The second category is about methods and tools. The
questions in this category are for example, "Do you have any
assessment methods, processes, or tools used in your
organization?" The third is a question to investigate purposes
of introducing WR. For example, the questions in this category
are, "What is WR used for? Why did you introduce A name of
WR used in the organization?" This last question is both for
understanding purpose of WR in the institutions, and also for
examining consistency with a choice of assessment methods.
III. RESULTS
The summary of the interviews with the seven
organizations is shown in Table ll.
TABLE II. INTERVIEW RESULTS
Answers to Three Questions
Assessment
Method
Assessment Tool
Purpose of Assessment
A
Original, VT-CV
Original Tool,
Tech Match
Living lab
Help autonomy of citizens
Reduce workload of
caregivers
B
n/a
Living lab
Help citizens independent
living Improve citizens’ QoL
Improve caregivers’ work
environment
Reduce costs
Assure stability and
reliability of WR
C
Original,
Springboard
Workshop
Help citizens, help hospitals
and help caregivers
D
n/a
MAST criteria
Check MAST seven criteria*
Evaluate patient acceptance
E
Original, PPD
Workshop,
Living lab
Value for citizens, value for
caregivers (reduce work
hours, etc.)
Economic benefits
Quality control of care
service
F
MAST
Workshop,
Living lab
Apply MAST process and
seven criteria*
G
Original ,
Innovation
Methodology
Workshop,
Living lab
Check predefined criteria in
three living labs.
* Health problem, Safety, effectiveness, patient perspectives, economic aspects organizational
aspects and socio-cultural aspects [20]
Six out of seven organizations have standard evaluation
methods. Among them, University F and Healthcare center D
use the MAST [20], which was collaboratively suggested by
EU researchers to evaluate telemedicine technologies while
other four organizations used their original assessment
method. Municipality B has no specific assessment methods,
but instead, it used co-design [27] and user involvement
methods such as workshops and living labs [28]–[30] to
evaluate WR together with care recipients and caregivers.
Such workshops and living lab methods hold unique
frameworks to evaluate WR with a specific process and
assessment criteria. For example, they involve varied related

people for long period. For instance, living labs conducted in
Municipality B often last for 3 months up to two years, inviting
citizens, their families and caregivers in addition to WR
developers and organizations during the period.
A. Characteristics of Original Methods
The characteristics and features of each assessment method
are different but has some commonalities. We review MAST
[6] used in the EU nations for telemedicine assessment and
other five original methods in this section.
1) MAST (Model for Assessment of telemedicine)
• A guideline with seven perspectives called “domains”
and assessment process for decision-makers such as
hospitals, healthcare-related organizations, and local
governments
• A standard for telemedicine assessment in Europe
• Developed to evaluate telemedicine applications by
EU researchers in 2010 as a common model, based on
EUnet HTA core model [24], the technical evaluation
on medicine
2) VT-CV (Curriculum vitae of welfare technology)
• A method for choosing the best technology by
tripartite key participants; public officers, caregivers,
and service recipients
• A collective information tool with a list of
technologies, the names of the service recipients, and
their history of previous technology usage
• A preliminary step for living lab for proof-of-concept
• Created by Municipality A based on two years
accumulated experiences on WR provision
3) Springboard
• A technology assessment method to support
extracting diverse ideas and viewpoints on relevant
WR from 360-degree perspectives with diverse
participants
• A workshop framework, consisting of preliminary
selection of two or three WR, a list of questions, a
selection of participants, a main springboard session
and a report with decisions
• Proposed by Municipality C
4) PPD (Production Presentation Day)
• A method for choosing technologies for field testing
through assertive inputs from WR companies
• A preliminary step for living lab project
• A workshop framework for comparative assessment
by multiple participants including developers,
municipality officials, nurses, medical doctors,
citizens and their families
• Developed by Healthcare center D
5) Innovation Methodology
• A systematic method for evaluating WR, assessing
effects on care process and targeted organization, and
supporting a deployment process
• A set of three living labs (Core Lab, Trusted User Lab,
and Scale Lab) with diverse assessors’ involvements
• Developed by Consulting G
B. Characteristics of Original Tools
1) Tech Match: A board game-style assessment tool
• A tool to support decision making of tech choice
through playing a board game with stakeholders
• One of the design games [31], [32] which explore
possibilities and needs among players through a
process of playing board games
• Created by Municipality A in collaboration with a
university
IV. ANALYSIS
Based on the data collected from interviews, WR
assessment characteristics, methods and tools used in practice
were analyzed, using KJ method [33]. The analysis focused
on assessment criteria, purpose and expected results. A total
of 283 extracted key aspects were labeled, categorized, and
relationships among categories were synthesized. In the end,
eight features with three categories were depicted as
characteristics of WR assessment criteria used in the field as
shown in Table lll.
TABLE III. KEY CHARACTERISTICS IN WR ASSESSMENT
A. Coherent Assessment Criteria
1
Assessment from three key stakes; users, caregivers and
organizations
2
Comparative assessment
3
Localization
B. Stakeholder Involvement
4
Ensuring diversity
5
Understanding local needs
6
Transforming mindset
C. Long term commitment
7
Integrating in daily life
8
Promoting care recipients’ independence and autonomy
A. Coherent Assessment Criteria
Category A "Coherent assessment criteria" indicates that
all organizations value official or own assessment method.
1) 1. Value on diversity and three key stakes
In every organization, assessment viewpoints are roughly
divided into three aspects: user, caregiver, and organizational.
From an organizational point of view, cost effectiveness of
introducing WR is a key. From caregiver's point of view,
reduction of workloads both physical loads and time saving
are important. In addition, influence on existing work
processes in the course of WR implementation are also widely
considered. From user's point of view, improving QoL and

ensuring privacy are important aspects.
2) 2. Comparative assessment
In organization A, B, C, E, F and G, comparative
evaluations among various WR were confirmed. Several
assessment methods hold comparative examinations in their
frameworks to evaluate multiple viewpoints (Municipality A.
Hereafter, only organization’s ID will be shown) and their
assessments were conducted by inviting multiple WR
developers in one session (E).
3) 3. Localization
In conducting assessments, two main trends were observed.
One is to use the existing assessment method, MAST, and the
other is to create unique assessment methods to fit to their
local condition. In both trends, localization was of necessity.
Let’s consider MAST first. MAST is regarded as a reliable
and applicable assessment method for WR due to its higher
explainability and validity. However, majority of
interviewees avoid practicing it directly in the daily assessing
task on WR due to its complexity. Two organizations,
Healthcare Center D and University F, utilized MAST, and
among them, Healthcare Center D modified its assessment
framework to meet the local needs as MAST was complex
and difficult to use.
The tools such as evaluation templates (E), a simple 5-
grade evaluation sheet for caregivers’ assessment (A) and the
design games (A) were very simple to use so that they fit to
the local work process. It is so simple that caregivers can
assess WR in the process of daily care routine. There are also
a set of templates in planning, process, summary and report,
such as those used in the three living labs (G). These tools are
created as predefined templates and integrated process of care
so that assessors with diverse work cultural backgrounds can
utilize easily, accomplishing consistent assessments.
B. Stakeholder Involvement
Category B “Stakeholder involvement” indicates that
stakeholder participation in assessment and its
implementation process is widely practiced.
1) 4. Ensuring diversity
For ensuring high acceptance, assessments have to be
conducted from multiple perspectives. The most basic as well
as crucial parts are diverse participations of care recipients
(sometimes their families, significant others and relatives),
formal caregivers, care institutions, ergonomics and business
experts (C) and medical doctors, nurses and hospital mangers
(G) in the assessment. In assessing, not only on WR itself, but
also on living lab and workshops where service process and a
whole concept around WR are tested, many organizations
assumed that various stakeholder should be involved. For
example, in WR assessment workshops (D), citizens,
caregivers and nurses as well as an engineering company and
potential buyers (In Denmark, typically municipalities),
analyze and evaluate the targeted WR, based on certain criteria.
2) 5. Understanding local need
Care institutions recognize it important to involve people
in practice in order to understand real challenges at care
institutions. Sometimes it is difficult to involve direct users or
busy medical doctors, then organizations consider inviting
related people. For example, if the targeted person is suffering
from dementia or mental illness, s/he may be difficult to
express her/his opinion. Instead, informal caregivers such as
family members, close relatives or friends, may act as an
interpreter who translate the care recipients’ wish by
sympathizing their needs (A).
3) 6. Transforming organizational mindset
It is essential for all parties to understand each
stakeholder’s point of view to prepare for mindset changes. To
carry out an appropriate assessment of WR, firstly current
maturity and WR acceptant level of the organization have to
be considered. Secondly, potential long-term impacts and
changes in person, care process, organization should be
estimated. It is difficult for a single person to understand and
imagine such potential organizational changes by mere
guessing work or through documents. Thus, when introducing
WR to care institutions, it is indispensable to derive issues and
to respect organizational culture (F) by directly involving
stakeholders (C). Since it is usually a big investment on time,
resource and money to deploy WR in care institutions and
homes, it is necessary for all stakeholders to recognize
challenges, seek for potential best solutions and commit to the
chosen solution (G).
C. Long-term commitment
Category C "Long-term commitment" indicates that the
assessment is always made from a long-term perspective,
considering changes of people’s mindset, organizations, and
environment.
1) 7. Integrating in daily life
In our investigation, a long-term perspective was
integrated in the WR assessments of the seven organizations.
The long-term perspective means an assessment of WR
deployment should cover a whole process from WR
introductory to incorporation into citizens’ daily life. The
incorporation happens through gradual technology acceptance
at personal and organizational level rather than mere WR
installments. The long-term transformation, initiated by an
introduction of WR, progresses and alters people’s mindset
gradually and local care process over months and years (A). In
many cases, people’s mindset changes over time. One of the
living lab case (F) shows that a few care facility residents with
dementia who did not respond to communication robots at first
began to respond almost one month later. This indicates the
long-term perspective is important to understand the impacts,
while short-term judgment should be avoided when it comes
to human and organization involved.
Long-term perspective also become a key for scaling WR
provision in society. Even if one organization succeeded an
introduction of WR, its implementation process and approach
of the particular robot are not necessarily scalable in other
places due to differences in organizational culture (A). Thus,
the scalability should be well considered potentially with best
practice sharing and long-term training programs together with
organizational transformation. It is important to note that
steadily and gradual localization is required (E).
2) 8. Promoting citizens’ Independence and autonomy
Often, organizations explain an importance of technology
assessment as for decision making for budget allocation (A-G).
This is true but not everything. From our cases, another

essential purpose was also extracted. It is for long-term
autonomy of care recipients in daily activities. Municipality A
claimed an importance of independence and autonomy of care
recipients, answering the purpose of WR provision, as follows.
“Consider a person who has difficulty in walking. S/he will
be able to walk again by using an auxiliary cane. WR is not
just a welfare tool, but a complemental technology expanding
people’s ability. When people with declined physical ability
regain it by using technology, so the body may be active again,
we consider WR as technology for promoting independent
living. Consider a smart phone application, which enlarges
characters of newspaper, or robot vacuum cleaner, which tidy
up your room. They are clearly devices that can became a part
of daily life and be used forever to help their daily life. With
the help of integrated technology, they can accomplish not
only independent but autonomous life again. The
independence and autonomy are nature of WR we value on”
(Municipality A)
"Autonomy" in this context means to take an action based
on your own will such as WANT to clean, WANT to walk to
shops (A), while independence means of doing without others’
help. The difference of autonomy and independence is shown
in its external and internal motivational causes and they can be
considered as a key to WR design. From this stance, WR
assessment should include aspects to encourage a person to
become those who want to be through WR.
V. DISCUSSION
Every organization in our cases has their WR assessment
framework used widely within the organization. Municipality
A has spent two years building up their original assessment
method in participatory processes and currently have tried to
scale the method from one organizational unit to another.
Similarly, in other organizations, methods or assessment tools
were created internally and independently or based on the
existing method. Although there is still no unified standard or
officially recognized assessment method for WR in practice,
the organizations recognize the importance of coherent
assessment criteria to choose, evaluate, implement and
integrate WR to care receivers’ daily life, thus defined their
standard. Our research indicates the successful provision of
WR in our case organizations is deeply indebted to the locally
appropriated coherent assessment framework.
The term of “Assessment of WR” might give an impression
of evaluation on technical safety and functional maturity.
However, interestingly, in our cases where WR has widely
implemented and integrated to recipients’ daily life, the key
evaluation domains are organizational, humans’ perspective
and need-based assessments rather than technical aspects.
Engineering safety and the maturity level as a device such as
obtaining the CE mark was one of the implicit prerequisites
and hardly mentioned in our study. The socio-technical
domains are deeply woven into the assessment framework.
Considering the importance of human and organizational
aspects such as potential improvement of QoL, impact of
work process changes, and cost effectiveness to investment, it
is not surprising that participation is emphasized and itegrated
in the depicted assessment characteristics, which is also
evident in the Health Technology Assessment international
reports [34].
Furthermore, our investigation shows the importance of
long-term perspectives in introduction, implementation and
deployment of WR, to properly respond to personal,
contextual and organizational changes. Through involvement
of stakeholders, not only satisfying the current local demands,
but also encouraging users consciously to increase their
technology acceptance in the long-term, and eventually to
acquire their life-time autonomy.
VI. FUTURE WORK
The data of this study was derived through interviewing
seven Danish healthcare organizations which actively use and
integrate WR to care recipient daily life. However, this
qualitative data will hardly generalize overall tendency of WR
usage due to its limited samples. Therefore, it is necessary to
further investigate and analyze whether the views derived
from this investigation can guarantee its validity widely. At
the same time, this study revealed the urgent need of an
assessment method and process for encouraging further
provision of WR in society. To establish a widely applicable
assessment method could be one of the important future
directions of this research. As a next step, authors consider
constructing a standardized WR assessment model and
evaluate its validity. It contributes filling the gap between WR
research & development and needs in care practice.
In addition, authors would like to consider and integrate
the aspect of autonomous living through the introduction of
WR as it will become an increasingly important perspective
in the future. A pre-Corona report published in 2019 analyzed
that the introduction of WR in seniors’ private houses to
support remote communication increased both users’ and
caregivers’ satisfaction [5]. It was explained that WR
improved user’s freedom and dignity as adult citizens by
providing a choice to refusing spontaneous caregivers’ visits
while ensuring safety. Potentially, WR could protect privacy,
independence, and furthermore autonomy. In the welfare
domain, care model has been shifting from taking care of
citizens to supporting recipients’ autonomous living during
last decades [3]. This tendency has been accelerated due to
the COVID19 epidemic, which require us reducing
unnecessary physical contacts. It is our urgent task to improve
QoL and expand supports under the new social situation. The
strong autonomy fostered through the use of WR indicates an
emerging opportunity to amplify care quality with meeting
social demands.
REFERENCES
[1] United Nations Department of Economic and Social Affairs, “World
Population Prospects 2019,” 2019.
[2] The Japan Research Insitute, “Digital Transformation on Healthcare
Service,” 2021. (In Japanes)
[3] H. Robinson, B. MacDonald, and E. Broadbent, “The Role of
Healthcare Robots for Older People at Home: A Review,” Int. J. Soc.
Robot., vol. 6, no. 4, pp. 575–591, 2014, doi: 10.1007/s12369-014-
0242-2.
[4] E. Broadbent, R. Stafford, and B. MacDonald, “Acceptance of

healthcare robots for the older population: Review and future
directions,” Int. J. Soc. Robot., vol. 1, no. 4, pp. 319–330, 2009, doi:
10.1007/s12369-009-0030-6.
[5] Local Government Denmark (KL), “Velfaerdsteknologioversigt,”
2019. Accessed: Oct. 12, 2020. [Online, in Danish]. Available:
https://www.kl.dk/media/20301/velfaerdsteknologioversigt.pdf.
[6] S. T. Hansen, H. J. Andersen, and T. Bak, “Practical evaluation of
robots for elderly in Denmark - An overview,” 5th ACM/IEEE Int.
Conf. Human-Robot Interact. HRI 2010, pp. 149–150, 2010, doi:
10.1145/1734454.1734517.
[7] R. Menicatti et al., “Collaborative Development Within a Social
Robotic, Multi-Disciplinary Effort: The CARESSES Case Study,”
Proc. IEEE Work. Adv. Robot. its Soc. Impacts, ARSO, vol. 2018-
Septe, no. February 2019, pp. 117–124, 2019, doi:
10.1109/ARSO.2018.8625740.
[8] H. D. Bui and N. Y. Chong, “An Integrated Approach to Human-
Robot-Smart Environment Interaction Interface for Ambient Assisted
Living,” in Proceedings of IEEE Workshop on Advanced Robotics
and its Social Impacts, ARSO, 2019, doi:
10.1109/ARSO.2018.8625821.
[9] M. Valentí Soler et al., “Social robots in advanced dementia,” Front.
Aging Neurosci., 2015, doi: 10.3389/fnagi.2015.00133.
[10] K. Ogawa, S. Nishio, K. Koda, G. Balistreri, T. Watanabe, and H.
Ishiguro, “Exploring the natural reaction of young and aged person
with Telenoid in a real world,” J. Adv. Comput. Intell. Intell.
Informatics, vol. 15, no. 5, pp. 592–597, 2011, doi:
10.20965/jaciii.2011.p0592.
[11] K. Ogawa, S. Nishio, K. Koda, K. Taura, T. Minato, and C. T. Ishii,
“Telenoid: Tele-presence android for communication,” ACM
SIGGRAPH 2011 Emerg. Technol. SIGGRAPH’11, p. 4503, 2011,
doi: 10.1145/2048259.2048274.
[12] R. Gregersen, J. B. Sylvestersen, and A. Hald, “Interactive
Technologies supporting Cognition in People with Dementia -
preliminary results,” in Poster session presented at 30th Alzheimer
Europe Conference, 2020.
[13] E. Mordoch, A. Osterreicher, L. Guse, K. Roger, and G. Thompson,
“Use of social commitment robots in the care of elderly people with
dementia: A literature review,” Maturitas, vol. 74, no. 1, pp. 14–20,
2013, doi: 10.1016/j.maturitas.2012.10.015.
[14] Z. Guo and H. Yu, “Development of a novel robotic omni-directional
hospital bed mover for patient transfer,” in Proceedings of IEEE
Workshop on Advanced Robotics and its Social Impacts, ARSO,
2016, doi: 10.1109/ARSO.2016.7736253.
[15] Mitsubishi Research Institute Inc., “How to Effectively Use Care
Robots? (In Japanese),” 2019. [Online]. Available:
https://www.mri.co.jp/knowledge/pjt_related/roujinhoken/dia6ou000
00204mw-att/H30_100_2_report.pdf.
[16] S. T. M. Peek et al., “Older Adults’ Reasons for Using Technology
while Aging in Place,” Gerontology, vol. 62, no. 2, pp. 226–237,
2016, doi: 10.1159/000430949.
[17] M. Heerink, B. Kröse, V. Evers, and B. Wielinga, “Assessing
acceptance of assistive social agent technology by older adults: The
almere model,” Int. J. Soc. Robot., 2010, doi: 10.1007/s12369-010-
0068-5.
[18] Techno Aids, “Research on examination of effective utilization of
Care Robots,” Tokyo, 2016.
[19] K. Homma and Y. Matsumoto, “Evaluation Framework of Robotic
Care Devices at the Introduction Phase,” in Proc. of IEEE
International Conference on Intelligence and Safety for Robotics
(ISR2021), 2021.(To be coming)
[20] K. Kidholm et al., “A model for assessment of telemedicine
applications: Mast,” Int. J. Technol. Assess. Health Care, vol. 28, no.
1, pp. 44–51, 2012, doi: 10.1017/S0266462311000638.
[21] L. de Witte, E. Steel, S. Gupta, V. D. Ramos, and U. Roentgen,
“Assistive technology provision: towards an international framework
for assuring availability and accessibility of affordable high-quality
assistive technology,” Disabil. Rehabil. Assist. Technol., 2018, doi:
10.1080/17483107.2018.1470264.
[22] M. MacLachlan et al., “Assistive technology policy: a position paper
from the first global research, innovation, and education on assistive
technology (GREAT) summit,” Disabil. Rehabil. Assist. Technol.,
2018, doi: 10.1080/17483107.2018.1468496.
[23] K. Baudin, M. Mullersdorf, A. Sundstrom, and C. Gustafsson, “The
Policies of Provision of Assistive and Welfare Technology—A
Literature Review,” Societies, vol. 10, no. 1, p. 22, 2020, doi:
10.3390/soc10010022.
[24] “HTA Core Model.” https://eunethta.eu/hta-core-model/.
[25] N. Backhaus, P. H. Rosen, A. Scheidig, H. M. Gross, and S.
Wischniewski, “Somebody help me, please?!’ Interaction Design
Framework for Needy Mobile Service Robots∗,” Proc. IEEE Work.
Adv. Robot. its Soc. Impacts, ARSO, vol. 2018-Septe, pp. 54–61,
2019, doi: 10.1109/ARSO.2018.8625721.
[26] M. N. C. and L. J. Yeo, A., Legard, R., Keegan, J., Ward, K.,
“Chapter 7: In-depth Interviews,” Qual. Res. Pract. A Guid. Soc. Sci.
Students Res., 2014.
[27] T. Binder, E. Brandt, J. Halse, M. Foverskov, S. Olander, and S.
Louise, “Living the ( Codesign ) Lab,” in Nordic Design Research
Conference, 2011, pp. 1–10.
[28] M. Yasuoka, F. Akasaka, A. Kimura, and M. Ihara, “Living labs as a
methodology for service design - An analysis based on cases and
discussions from a systems approach viewpoint,” Proceedings of
International Design Conference, DESIGN, vol. 1, no. 2012. pp.
127–136, 2018, doi: 10.21278/idc.2018.0350.
[29] S. Leminen, M. Westerlund, and A. Nyström, “Living Labs as open-
innovation networks,” Technol. Innov. Manag. Re, no. September,
pp. 6–11, 2012.
[30] B. B. Kareborn and A. Stahlbrost, “Living Lab: an open and citizen-
centric approach for innovation,” Int. J. Innov. Reg. Dev., vol. 1, no.
4, p. 356, 2009, doi: 10.1504/IJIRD.2009.022727.
[31] E. Brandt and J. Messeter, “Facilitating collaboration through design
games,” in Proceedings of the eighth conference on Participatory
design Artful integration: interweaving media, materials and
practices - PDC 04, 2004, vol. 1, p. 121, doi:
10.1145/1011870.1011885.
[32] M. Yasuoka, K. Kadoya, and T. Niwa, “Introducing a game approach
towards IS requirements specification,” Proc. Annu. Hawaii Int.
Conf. Syst. Sci., pp. 3687–3696, 2014, doi:
10.1109/HICSS.2014.459.
[33] R. Scupin, “The KJ method: A technique for analyzing data derived
from Japanese ethnology,” Hum. Organ., 1997, doi:
10.17730/humo.56.2.x335923511444655.
[34] A. Pichon-Riviere, N. Soto, F. Augustovski, S. Garcia-Marti, and L.
Sampietro-Colom, “Involvement of relevant stakeholders in health
technology assessment development,” Second Lat. Am. Forum Heal.
Technol. Assess. Policies, pp. 1–13, 2017.