Conference PaperPDF Available

"Why is the Doctor a Man": Reactions of Older Adults to a Virtual Training Doctor


Abstract and Figures

Shared decision making (SDM) is increasingly considered as the best way to reach a treatment decision in a clinical environment. However, the use of SDM in practice can be obstructed by a number of factors, such as time constraints or lack of applicability due to patient characteristics. Our project, PrepDoc, explores how a Virtual Training Doctor (VTD) can help patients overcome some of these obstacles to experiencing effective SDM during doctor's visits. In this paper, we report on user studies conducted with 19 participants in Scotland aged 65+. The goal of these studies was to identify the reactions of this audience to the PrepDoc system, evaluate its suitability within Scotland, and elicit suggestions to improve it. Our findings revealed that the idea of empowering people to participate in SDM using a virtual agent was positively received by all participants. However, the reactions to how this idea was implemented in the PrepDoc system varied greatly across participants. Based on this, our paper outlines recommendations for enhancing the user experience with VTDs, accommodating individual differences of older adults, and accounting for the national context.
Content may be subject to copyright.
Edinburgh Research Explorer
"Why is the Doctor a Man?" Reactions of Older Adults to a
Virtual Training Doctor
Citation for published version:
Constantin, A, Lai, C, Farrow, E, Alex, B, Pel-Littel, R, Nap, HH & Jeuring, J 2019, "Why is the Doctor a
Man?" Reactions of Older Adults to a Virtual Training Doctor. in Extended Abstracts of the 2019 CHI
Conference on Human Factors in Computing Systems., LBW1719, CHI EA '19, Glasgow, Scotland UK,
ACM CHI Conference on Human Factors in Computing Systems 2019, Glasgow, United Kingdom, 4/05/19.
DOI: 10.1145/3290607.3312811
Digital Object Identifier (DOI):
Link to publication record in Edinburgh Research Explorer
Document Version:
Peer reviewed version
Published In:
Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems
General rights
Copyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)
and / or other copyright owners and it is a condition of accessing these publications that users recognise and
abide by the legal requirements associated with these rights.
Take down policy
The University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorer
content complies with UK legislation. If you believe that the public display of this file breaches copyright please
contact providing details, and we will remove access to the work immediately and
investigate your claim.
Download date: 16. May. 2019
“Why is the Doctor a Man?”
Reactions of Older Adults to a
Virtual Training Doctor
Aurora Constantin
Catherine Lai
Elaine Farrow
Beatrice Alex
University of Edinburgh
Edinburgh, UK
Ruth Pel-Liel
Henk Herman Nap
Utrecht, The Netherlands
Johan Jeuring
Utrecht University
Utrecht, The Netherlands
Also with Alan Turing Institute.
Shared Decision Making (SDM)
Shared decision making in the con-
text of health care services is the process of a
practitioner and a patient jointly choosing an
appropriate medical test or treatment as a way
to enable patient-centred care.
Shared decision making (SDM) is increasingly considered as the best way to reach a treatment decision
in a clinical environment. However, the use of SDM in practice can be obstructed by a number of
factors, such as time constraints or lack of applicability due to patient characteristics. Our project,
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the
full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact
the owner/author(s).
CHI’19 Extended Abstracts, May 4–9, 2019, Glasgow, Scotland UK
©2019 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-5971-9/19/05.
PrepDoc, explores how a Virtual Training Doctor (VTD) can help patients overcome some of these
obstacles to experiencing eective SDM during doctor’s visits. In this paper, we report on user studies
conducted with 19 participants in Scotland aged 65+. The goal of these studies was to identify the
reactions of this audience to the PrepDoc system, evaluate its suitability within Scotland, and elicit
suggestions to improve it. Our findings revealed that the idea of empowering people to participate in
SDM using a virtual agent was positively received by all participants. However, the reactions to how
this idea was implemented in the PrepDoc system varied greatly across participants. Based on this,
our paper outlines recommendations for enhancing the user experience with VTDs, accommodating
individual dierences of older adults, and accounting for the national context.
User Experience; Evaluation Study; Technology for Older Adults; Shared Decision Making; Health.
Shared decision making (SDM) facilitates the discussion between health care professionals and
patients when decisions have to be made about desired care and treatment [3]. SDM has numerous
benefits (e.g., patients have increased knowledge of the options, more accurate risk perception, greater
comfort with decisions [
]), and several countries have developed programs to stimulate SDM in
discussions between health care professionals and patients [
]. These programs target awareness
of SDM among health care professionals and patients, and sometimes oer training to health care
professionals. However, SDM is not yet common practice [
]. Despite the general agreement about
its benefits, uptake of SDM faces a series of barriers, such as time constraints, lack of applicability
due to patient characteristics, and lack of applicability due to the clinical situation [
]. Elwyn and
collaborators [
] suggest a 3-step model to support SDM which, they emphasise, “has to be built on
the core skills of good clinical communication skills”. Practising SDM conversations can help overcome
patient fears over being seen as diicult or feeling like an unequal partner in the conversation. They
can also help patients improve their health literacy, e.g., understanding of medical terms, before
appointments. Such preparation can be crucial given tight time constraints in GP visits.
A number of applications have been designed for training communication skills. For example,
Cláudio et al. [
] present a game for communication training and assessment of self-medication
consultation skills, allowing students in Pharmaceutical Sciences to communicate with Virtual Humans
(VH) in an environment that simulates realistic self-medication scenarios. Other applications have
targeted the sales sector [
], and how to handle diicult passengers on public transport [
]. Zhang and
Bickmore [
] describe a virtual decision coach providing guidance around prenatal testing options.
However, as far as we are aware there are no existing applications designed for practising SDM in
clinical environments.
The goal of the PrepDoc
project is to explore how an online application can be designed to help
patients aged 65+ to prepare for GP visits by practising SDM in conversations with a Virtual Training
Doctor (VTD) using multi-modal interaction (mouse, text, audio, and speech).
Example Scenario
In one scenario a user is told that they need to
undergo hip surgery, but also that they want to
go on a walking holiday next month. Their task
is to ensure that in the conversation with the
doctor, their personal situation is discussed.
Figure 1: Daniel, the virtual training doc-
Figure 2: Sarah, the virtual assistant in the
training scenarios
Building on our previous work designing a serious game to support students in higher education
in practising communication skills [
] and on the 3-step model to support SDM [
] developed by
Elwyn and collaborators, we designed an online application that oers users several scenarios in
which they can practise SDM in conversations with a VTD [
]. The system was initially developed in
Dutch and then ported to English. This paper presents the results from an evaluation study of the
English-language version, carried out in Scotland. From these studies we present recommendations
for enhancing the user experience with VTD systems. Our work is expected to bring a number of
contributions to the CHI community. First, we identify dierent factors that can impact the experience
of older adults (age 65+) with VTD systems for SDM. Second we outline the implications of these for
the future design of VTD systems for SDM.
The PrepDoc system is an online application that oers a user several scenarios in which they can
practise SDM in conversations with two virtual characters (VCs): a doctor and an assistant. The initial
design of the system was informed by co-creation sessions organised in Utrecht with Dutch people
aged 65+, including one GP and one GP assistant.
Each scenario is completely scripted (see example in sidebar). The doctor starts the conversation,
and the user proceeds, either by selecting an answer from a list of options with the mouse, or by
speaking or typing their answer (Figure 1). A few questions allow free responses, but most are limited
to predefined options. Aer each conversation with the doctor, the user interacts with the assistant
(Figure 2) who highlights the main points of the conversation with the doctor and helps the user
reflect on what they have learned and prepare for their own GP visit.
We recruited 19 participants aged 66 to 87, using a snowball methodology. 15 participants had a
university degree and two had professional qualifications. Eight (6 males) of the participants had
worked in computer science (CS) and were familiar with virtual characters and dialogue systems (
); the remaining 11 (2 males) came from a variety of educational and occupational backgrounds
no-CS group
). We were interested in understanding whether prior familiarity with CS has an impact
on the perceived user experience and/or on users’ reactions to the system.
Each participant worked with the PrepDoc system on a laptop, individually and at their own
pace, for 60-90 minutes. The Think Aloud protocol [
] was employed while the participants were
using the system. Aerwards, they completed a short online questionnaire which included a System
Usability Scale (SUS) questionnaire [
]. The aim of the questionnaire was to understand the overall
perceived experience of using the system, and other related issues. Each participant also took part in
a semi-structured interview at the end of their session. The interview questions were designed to shed
more light on the user experience with the system and to collect suggestions for improvements. Two
researchers from the team were present at each session to observe the participants and take notes. In
addition, a camera was used to record the audio and the image of the laptop screen.
Figure 3: Individual System Usability
Scale (SUS) scores from participants (CS
group is shown in orange).
Figure 4: Individual scores for perceived
experience on a 5-point Likert scale, from
1 (“not good at all”) to 5 (“very good”). The
CS group is shown in orange.
Figure 5: Individual scores for perceived
usefulness on a 5-point Likert scale, from
1 (“not good at all”) to 5 (“very good”). The
CS group is shown in orange.
Because the number of participants was small, we only used basic statistics for the ordinal data. To
synthesise the notes and the video transcripts we employed open and axial coding [
]. The idea of
empowering older adults in SDM during GP visits was unanimously praised. However, our findings
revealed large dierences in how participants perceived the implementation of this idea in PrepDoc.
Usability. The overall SUS score (Figure 3) for all participants (
SD =
39) was very
close to 68, suggesting good usability [
]. However, 6 no-CS group participants and 2 CS group
participants scored the system below 68, with one giving a score of 45, which is below the threshold
to be considered “acceptable”. The system scored slightly beer on average (
SD =
with the no-CS group than with the the CS group (
SD =
59). That may be because
participants from the CS group start with higher expectations of the system.
Perceived Experience. There was also a slight tendency in the no-CS group to score PrepDoc higher
than people in the CS group in terms of perceived experience with the system (Figure 4). The median
and mode in both groups was 4 (
SD =
36 for the CS group,
SD =
47 for the no-CS group). None
of the participants in the CS group scored it “very good” and two of them scored it as “not good at
all”. Within the no-CS group the scores were more consistent, with three participants scoring their
experience “very good”. This may be again the consequence of participants in CS group having higher
expectations of the system, but also the fact that all of them are highly educated and have broad
general knowledge. For example, a professor commented: “This is not what I expected it to be. In fact,
everything is basically a presentation of things that I believe that most people know”.
Perceived usefulness. There was a greater divergence in the perceived usefulness of the system
between the two groups (Figure 5) The median for both groups was 4, whereas the mode was 4 for
the CS group and 5 for the no-CS group (
SD =
20 for the CS group,
SD =
75 for the no-CS group).
Only one participant in the CS group scored the system “very useful” and one participant scored it
“not useful at all”, whereas four people from the no-CS group found it “very useful”. That is probably
due to the fact that the people in the CS group already felt confident in discussions with their GPs, as
they are highly educated and have excellent communication skills, evidenced by comments like, “If I
were less skilled, I think that would have been useful”. People in the no-CS group also tended to think
that the usefulness may be related to education level, personality, and intelligence. In the interview, a
no-CS participant stated, “I think it depends on your education and your personality and. . .Might be
good for some people a lile bit less bright”.
The tool is useful for showing you that you
should prepare for GP visits. GP time is def-
initely short and being organised is impor-
tant. (P10)
‘What’ questions are very good to help peo-
ple prepare for the GP visit... they are the
best bit of the whole thing. (P1)
Obviously, it’s good advice to think about
questions to ask a doctor and talk about to
somebody else. I do that with my wife. (P5)
Several participants expressed that
the system was useful to help them
prepare for visits to the doctor.
Age- & Gender-related Stereotypes. Nine of the participants (5 from the CS group) perceived the
dialogue as patronising. Some of them specifically disliked the repetition and reinforcement in the
dialogue, describing it as “condescending” or “patronising”, though they recognised that it may be
useful for certain people. Others had stronger reactions. For example, one of the professors declared,
“The trouble with this is making the assumption that older people are less intelligent, or older people
are less well-informed by default”. Five participants commented on the gender of the doctor, with one
male participant asking "Why is the doctor a man?" – the majority of GPs in the UK are female.
Dialogue Structure. Nearly all participants (17 of 19) felt that the dialogue did not cover all possible
options and/ or was not flexible enough to allow them to shape the discussion, saying, for example, “It‘s
quite frustrating that it [the system] does not allow you to shape the conversation”. Some participants
considered the dialogue simplistic: “I think the system is conceptually good, but it did not cover all
possibilities. I am amazed that the GP did not bring into discussion a critical issue like weight”.
Individual Dierences. Most participants (14 of 19; 6 of 8 in the CS group) discussed individual
dierences and how these should be approached within the system. For example, one participant (a
retired doctor) suggested that the system should collect information about the users at the beginning.
Thus, if the user is a doctor or a nurse, the VTD should know that "some of the answers are going to
be coloured by that”. Another participant suggested allowing the users to choose whether they would
like to drive the dialogue, or prefer the VTD to do that.
I like the lesson that it’s okay to ask GPs
questions, and that there are alternatives.
You can say “no” to a treatment. (P14)
It’s very useful that it emphasises that
healthcare is “a partnership” between the
patient and the doctor. (P1)
The system helped some participants
to realise the benefits of shared deci-
sion making.
National Context. Many participants (10 of 19; 4 of 8 in the CS group) noticed that the specific
recommendations of the VTD are sometimes inappropriate for the UK context: “the corticosteroid
injection is not common here in Scotland, but is common ‘on the continent”’.
Multimodal Interaction with virtual characters. There was a clear preference for using voice over
clicks to avoid physical eort, with no dierence between the two groups. However, because of some
technical problems (e.g. the recording of open-ended answers was cut o too early), most of the
participants used clicks. With regard to the VCs, participants’ preferences varied. Some liked them (9
of 19; 4 of 8 in the CS group) for various reasons (“Daniel is good, he listens, but some doctors don’t”,
“Daniel is very pleasant”). Other disliked them (“her smile is gruesome”).
VTD systems to support SDM during GP visits were highly appreciated by our participants. They
found the system useful for preparing the patients for SDM during GP visits and suggestion a number
of ways to extend its use. However, more research should be conducted to beer accommodate needs
related to ageing and individual dierences in this target population, and to adapt it to the national
context. Reflecting on our findings, we have the following recommendations for the field:
R1: Avoid age-& gender-related stereotypes. Focus on physical diiculties that arise with ageing
rather than communication issues. Provide customisation options, so that the user can choose the
gender of the VCs.
This system is very useful to approach sensi-
tive topics, such as pregnancy or menopause.
A candidate scenario could be parents bring-
ing in their children. The question is when do
you interact with the child versus the parent?
These are known to be diicult situations
which may benefit from some exploration
before the doctor’s visit. (P8)
Participants oered suggestions for
future scenarios that could be devel-
oped for a tool of this kind.
R2: Make the scenarios more flexible and allow free input. Scenario design should be beer informed
by user studies involving a wider variety of stakeholders and encompassing various national contexts.
R3: Incorporate a user profile (covering personality, medical knowledge, interests) to deliver content
that matches the user’s needs and interests. This could be achieved through a personalised and
adaptive digital environment.
The PrepDoc project (the activity “Empowering
older people in conversations with health-care
professionals”) has received funding from the
European Institute of Innovation and Technol-
ogy (EIT). This body of the European Union re-
ceives support from the European Union’s Hori-
zon 2020 research and innovation programme.
Many thanks are due to the participants in the
evaluation sessions for their contribution and
T. Bosse and C. Gerritsen. 2016. Towards Serious Gaming for Communication Training - A Pilot Study with PoliceAcademy
Students. In INTETAIN.
A.P. Cláudio, M.B. Beatriz Carmo, V. Pinto, and A. Cavaco. 2015. Virtual Humans for Training and Assessment of
Self-medication Consultation Skills in Pharmacy Students. In Proceedings ICCSE 2015. IEEE, 175–180.
G. Elwyn, D. Frosch, R. Thomson, N. Joseph-Williams, A. Lloyd, P. Kinnersley, E. Cording, D. Tomson, C. Dodd, S. Rollnick,
A. Edwards, and M. Barry. 2012. Shared decision making: a model for clinical practice. Journal of general internal medicine
27, 10 (2012), 1361–1367.
G. Elwyn, I. Scholl, C. Tietbohl, M. Mann, A.G. Edwards, C. Clay, F. Legare, T. van der Weijden, C.L. Lewis, R.M. Wexler,
and D.L. Frosch. 2013. "Many miles to go ...": a systematic review of the implementation of patient decision support
interventions into routine clinical practice. BMC medical informatics and decision making 13, Suppl 2 S14 (2013).
[5] K. A. Ericsson and H. A. Simon. 1993. Protocol analysis. MIT press Cambridge, MA.
K. Gravel, F. Legare, and I.D. Graham. 2006. Barriers and facilitators to implementing shared decision-making in clinical
practice: a systematic review of health professionals’ perceptions. Implementation science 1 (2006), 16.
J. Jeuring, F. Grosfeld, B. Heeren, M. Hulsbergen, R. IJntema, V. Jonker, N. Mastenbroek, M. van der Smagt, F. Wijmans,
M. Wolters, and H. van Zeijts. 2015. Communicate! — A Serious Game for Communication Skills. In Proc. EC-TEL 2015
(LNCS), Vol. 9307. Springer, 513–517.
T. J. Muller, A. Heuvelink, K. van den Bosch, and I. Swartjes. 2012. Glengarry Glen Ross: Using BDI for Sales Game
Dialogues. In Proc. AIIDE 2012. AAAI Press, 167–172.
R. Pel-Liel, H. van Zeijts, N. Schram, H. H. Nap, and J. Jeuring. 2018. A training simulation for practicing shared decision
making for older patients. In Proceedings of ICTH 2018 (Procedia Computer Science), Vol. 141. Springer, 287–293.
[10] Johnny Saldaña. 2015. The coding manual for qualitative researchers. Sage.
[11] Je Sauro. 2011. Measuring usability. A Practical Guide to the System Usability Scale 1 (2011).
D. Stacey, F. Legare, K. Lewis, M.J. Barry, C.L. Benne, K.B. Eden, M. Holmes-Rovner, H. Llewellyn-Thomas, A. Lyddia, R.
Thomson, and L. Trevena. 2017. Decision aids for people facing health treatment or screening decisions. The Cochrane
database of systematic reviews 4 (2017).
[13] Z. Zhang and T. Bickmore. 2018. Medical Shared Decision Making with a Virtual Agent. Proc. IVA ’18 (2018), 113–118.

Supplementary resource (1)

... These changes may be done by collaboratively recognizing users' preferences [39], actions [40]- [42], or emotions [43]. Health information also can be collaboratively personalized by collecting user data during the interaction and helping users with medication [44], [45], diet [42], and recommendation for treatments [30], [31], [46], [47]. A specific subject [48] or instruction also can be personalized to help people in physical [4], [34], [49]- [51] or everyday activities [21], [24]. ...
... In some studies, the context adapted is health information. These agents can help users by encouraging them to take their medications [44], keeping a healthy routine [47], maintaining a diet [52]. Another kind of health information is recommending treatments as self-care plans [30], [31], [46], specific care [53], or the usage of appropriate drugs [45]. ...
... Agents embedded in Desktop Systems have the purpose to be used without mobility in a room [20], [44], [47], [49], [51]. ...
... The gender of the agent also varied across studies. Agents were primarily femal (7). Four results used male agents. ...
... The results suggest that characteristics of the agent were chosen based on the task the agent was doing [7,21,23], in consultation with a focus group [1], or modified to fit the target population [4,19]. These results support research that traditional gender norms play a role in VUIs [12]. ...
... Our analysis of the manuscripts revealed a strong emphasis on capturing older adults' views on usability [4,7,21,26]. Key usability questions raised in the papers relate to topics such as older adults' views about the ease of use of VUIs [1,4,19,26] and whether they prefer VUIs to have a physical (such as being part of a robot), or a virtual (voice assistant) form [14,19,20]. ...
... They also felt more enjoyable and confident and were more willing to continue to learn than the Video approach although the performance of Synapse (Trial-and-Error) and the Video approach was not significantly different. Participants preferred voice interfaces to other approaches, such as clicking and touching the screen [5,11,23,32,44]. In addition to the natural language interaction (e.g., voice assistance), older adults highly praised the error-recovery feature, which encouraged them to perform trial-and-error that they would have otherwise apprehended. ...
Full-text available
As smartphones are widely adopted, mobile applications (apps) are emerging to provide critical services such as food delivery and telemedicine. While bring convenience to everyday life, this trend may create barriers for older adults who tend to be less tech-savvy than young people. In-person or screen sharing support is helpful but limited by the help-givers' availability. Video tutorials can be useful but require users to switch contexts between watching the tutorial and performing the corresponding actions in the app, which is cumbersome to do on a mobile phone. Although interactive tutorials have been shown to be promising, none was designed for older adults. Furthermore, the trial-and-error approach has been shown to be beneficial for older adults, but they often lack support to use the approach. Inspired by both interactive tutorials and trial-and-error approach, we designed an app-independent mobile service, Synapse, for help-givers to create a multimodal interactive tutorial on a smartphone and for help-receivers (e.g., older adults) to receive interactive guidance with trial-and-error support when they work on the same task. We conducted a user study with 18 older adults who were 60 and over. Our quantitative and qualitative results show that Synapse provided better support than the traditional video approach and enabled participants to feel more confident and motivated. Lastly, we present further design considerations to better support older adults with trial-and-error on smartphones.
... In the context of education, text-based agents have been investigated for learning languages (Fryer et al., 2017), collaborative problem solving (Hayashi, 2013;Herborn et al., 2018), and programming education (Hobert, 2019). In the area of health, researchers have focused on, for example, chatbots for therapy (Bell et al., 2019;Constantin et al., 2019), for raising individual health awareness (Meier et al., 2019), or supporting people with allergies (Hsu et al., 2017).Research on text-based CAs focuses on constructs related to perception (e.g., humanness), attitude (e.g., attractiveness), performance (e.g., responsiveness), and acceptance (e.g., perceived usefulness). Regarding text-based agents, studies mainly investigate how design variations concerning the agent's identity (e.g., Araujo (2018) Complementary to the question of how to design human-like text-based agents, researchers have discussed how an anthropomorphic design might trigger perceptions of uncanniness in the interaction (e.g., Gnewuch et al. (2018), Wünderlich and Paluch (2017)). ...
Full-text available
Conversational agents (CAs), described as software with which humans interact through natural language, have increasingly attracted interest in both academia and practice, due to improved capabilities driven by advances in artificial intelligence and, specifically, natural language processing. CAs are used in contexts like people's private life, education, and healthcare, as well as in organizations, to innovate and automate tasks, for example in marketing and sales or customer service. In addition to these application contexts, such agents take on different forms concerning their embodiment, the communication mode, and their (often human-like) design. Despite their popularity, many CAs are not able to fulfill expectations and to foster a positive user experience is a challenging endeavor. To better understand how CAs can be designed to fulfill their intended purpose, and how humans interact with them, a multitude of studies focusing on human-computer interaction have been carried out. These have contributed to our understanding of this technology. However, currently a structured overview of this research is missing, which impedes the systematic identification of research gaps and knowledge on which to build on in future studies. To address this issue, we have conducted an organizing and assessing review of 262 studies, applying a socio-technical lens to analyze CA research regarding the user interaction, context, agent design, as well as perception and outcome. We contribute an overview of the status quo of CA research, identify four research streams through a cluster analysis, and propose a research agenda comprising six avenues and sixteen directions to move the field forward.
Full-text available
Background Smart speaker–based voice assistants promise support for the aging population, with the advantages of hands-free and eyes-free interaction modalities to handle requests. However, little is known about how older adults perceive the benefits of this type of device. Objective This study investigates how older adults experience and respond to a voice assistant when they first interact with it. Because first impressions act as strong predictors of the overall attitude and acceptability of new technologies, it is important to understand the user experiences of first exposure. Methods We conducted semistructured interviews with 18 people 74 years and older who had never used a smart speaker before, investigating the patterns of use, usability issues, and perspectives that older adults have when using a voice assistant for the first time. Results The overall first response to a voice assistant was positive, thanks to the simplicity of a speech-based interaction. In particular, a positive and polite response to complete the interaction with a voice assistant was prevalent, such as expressing gratitude or giving feedback about the quality of answers. Two predominant topics of commands made in the first interaction include asking health care–related questions and streaming music. However, most of the follow-up reactions were unfavorable because of the difficulty in constructing a structured sentence for a command; misperceptions about how a voice assistant operates; and concerns about privacy, security, and financial burdens. Overall, a speech-based interaction was perceived to be beneficial owing to its efficiency and convenience, but no other benefits were perceived. Conclusions On the basis of the findings, we discuss design implications that can positively influence older adults' first experiences with a voice assistant, including helping better understand how a voice assistant works, incorporating mistakes and common interaction patterns into its design, and providing features tailored to the needs of older adults.
Conference Paper
Full-text available
Shared decision making (SDM) is a process in which patients and clinicians work together to make medical decisions based on clinical evidence that balances risks and expected outcomes with patient preferences and values. SDM has been advocated as the ideal model for patient decision-making in healthcare. However, SDM is rarely used in practice due, in part, to limited availability of healthcare professionals trained in these techniques. In this work, we describe the design of a virtual decision coach to help women participate in SDM about prenatal testing for Down syndrome. In a quasi-experimental evaluation study, participants demonstrated significant increases in knowledge, high levels of satisfaction with their final decision and low levels of decisional conflict and regret, indicating that virtual agents can effectively perform in the role of decision coach and facilitate the implementation of SDM.
Full-text available
Communicate! is a serious game for practicing communication skills. It supports practicing interpersonal communication skills between a health care professional such as a doctor or a pharmacist, or a (business) psychologist, and a patient or client. A player selects a scenario, and holds a consultation with a virtual character. In the consultation, the player chooses between the various options offered in the scenario. The player scores on the learning goals addressed by the scenario, and gets immediate feedback through the effect of the choice between the answer options on the utterance and emotion of the virtual character. Communicate! also offers an editor for scenarios. A scenario is a graph-like structure, extended with several constructs to avoid the development of repetitive structures. We have performed several experiments with Communicate!, both with students to evaluate the use of Communicate! in various programs at Utrecht University, and with teachers to evaluate the development of scenarios for Communicate!
Conference Paper
Full-text available
Serious games offer an opportunity for players to learn communication skills by practicing conversations with non-playing characters (NPCs). To realize this potential, the player needs freedom of play to discover the relationships between its actions and their effects on the partner and the conversation. Scripting is currently the common approach to design in-game dialogue. Although scripting is a robust technique, the approach tends to produce deterministic con-versations, allowing little control to the player. It is claimed that a Belief-Desire-Intention (BDI) approach to model the behavior of NPCs allows greater freedom to the player, and delivers better scalability and re-use of dialogues. This claim is evaluated by using BDI in the development of a sales-talk training game in the real-estate domain. It is con-cluded that BDI enables representative NPCs that respond appropriately and the game allows the player its freedom of choice to explore. The results also showed that BDI brings about new challenges to address, in order to further increase the quality of in-game dialogue.
Full-text available
Two decades of research has established the positive effect of using patient-targeted decision support interventions: patients gain knowledge, greater understanding of probabilities and increased confidence in decisions. Yet, despite their efficacy, the effectiveness of these decision support interventions in routine practice has yet to be established; widespread adoption has not occurred. The aim of this review was to search for and analyze the findings of published peer-reviewed studies that investigated the success levels of strategies or methods where attempts were made to implement patient-targeted decision support interventions into routine clinical settings. An electronic search strategy was devised and adapted for the following databases: ASSIA, CINAHL, Embase, HMIC, Medline, Medline-in-process, OpenSIGLE, PsycINFO, Scopus, Social Services Abstracts, and the Web of Science. In addition, we used snowballing techniques. Studies were included after dual independent assessment. After assessment, 5322 abstracts yielded 51 articles for consideration. After examining full-texts, 17 studies were included and subjected to data extraction. The approach used in all studies was one where clinicians and their staff used a referral model, asking eligible patients to use decision support. The results point to significant challenges to the implementation of patient decision support using this model, including indifference on the part of health care professionals. This indifference stemmed from a reported lack of confidence in the content of decision support interventions and concern about disruption to established workflows, ultimately contributing to organizational inertia regarding their adoption. It seems too early to make firm recommendations about how best to implement patient decision support into routine practice because approaches that use a 'referral model' consistently report difficulties. We sense that the underlying issues that militate against the use of patient decision support and, more generally, limit the adoption of shared decision making, are under-investigated and under-specified. Future reports from implementation studies could be improved by following guidelines, for example the SQUIRE proposals, and by adopting methods that would be able to go beyond the 'barriers' and 'facilitators' approach to understand more about the nature of professional and organizational resistance to these tools. The lack of incentives that reward the use of these interventions needs to be considered as a significant impediment.
Background: Decision aids are interventions that support patients by making their decisions explicit, providing information about options and associated benefits/harms, and helping clarify congruence between decisions and personal values. Objectives: To assess the effects of decision aids in people facing treatment or screening decisions. Search methods: Updated search (2012 to April 2015) in CENTRAL; MEDLINE; Embase; PsycINFO; and grey literature; includes CINAHL to September 2008. Selection criteria: We included published randomized controlled trials comparing decision aids to usual care and/or alternative interventions. For this update, we excluded studies comparing detailed versus simple decision aids. Data collection and analysis: Two reviewers independently screened citations for inclusion, extracted data, and assessed risk of bias. Primary outcomes, based on the International Patient Decision Aid Standards (IPDAS), were attributes related to the choice made and the decision-making process.Secondary outcomes were behavioural, health, and health system effects.We pooled results using mean differences (MDs) and risk ratios (RRs), applying a random-effects model. We conducted a subgroup analysis of studies that used the patient decision aid to prepare for the consultation and of those that used it in the consultation. We used GRADE to assess the strength of the evidence. Main results: We included 105 studies involving 31,043 participants. This update added 18 studies and removed 28 previously included studies comparing detailed versus simple decision aids. During the 'Risk of bias' assessment, we rated two items (selective reporting and blinding of participants/personnel) as mostly unclear due to inadequate reporting. Twelve of 105 studies were at high risk of bias.With regard to the attributes of the choice made, decision aids increased participants' knowledge (MD 13.27/100; 95% confidence interval (CI) 11.32 to 15.23; 52 studies; N = 13,316; high-quality evidence), accuracy of risk perceptions (RR 2.10; 95% CI 1.66 to 2.66; 17 studies; N = 5096; moderate-quality evidence), and congruency between informed values and care choices (RR 2.06; 95% CI 1.46 to 2.91; 10 studies; N = 4626; low-quality evidence) compared to usual care.Regarding attributes related to the decision-making process and compared to usual care, decision aids decreased decisional conflict related to feeling uninformed (MD -9.28/100; 95% CI -12.20 to -6.36; 27 studies; N = 5707; high-quality evidence), indecision about personal values (MD -8.81/100; 95% CI -11.99 to -5.63; 23 studies; N = 5068; high-quality evidence), and the proportion of people who were passive in decision making (RR 0.68; 95% CI 0.55 to 0.83; 16 studies; N = 3180; moderate-quality evidence).Decision aids reduced the proportion of undecided participants and appeared to have a positive effect on patient-clinician communication. Moreover, those exposed to a decision aid were either equally or more satisfied with their decision, the decision-making process, and/or the preparation for decision making compared to usual care.Decision aids also reduced the number of people choosing major elective invasive surgery in favour of more conservative options (RR 0.86; 95% CI 0.75 to 1.00; 18 studies; N = 3844), but this reduction reached statistical significance only after removing the study on prophylactic mastectomy for breast cancer gene carriers (RR 0.84; 95% CI 0.73 to 0.97; 17 studies; N = 3108). Compared to usual care, decision aids reduced the number of people choosing prostate-specific antigen screening (RR 0.88; 95% CI 0.80 to 0.98; 10 studies; N = 3996) and increased those choosing to start new medications for diabetes (RR 1.65; 95% CI 1.06 to 2.56; 4 studies; N = 447). For other testing and screening choices, mostly there were no differences between decision aids and usual care.The median effect of decision aids on length of consultation was 2.6 minutes longer (24 versus 21; 7.5% increase). The costs of the decision aid group were lower in two studies and similar to usual care in four studies. People receiving decision aids do not appear to differ from those receiving usual care in terms of anxiety, general health outcomes, and condition-specific health outcomes. Studies did not report adverse events associated with the use of decision aids.In subgroup analysis, we compared results for decision aids used in preparation for the consultation versus during the consultation, finding similar improvements in pooled analysis for knowledge and accurate risk perception. For other outcomes, we could not conduct formal subgroup analyses because there were too few studies in each subgroup. Authors' conclusions: Compared to usual care across a wide variety of decision contexts, people exposed to decision aids feel more knowledgeable, better informed, and clearer about their values, and they probably have a more active role in decision making and more accurate risk perceptions. There is growing evidence that decision aids may improve values-congruent choices. There are no adverse effects on health outcomes or satisfaction. New for this updated is evidence indicating improved knowledge and accurate risk perceptions when decision aids are used either within or in preparation for the consultation. Further research is needed on the effects on adherence with the chosen option, cost-effectiveness, and use with lower literacy populations.
Conference Paper
Serious games are increasingly being used for training of communicative skills. The main idea is to create a virtual environment in which a trainee can interact with graphically embodied virtual characters. By designing scenarios in such way that the character’s behaviour provides direct feedback on the correctness of the trainee’s choices, an interactive learning experience is created. This paper explores the potential of this approach in the domain of law enforcement. A prototype has been developed of a serious game that enables police academy students to train their communicative skills. A pilot study with 41 students has been conducted. The results show that this is a promising instrument for education in this domain, but also point out several suggestions for improvement.
Conference Paper
This Virtual patients have been used since the 1990's in pharmacy education for teaching and assessment of clinical and communication skills. Although their use is still limited, they allow students' engagement in true-to-life situations in a controllable environment. This paper describes an interactive application to assist in the academic training and assessment of Pharmaceutical Sciences students. Virtual humans play the role of patients and communicate with the student by speech and by facial and body language. The application comprises two usage modes: (i) the training mode, used autonomously by the student, gives detailed information about his performance and indicates places where he may find additional information, and (ii) the assessment mode which is used by the teacher to evaluate the student's performance. The process used to feed case-scenarios in the application does not require informatics skills because it resorts to an easy-to-use graphical interface. A BackOffice Web application was implemented to allow the creation of new self-medication situations and the collection of data about students' performance. The application was tested by a restricted group of experts whose overall opinion was quite positive about the usefulness of the application as a tool to improve students' as well as professionals' communication skills.