Deriving Design Principles for Educational Chatbots from Empirical Studies on Human–Chatbot Interaction

Full text

Deriving Design Principles for Educational Chatbots from Empirical Studies on Human–Chatbot
Interaction
교육용 챗봇 설계 원리 도출: 휴먼-챗봇 상호작용에 관한 실증연구를 토대로
저자
(Authors)
Hyojung Jung, Jinju Lee, Chaeyeon Park
출처
(Source)
한국디지털콘텐츠학회 논문지 21(3), 2020.3, 487-493 (7 pages)
Journal of Digital Contents Society 21(3), 2020.3, 487-493 (7 pages)
발행처
(Publisher)
한국디지털콘텐츠학회
Digital Contents Society
URL http://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE09321283
APA Style Hyojung Jung, Jinju Lee, Chaeyeon Park (2020). Deriving Design Principles for Educational Chatbots from
Empirical Studies on Human–Chatbot Interaction. 한국디지털콘텐츠학회 논문지, 21(3), 487-493.
이용정보
(Accessed)
저작권 안내
DBpia에서 제공되는 모든 저작물의 저작권은 원저작자에게 있으며, 누리미디어는 각 저작물의 내용을 보증하거나 책임을 지지 않습니다. 그리고 DBpia에서 제공되는 저작물은 DBpia와 구독
계약을 체결한 기관소속 이용자 혹은 해당 저작물의 개별 구매자가 비영리적으로만 이용할 수 있습니다. 그러므로 이에 위반하여 DBpia에서 제공되는 저작물을 복제, 전송 등의 방법으로 무단
이용하는 경우 관련 법령에 따라 민, 형사상의 책임을 질 수 있습니다.
Copyright Information
Copyright of all literary works provided by DBpia belongs to the copyright holder(s)and Nurimedia does not guarantee contents of the literary work or assume responsibility for the
same. In addition, the literary works provided by DBpia may only be used by the users affiliated to the institutions which executed a subscription agreement with DBpia or the
individual purchasers of the literary work(s)for non-commercial purposes. Therefore, any person who illegally uses the literary works provided by DBpia by means of reproduction or
transmission shall assume civil and criminal responsibility according to applicable laws and regulations.
단국대학교 죽전캠퍼스
220.149.***.10
2020/07/09 11:12 (KST)

Copyright ⓒ 2020 The Digital Contents Society
487
http://www.dcs.or.kr
pISSN: 1598-2009
eISSN: 2287-738X
JDCS
디지털콘텐츠학회논문지
Journal of Digital Contents Society
Vol. 21, No. 3, pp. 487-493, Mar. 2020
교육용 챗봇 설계 원리 도출: 휴먼-챗봇 상호작용에 관한 실증연구를 토대로
정 효 정1 · 이 진 주 2*· 박 채 연3
1단국대학교 자유교양대학 조교수
2한양대학교 교육공학과 박사과정
3한양대학교 교육공학과 석사과정
Deriving Design Principles for Educational Chatbots from
Empirical Studies on HumanChatbot Interaction
Hyojung Jung1 · Jinju Lee2* · Chaeyeon Park3
1Assistant Professor, College of General Education, Dankook University, Gyeonggi-do 16890, South Korea
2Doctoral Course, Department of Educational Technology, Hanyang University, Seoul 04763, South Korea
3Master’s Course, Department of Educational Technology, Hanyang University, Seoul 04763, South Korea
[요 약]
이 연구에서는 교육용 챗봇에 대한 체계적인 연구를 통해 챗봇 설계 시 고려해야 할 원리를 도출하였다. 이를 위하여 교육용 챗
봇에 대한 선행연구 분석을 진행하였으며, 분석 결과를 토대로 챗봇의 역할을 고려한 설계 원리를 제안하였다. 선행연구를 토대로
교육용 챗봇의 역할은 크게 튜터, 평가자, 응답자, 중재자, 학습동료로 구분할 수 있었다. 역할별로 고려해야 할 설계 원리를 탐색
한 결과, 튜터챗봇은 감성 원리(Live emotion principle), 양식 원리(Modality principle), 외생적 부하 조절 원리(Extraneous principle)
를 고려해야 하는 것으로 나타났다. 평가자 역할의 챗봇은 봇 효과 원리(Bot effect principle)를, 응답자 챗봇을 개발할 때는 성 원리
(Gender principle)과 양식 원리를 고려해야 한다. 중재자 챗봇의 경우 중립적 감정 원리(Neutral emotion principle)을, 동료 학습자
챗봇의 경우, 양식 원리와 더불어 모방 원리(Imitation principle), 중립적 감정 원리(Neutral emotion principle)를 고려해야 한다. 앞
으로는 챗봇의 역할에 따른 콘텐츠 제시 방법과 교육 챗봇의 차별화된 역할에 대한 연구를 더욱 심층적으로 수행할 필요가 있다.
[Abstract]
This study derives design principles according to the role of chatbots through a systematic review of educational chatbots. We
propose design principles that should be considered, depending on the role of the chatbot. When designing a chatbot that plays
the role of a tutor, it is necessary to consider the Live emotion principle, Modality principle, and Extraneous principle. When
designing a chatbot that acts as an evaluator, the Bot effect principle should be considered. When developing a chatbot that acts
as a responder, the Gender principle and Modality principle should be considered. In the case of a chatbot that plays the role of
a moderator, it is necessary to consider the Neutral emotion principle, and in the case of a chatbot that plays the role of peer
learner, the Modality principle (voice), the Imitation principle, and the Neutral emotion principle should be considered. In the
future, it is necessary to study the method of contents presentation and the differentiated role of educational chatbots.
색인어 : 챗봇, 챗봇매개학습(CML), 설계원리
Key word : Chatbot, Chatbot-Mediated Learning (CML), Design Principles
http://dx.doi.org/10.9728/dcs.2020.21.3.487
This is an Open Access article distributed under
the terms of the Creative Commons Attribution
Non-CommercialLicense(http://creativecommons
.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial
use, distribution, and reproduction in any medium, provided the
original work is properly cited.
Received 25 January 2020; Revised 15 March 2020
Accepted 25 March 2020
*Corresponding Author; Jinju Lee
Tel: +82-2-2220-1128
E-mail: jinju.a.lee@gmail.com
단국대학교 죽전캠퍼스 | IP:220.149.***.10 | Accessed 2020/07/09 11:12(KST)

디지털콘텐츠학회논문지
(J. DCS) Vol. 21, No. 3, pp. 487-493, Mar. 2020
http://dx.doi.org/10.9728/dcs.2020.21.3.487
488
Ⅰ. Introduction
Chatbots are computer programs that help humans
communicate with computers through text or voice interactions.
With the proliferation of Massive Open Online Courses (MOOCs)
and the widespread use of messaging apps, the need for chatbots
in education is increasing. There are three reasons for introducing
chatbots. First, customer management costs can be lowered [11].
Second, they can shorten the time within which a response is
provided to the customer, can support the service 24 hours a day,
and can improve user satisfaction through customized
consultation. Third, it is possible to improve the product or
service by collecting information about the customer’s needs
during the conversation with the chatbot. We may expect the
same possibility in the context of education. When using chatbot
technology for educational purposes, providing feedback to
learners can be made more efficient, and it can be done all the
time, increasing learners’ satisfaction. In addition, learning
support may be optimized by collecting a variety of information
about the learners. However, while chatbot technology is
evolving, its integration into education tends to be rather sluggish
[1]. There is a lack of research on the design principles to
consider when developing an educational chatbot. This study
aims to promote the development of educational chatbots by
setting out the principles to be considered in designing
educational chatbots, based on systematic analysis.
RQ1: How have chatbots been incorporated into empirical
studies on human–chatbot interaction?
RQ2: What implications for educational chatbots can be
derived from the studies?
Ⅱ. Theoretical Background
2-1 Expectations and Roles of Chatbots
A chatbot is a computer program to simulate human
conversation via text or voice interaction [19]. Other terms for
chatbots include talkbots, chatterbots, conversational agents,
artificial conversational entities, and a conversational system.
Efforts have been made to introduce chatbots or similar
technologies in the education field, and related terms include a
pedagogical agent or intelligent pedagogical agent (IPA),
intelligent tutoring systems (ITS), and Artificial Intelligence
Markup Language (AIML) -based chatbot. In the context of
technology-mediated learning [2], chatbot-mediated learning
(CML) contributes to motivation, self-directed learning, and
individual learning by providing learners with individual learning
environments that enhance the learning process and its outcomes.
More specifically, chatbots can influence the learner’s learning
process – the way in which information is found and
communicated. In other words, rather than being provided with
the contents passively, learners can support themselves to ask
questions and lead the way. Second, learners can effectively
support the learning process in large classrooms or in large online
courses such as MOOCs. This may contribute to lowering the
dissatisfaction experienced by learners and lowering the dropout
rate. Third, learners can help them to make the right judgment by
providing optimal information at the right time, and can provide
continuous feedback to learners / teachers.
Research
Application area
Role of bot
Goal
[16]
Demonstration
Peer learner
Demonstration Partners
[11]
Ideation
Peer learner
Provide peer feedback
[20]
Fitness
Peer learner
Fitness companion
[7]
Q&A
Guide
Website navigation
[10]
Survey
Guide
Record response
[15]
Information retrieval
Guide
Search support
[22]
Customer service
Guide
Customer Agent
표
1.
챗봇 관련 선행연구
Table 1. An empirical study of a chatbot
In general, chatbots are responsible for providing guidance,
answering questions, or facilitating specific actions as coaches or
colleagues (Table 1). In the educational context, the role of the
chatbot can be set in various ways, which can be divided into five
roles (see table 2). They are: tutors who guide and support the
learning process of individual learners; evaluators who check the
learner’s progress and diagnose performance; respondents who
answer learners’ questions; communicators who mediate
instructors and learners through interaction with learners; and
fellow learners who exchange everyday conversations.
Educational role
of chatbot
Details
Tutor
Provide individual and personalized support
Evaluator
Assess learner’s progress and performance
Responder
Answer questions related to learning task
Moderator
Be a communicating channel between instructor and
learner
Peer learner
Be an interlocutor for common dialogue and conversation
표
2.
교육 영역에서 챗봇의 역할
Table 2. Educational roles of a chatbot
2-2 Principles of Chatbot Design
The following should be considered when designing chomps
derived from Facebook (bot) [8], interoperability [12], and
Microsoft [17] design and development principles.
The principles in table 3 provide guidelines on how to interact
with chatbots from the UI or UX standpoint, but do not provide a
단국대학교 죽전캠퍼스 | IP:220.149.***.10 | Accessed 2020/07/09 11:12(KST)

Deriving Design Principles for Educational Chatbots from Empirical Studies on Human–Chatbot Interaction
489
http://www.dcs.or.kr
standard on the purpose for which it should be used. In order to
actively use a chatbot in an educational context, design and
development guidelines should be prepared from the viewpoint of
teaching and learning.
Category
Principles
Source
Consistency
Use the UI components of the chat platform
uniformly
[12]
Optimize for all users and usage
[12]
Shortening
Support a way to solve problems faster
[17]
Provide button and button-type replies to help
quick selection in limited circumstances
[8], [17]
Feedback
Minimize the waiting process and make the user
aware of the waiting state
[8], [17]
Provide notifications in appropriate situations
[17]
Conversation
Organize the flow of words and contexts
naturally, and maintain the standards of dialogue
[8], [12],
[17]
Ask your questions carefully and check your
intentions
[8]
Provide appropriate humor
[8]
Problem response
Provide opportunities to respond to failures
[8], [17]
Provide the ability to go back and cancel
[8], [12]
Recognition
Let users know clearly how to use chatbot
[8]
Make intuitive awareness of the chatbot’s UI
components
[8], [17]
표
3.
챗봇 개발을 위한 설계 원리의 예
Table 3. Chatbot design principles (example)
NOTE: [8] Facebook, [12] Intercom, [17] Microsoft
Hints for deriving chatbot design principles can be found in the
Conversational Agents (CA) study. Traditional research was
mainly on agent support, voice, and appearance (see table 4). The
research that is required for the future is empirical and qualitative
study of the change due to the agent’s participation, and research
into the role of the agent.
Principle
Contents
Reference
Personification
principle
The learner learns better when the agent is
represented by a personalized method rather
than a non-personalized method.
[4], [13]
Voice principle
The learner learns better when exposed to a
human voice method (human-voice
method) rather than a machine-voice
method.
Image principle
The learner learns better when the speaker’s
face appears on the screen (image-present
method) rather than when it does not appear
(no-image method).
표
4.
대화형 에이전트의 설계 원리에 대한 연구
Table 4. Research related to conversational agents
Ⅲ. Methodology
In order to establish an empirical ground from which to derive
design principles for educational chatbots, we first explored
previous chatbot studies and summarized their findings. From
there we extracted several implications for a chatbot design that is
suitable in an educational context. The review process began by
identifying the relevant research papers from Social Science
Citation Index (SSCI) and Science Citation Index Expanded
(SCIE) journals, which are of high quality and impact.
Conference proceedings and conceptual papers were excluded
from the search. Research papers published since 2005 were
collected using the keywords “conversational agent”, “chatbot”,
“pedagogical agent”, “conversational system”, “dialog system”,
“chatterbot”, “chat bot”, “chat-bot”, and “intelligent pedagogical
agent”. After the search process, we screened the articles by
distinguishing empirical studies that focused on interactions
between humans and chatbots. A total of seven studies from six
articles were reviewed.
Ⅳ. Findings
4-1 Research question 1: How have chatbots been
incorporated into empirical studies on
human-chatbot interaction?
To answer the research question, we organized the review
findings into two sets; one sorted by chatbot feature and the other
by research variables and results. Basic information on each study
was included in the first set (see table 5). Of the seven studies
reviewed, all the researches were conducted under a higher
education setting except for that of Corti and Gillespie (2016) [6],
which was in an open setting, and that of van der Meij, van der
Meij, and Harnsen (2015) [21], at a secondary school. The articles
covered target knowledges in a varied range of disciplines such as
healthy eating behavior [3], the circulatory system [9],
instructional planning [14], and kinematics [21]. The chatbots
used in the studies also differed from each other.
The chatbot features examined in the studies were mostly
variations of delivery types (or representation types). They
included expressions made by chatbots (e.g., facial expression,
emotional expression, empathetic expression), the gender of the
chatbots (i.e., male and female), modality (e.g., voice, text), and
other representation types (e.g., head movement). A few studies
incorporated instructional features into chatbots by providing
prompts and feedback [9] and motivational scaffolding [21].
Ref.
no.
Setting
Participants
Context
Target
knowledge
Chatbot type
Chatbot
feature
[3]
Higher
education
144
-
Healthy
eating
behavior
Embodied
conversation
al
agent
(GRETA)
- Various
pr es en ta ti on
types
- Facial
expression
- Emotional
expression
- Modality
표
5.
챗봇의 역할에 따라 구분한 선행 연구
Table 5. Articles reviewed sorted by chatbot features
단국대학교 죽전캠퍼스 | IP:220.149.***.10 | Accessed 2020/07/09 11:12(KST)

디지털콘텐츠학회논문지
(J. DCS) Vol. 21, No. 3, pp. 487-493, Mar. 2020
http://dx.doi.org/10.9728/dcs.2020.21.3.487
490
Ref.
no.
Setting
Participants
Context
Target
knowledge
Chatbot type
Chatbot
feature
[6]
-
108 adults
Lab
experiment
-
Artificial
conversation
al
agent
(Cleverbot)
- Modality
[9]
Higher
education
123
undergrads
Meta tutor
learning
envir onme
nt
Ci rc ul at ory
system
Four
pedagogica l
agents
- Gavin the
guide
- Mary the
monitor
- Pam the
planner
- Sam the
strategizer
- Prompt
and
feedback
[14]
Higher
education
142
college
students
Co mp ut er
literacy
course
Instructi onal
planning
Peda gogi cal
agent as a
learning
co mp an io n
(PAL)
- Gender
difference
- Emotional
expression
[14]
Higher
education
56
pre-service
teachers
Course in
introductor
y
educational
technology
Instructi onal
planning
Peda gogi cal
agent as a
learning
co mp an io n
(PAL)
- Gender
difference
- Empathic
expression
[18]
Higher
education
60
undergrads
Common
dialogue
-
E m bo d i ed
conversation
al
agent (ECA)
- Facial
expression
- Head
movement
[21]
Secondary
school
61
third-years
Inquiry
learning
Kinematics
Animated
pedagogica l
agent (APA)
- Motivational
scaffolding
- Modality
The major findings of the studies are listed in table 6. Overall,
the results showed a tendency for participants to project their
human-to-human interaction practices to their human-to-chatbot
interaction, especially when the chatbot was designed to be more
human-like. In detail, participants report more positive outcomes
when the chatbots express or represent emotion than when they
interact with chatbots designed to exhibit neutral emotion [3, 14,
18]. They also exhibited social stereotyping towards a gendered
chatbot [14]. In cases of modality, though the results were not
perfectly consistent, participants seemed to better understand a
text-based chatbot than a speaking chatbot [3], while they showed
more human-like interaction with the latter [6, 18].
4-2 Research question 2: What implications for
educational chatbots can be derived from the
studies?
From the review, we reorganized the findings with similar
attributes and characteristics. Explanations for each attribute were
Ref.
no.
Intervention
Dependent variable
Result
[3]
Presentation type
- Neutral expression
- Neutral expression
(human)
- Voice only
- Text only
- Consistent expression
- Inconsistent expression
Perception
- Likelihood of
following
- Ease of
understanding
- Trustworthiness
- Helpful
- Likeable
- Quality of
evidence
- Convincingness
Memory
Ease of understanding
- Text >
Neutral, human, voice
Trustworthiness
- Neutral, text, voice >
Human
Helpful
- Neutral, human >
Voice
Likeable
- Neutral, human >
Voice
표
6.
챗봇 설계와 연관된 선행 연구 결과 요약
Table 6. Summary of results in articles
Ref.
no.
Intervention
Dependent variable
Result
performance
Memory performance
- Voice, human, text >
Neutral
- Consistent >
Neutral, inconsistent
[6]
Screen
- Text
- Voice
Aware
- Participants were
informed that their
interlocutor is a chatbot
- Not informed
Intersubjective
effort
- Voice > Text
- Informed > Not informed
[9]
- Prompt and feedback
- No prompt and no
feedback
-Achievement
emotions
- Personality
- Agent response
- Pre-test
- Post-test
- Relationship between trait
emotions (anger, anxiety)
and personality
(agreeableness,
conscientiousness,
neuroticism)
for agent-directed emotion
(enjoyment, pride, boredom,
neutral)
- No significant relationship
between personality and trait
emotion on learning gain
[14]
Emotional expression
- Positive
- Negative
- Neutral
Gender of agent
- Male
- Female
- Social judgement
- Interest
- Self-efficacy
- Learning
Social judgement
- Positive, neutral > Negative
- Positive male > Positive
female
Interest
- Positive male > Positive
female
Learning
[14]
Empathetic response
- Responsive
- Nonresponsive
Gender of agent
- Male
- Female
- Social judgement
- Interest
- Self-efficacy
- Learning
Social judgement
- Male > Female
Interest
- Responsive >
Nonresponsive
Self-efficacy
- Responsive >
Nonresponsive
[18]
Interaction mode
- Written input
- Spoken input
Subject groups
- Science
- Humanities
User attitude
- Spoken input produces a
warmer attitude and richer
language use
- This effect is more evident
in the Humanities group
[21]
Time
- Pre-intervention
- During intervention 1
- During intervention 2
- After intervention
Condition
- Visible agent with
voice
- Voice only
- No agent
Student gender
- Boy
- Girl
- Task relevance
change
- Self-efficacy over
time
- Agent appraisal
- Pre-test
- Post-test
Self-efficacy
- Boy > Girl
- No main effect for
condition
Agent appraisal
- Girl > Boy
Learning
- Condition & gender fixed,
students made significant
progress over time
- Benefits of agent group
over control group is
doubtful
단국대학교 죽전캠퍼스 | IP:220.149.***.10 | Accessed 2020/07/09 11:12(KST)

Deriving Design Principles for Educational Chatbots from Empirical Studies on Human–Chatbot Interaction
491
http://www.dcs.or.kr
then elaborated in the learning context. The implications are as
follows.
ŸLive emotion – chatbots are better when designed to display
consistent facial expressions or positive emotional
expressions.
ŸNeutral emotion – a chatbot with a neutral emotional
expression is more acceptable for persuasion.
ŸModality – written text is better for delivering information or
a guiding process; spoken text is better for affective support.
ŸExtraneous – too many animated or visual graphics have a
detrimental effect on performance.
ŸGender – people project social gender stereotyping according
to the chatbot’s gender; people value information from a
chatbot differently, depending on its gender representation.
ŸBot effect – a chatbot can perform works that are redundant
and require accuracy better than a human can.
ŸImitation – more human-like chatbots drive more human-like
interactions and establish a trusting relationship when giving
information.
After extracting the implications, they were matched with each
role of the educational chatbot (i.e., tutor, assessment, question
and answer, communication, common dialogue); see table 7.
Ⅴ. Discussion
This study derives design principles according to the role of a
chatbot by using a systematic review of recently published
literature on educational chatbots. This approach can be expected
to help in the design and development of educational chat-bots in
situations where there is insufficient chatbot development and
related research in an educational context. The findings of this
study can be summarized as follows.
5-1 Key result
In order to derive design principles for educational chatbots,
the seven studies examined in this study examined how
appearance characteristics such as facial expressions, gender, and
style of chatbot affect the learning process and performance. As a
result, when the chatbot expresses emotionally rather than
neutrally, text-based rather than speech-based human interactions
contribute more to learning. The design principles derived from
this are the Live emotion principle, Neutral emotion principle,
Modality principle, Extraneous principle, Gender principle, Bot
effect principle, Imitation principle, and so on. In addition, this
study matched design principles to be considered according to the
role of chatbot when designing an educational chatbot. When
designing a chatbot that plays the role of a tutor, it is necessary to
consider the Live emotion principle, Modality principle, and
Extraneous principle. When designing a chatbot that acts as an
evaluator, the Bot effect principle should be considered. When
developing a chatbot that acts as a responder, the Gender
principle and Modality principle should be considered. In the case
of a chatbot that plays the role of a moderator, it is necessary to
consider the Neutral emotion principle, and in the case of a
chatbot that plays the role of peer learner, the Modality principle
(voice), the Imitation principle, and the Neutral emotion principle
should be considered. In this study, we explored some principles
for educational chatbots based on previous studies, but most of
them were related to the appearance characteristics of chatbots. In
the future, research is needed on the contents presentation method
of chatbots and differentiated roles.
5-2 Areas for further study
As mentioned above, there are relatively few studies on the
principles to be considered in the design of educational chatbots
and the appropriate design principles according to the role of the
chatbots. Related research needs to be actively conducted in the
future, and research on suitable design principles is required
according to the purpose and role of the chatbot.
Prior studies have found that it is difficult to find consensus on
the characteristics of educationally effective chatbots, but learners
Educational role of
chatbot
Implication from the studies
Tutor
Live emotion – chatbots are better when designed to
display consistent facial expressions or positive
emotional expressions
Modality – written text is better for delivering
information or a guiding process; spoken text is better for
affective support
Extraneous – too many animated or visual graphics have
a detrimental effect on performance
Evaluator
Bot effect – a chatbot can perform works that are
redundant and require accuracy better than a human can
Responder
Gender – people project social gender stereotypes to the
chatbot’s gender; people value information from a
chatbot differently, depending on its gender
representation
Modality; text – written text is better for delivering
information
Moderator
Neutral emotion – a chatbot with a neutral emotional
expression is more acceptable for persuasion and
establishing a trusting relationship than for giving
information
Peer learner
Modality; voice – spoken input produces a warmer
attitude and richer language use
Imitation – more human-like chatbots drive more
human-like interaction
Neutral emotion – a chatbot with a neutral emotional
expression is more acceptable for persuasion and
establishing a trusting relationship than for giving
information
표
7.
도출된 교육용 챗봇 설계 원리
Table 7. Implication from review for educational chatbot
단국대학교 죽전캠퍼스 | IP:220.149.***.10 | Accessed 2020/07/09 11:12(KST)

디지털콘텐츠학회논문지
(J. DCS) Vol. 21, No. 3, pp. 487-493, Mar. 2020
http://dx.doi.org/10.9728/dcs.2020.21.3.487
492
want to learn with more human and emotional chatbots. Although
this may be beneficial in terms of motivation, further research is
needed to determine whether it will have significant effects on
learning outcomes. In addition, it is necessary to study the
differences between education through chatbots and through other
educational methods, and in short- and long-term settings.
It is also necessary to study how the role of the instructor and
how the interaction between the instructor and the learner is
changed by the educational use of the chatbot. Research is also
required on the side effects of using chatbots and the degree of
acceptance according to learners’ characteristics; for example,
study of how the chatbot’s performance varies according to a
learner’s ability to use a computer, propensity to cooperate,
learning style, and learning level. There is also a need for research
on the cost-effectiveness of educational use. It is also necessary to
discuss which educational context is the most effective when a
chatbot is used for any educational purpose, and that from a
cost-effectiveness analysis it is worth introducing a chatbot.
5-3 Limitation
This study has some limitations. First of all, although some
papers have educational contexts, they include cases that are not
for educational purposes, so it is hard to say that they derive
principles entirely for educational chatbots. Since this study did
not examine the gray literature, such as theses, current research,
academic journals, and research reports, there is a possibility of
publication bias. It is also difficult to avoid language bias because
it includes papers in English only. However, this study attempted
to study the special area of the educational chatbot, which was not
sufficiently examined in the past, and it is considered to have
sufficient advantages because it tried to derive differentiated
principles. In order to develop a chatbot with various purposes
and roles for educational purposes, it is necessary to make various
efforts with various experts.
Acknowledgements
This work was supported by National Research Foundation of
Korea Grant funded by the Korean Government(KRF-2019-S1A5
A8-036708)
References
[1] 6 Ways Artificial Intelligence and Chatbots Are Changing
Education. Chatbots magazine. Available:
https://chatbotsmagazine.com/six-ways-a-i-and-chatbots-ar
e-changing-education-c22e2d319bbf.
[2] Alavi, M., Leidner, D. E, “Research commentary:
Technology-mediated learning—A call for greater depth
and breadth of research”, Information systems research,
Vol. 12, No. 1, pp. 1-10, 2001.
[3] Berry, D., Butler, L., de Rosis, F, “Evaluating a realistic
agent in an advice-giving task”, International Journal of
Human-Computer Studies, Vol. 63, No. 3, pp. 304-327,
2005.
[4] Bodemer, D., Ploetzner, R., Feuerlein, I., Spada, H, “The
active integration of information during learning with
dynamic and interactive visualisations”, Learning and
Instruction, Vol. 14, No. 3, pp. 325-341, 2004.
[5] Chatbots Infographic - Key Statistics 2017. Available:
https://www.bevytechnologies.com/infographic-chatbots-k
ey-statistics-2017.
[6] Corti, K., Gillespie, A, “Co-constructing intersubjectivity
with artificial conversational agents: People are more
likely to initiate repairs of misunderstandings with agents
represented as human”, Computers in Human Behavior,
Vol. 58, pp. 431-442, 2016.
[7] Eisman, E., López, V., Castro, J, “A framework for
designing closed domain virtual assistants”, Expert
Systems with Applications, Vol. 39, No. 3, pp. 3135-3144,
2012.
[8] Facebook for developers: Design Principles. Available:
https://developers.facebook.com/docs/messenger-platform/
introduction/general-best-practices/
[9] Harley, J., Carter, C., Papaionnou, N., Bouchet, F., Landis,
R., Azevedo, R., Karabachian, L, “Examining the
predictive relationship between personality and emotion
traits and stu-dents’ agent-directed emotions: towards
emotionally-adaptive agent-based learning environments”,
User Modeling and User-Adapted Interaction, Vol. 26, No.
2-3, pp. 177-219, 2016.
[10] Hasler, B., Tuchman, P., Friedman, D, “Virtual research
assistants: Replacing human interviewers by automated
avatars in virtual worlds”, Computers in Human Behavior,
Vol. 29, No. 4, pp. 1608-1616, 2013.
[11] Hayashi, Y., Ono, K, In 2013 IEEE International Workshop
on Robot and Human Interactive Communication, pp.
120-125. IEEE Press, Roman, 2013.
[12] Intercom: Principles of Bot design: Inside Intercom.
Available: https://blog.intercom.io/principles-bot-design/
[13] Kester, L., Kirschner, P., van Merriënboer, J, “The
management of cognitive load during complex cognitive
skill acquisition by means of computer-simulated problem
solving”, British Journal of Educational Psychology, Vol.
75, No. 1, pp. 71-85, 2005.
[14] Kim, Y., Baylor, A., Shen, E, “Pedagogical agents as
단국대학교 죽전캠퍼스 | IP:220.149.***.10 | Accessed 2020/07/09 11:12(KST)

Deriving Design Principles for Educational Chatbots from Empirical Studies on Human–Chatbot Interaction
493
http://www.dcs.or.kr
learning companions: the impact of agent emotion and
gender”, Journal of Computer Assisted Learning, Vol. 23,
No. 3, pp. 220-234, 2007.
[15] Louvet, J., Duplessis, G., Chaignaud, N., Vercouter, L.,
Kotowicz, J, “Modeling a collaborative task with social
commitments”, Procedia Computer Science, Vol. 112, pp.
377-386, 2017.
[16] Mell, J., Lucas, G., & Gratch, J, “An effective conversation
tactic for creating value over repeated negotiations”, In: the
2015 International Conference on Autonomous Agents and
Multiagent Systems, pp. 1567-1576. International
Foundation for Autonomous Agents and Multiagent
Systems, Istanbul, 2015.
[17] Microsoft: Principles of bot design. Available:
https://docs.microsoft.com/en-us/azure/bot-service/bot-ser
vice-design-principles?view=azure-bot-service-4.0
[18] Novielli, N., de Rosis, F., Mazzotta, I, “User attitude
towards an embodied conversational agent: Effects of the
interaction mode”, Journal of Pragmatics, Vol. 42, No. 9,
pp. 2385-2397, 2010.
[19] Rouse, M, “What is chatbot?” [Online]. Available:
https://searchcustomerexperience.techtarget.com/definition
/chatbot.
[20] Turunen, M., Hakulinen, J., Ståhl, O., Gambäck, B.,
Hansen, P., Rodríguez Gancedo, M., de la Cámara, R.,
Smith, C., Charlton, D., Cavazza, M, “Multimodal and
mobile conversational Health and Fitness Companions”,
Computer Speech & Language, Vol. 25, No. 2, pp.
192-209, 2011.
[21] van der Meij, H., van der Meij, J., Harmsen, R, “Animated
pedagogical agents effects on enhancing student
motivation and learning in a science inquiry learning
environment”, Educational Technology Research and
Development, Vol. 63, No. 3, pp. 381-403, 2015.
[22] Xu, K., Lombard, M, “Persuasive computing: Feeling peer
pressure from multiple computer agents”, Computers in
Human Behavior, Vol. 74, pp. 152-162, 2017.
정효정(Hyojung Jung)
이진주(Jinju Lee)
박채연(Chaeyeon Park)
단국대학교 죽전캠퍼스 | IP:220.149.***.10 | Accessed 2020/07/09 11:12(KST)