Chatbots for language learning-Are they really useful? A systematic review of chatbot-supported language learning

Abstract

Background:
The use of chatbots as learning assistants is receiving increasing attention in language learning due to their ability to converse with students using natural language. Previous reviews mainly focused on only one or two narrow aspects of chatbot use in language learning. This review goes beyond merely reporting the specific types of chatbot employed in past empirical studies and examines the usefulness of chatbots in language learning, including first language learning, second language learning, and foreign language learning.

Aims:
The primary purpose of this review is to discover the possible technological, pedagogical, and social affordances enabled by chatbots in language learning.

Materials & Methods:
We conducted a systematic search and identifies 25 empirical studies that examined the use of chatbots in language learning. We used the inductive grounded approach to identify the technological and pedagogical affordances, and the challenges of using chatbots for students’ language learning. We used Garrison’s social presence framework to analyze the social affordances of using chatbots in language learning.

Results:
Our findings revealed three technological affordances: timeliness, ease of use, and personalization; and five pedagogical uses: as interlocutors, as simulations, for transmission, as helplines, and for recommendations. Chatbots appeared to encourage students’ social presence by affective, open, and coherent communication. Several challenges in using chatbots were identified: technological limitations, the novelty effect, and cognitive load.

Discussion and Conclusion:
A set of rudimentary design principles for chatbots are proposed for meaningfully implementing educational chatbots in language learning, and detailed suggestions for future research are presented.

Full text

REVIEW ARTICLE
Chatbots for language learning—Are they really useful?
A systematic review of chatbot-supported language learning
Weijiao Huang | Khe Foon Hew | Luke K. Fryer
Faculty of Education, The University of Hong
Kong, Hong Kong, China
Correspondence
Weijiao Huang, Faculty of Education, The
University of Hong Kong, Hong Kong, China.
Email: wjhuang1@connect.hku.hk
Funding information
University of Hong Kong Teaching
Development Grant 2019, Grant/Award
Number: 730
Abstract
Background: The use of chatbots as learning assistants is receiving increasing atten-
tion in language learning due to their ability to converse with students using natural
language. Previous reviews mainly focused on only one or two narrow aspects of
chatbot use in language learning. This review goes beyond merely reporting the spe-
cific types of chatbot employed in past empirical studies and examines the usefulness
of chatbots in language learning, including first language learning, second language
learning, and foreign language learning.
Aims: The primary purpose of this review is to discover the possible technological,
pedagogical, and social affordances enabled by chatbots in language learning.
Materials & Methods: We conducted a systematic search and identifies 25 empirical
studies that examined the use of chatbots in language learning. We used the induc-
tive grounded approach to identify the technological and pedagogical affordances,
and the challenges of using chatbots for students’language learning. We used Garri-
son’s social presence framework to analyze the social affordances of using chatbots
in language learning
Results: Our findings revealed three technological affordances: timeliness, ease of
use, and personalization; and five pedagogical uses: as interlocutors, as simulations,
for transmission, as helplines, and for recommendations. Chatbots appeared to
encourage students’social presence by affective, open, and coherent communica-
tion. Several challenges in using chatbots were identified: technological limitations,
the novelty effect, and cognitive load.
Discussion and Conclusion: A set of rudimentary design principles for chatbots are
proposed for meaningfully implementing educational chatbots in language learning,
and detailed suggestions for future research are presented.
KEYWORDS
affordances, chatbot usefulness, chatbots, language learning
1|INTRODUCTION
A chatbot is a dialogue software program that can interact with users and
process their inputs using natural language. More than half a century ago,
the first chatbot, ELIZA, was developed by Joseph Weizenbaum (1966).
Current uses of chatbots include information delivery and answering
frequently asked questions (Smutny & Schreiberova, 2020). Recently, we
have witnessed the uses of chatbots in educational contexts, such as
maintaining students' motivation in scientific learning (Chen & Chou,
2015), supporting freshmen's adaptation to university life
(Carayannopoulos, 2018), and helping instructors manage large in-class
activities (Schmulian & Coetzee, 2019).
Received: 22 December 2020 Revised: 22 June 2021 Accepted: 22 August 2021
DOI: 10.1111/jcal.12610
J Comput Assist Learn. 2021;1–21. wileyonlinelibrary.com/journal/jcal © 2021 John Wiley & Sons Ltd 1

1.1 |A niche for chatbots in language learning
Chatbots have caught the attention of language teaching researchers
due to their capacity to communicate with users in the target lan-
guage (Fryer et al., 2019; Jia, Chen, Ding, & Ruan, 2012; Tegos,
Demetriadis, & Karakostas, 2015). Chatbot-supported language learn-
ing refers to the use of a chatbot to interact with students using natu-
ral language for daily language practice (e.g., conversation practice;
Fryer et al., 2017), answering language learning questions (e.g.,
storybook reading; Xu, Wang, Collins, Lee, & Warschauer, 2021) and
conducting assessment and providing feedback (e.g., vocabulary test;
Jia et al., 2012). With the help of visual chatbot development plat-
forms, teachers can create chatbots by themselves without prior pro-
gramming experience. For instance, Dialogflow from Google enables
users to customize conversational contents by adding pre-set data-
bases. BotStar, an online chatbot platform, allows users to drag and
drop conversational flows using a design dashboard, by which
teachers can script students' learning experience with the intended
learning objectives. Recently, Artificial Intelligence and Machine
Learning techniques can enhance chatbots' ability to adapt to end-
users' unstructured inputs.
Active dialogue practice and sufficient immersion in language
learning contexts are critical drivers of learners' communication com-
petence and language proficiency. However, language teachers are
often challenged by the unwillingness of many students to communi-
cate in their second or foreign language. Chatbot researchers have
suggested that a more interactive and authentic language environ-
ment enabled by chatbot-supported activities can improve student
language learning outcomes (Lu, Chiou, Day, Ong, & Hsu, 2006;
Wang, Petrina, & Feng, 2017). Fryer and Carpenter (2006) highlighted
the potential of chatbots to diminish the shyness that students may
feel during language practice compared with talking with a human
partner. Chatbots may also reduce the transactional distance between
learners and instructors in an online learning space. According to
Moore's (1993) theory of transactional distance, there is a psychologi-
cal and communication gap between the instructor and the learner in
an online learning space, which creates room for potential misunder-
standing. If the transactional distance is reduced, learners are more
likely to feel satisfied with their learning environment. Chatbots can
help reduce the transactional distance by providing a dialogue for the
learner to interact with the course content.
Educational chatbots in language learning contexts can generally
be identified by the following three common features. First, they are
available to support students 24/7 (Garcia, Fuertes, & Molas, 2018).
Students can practise their language skills with chatbots anytime they
like, which a human partner could not easily do (Haristiani, 2019;
Winkler & Soellner, 2018). Second, chatbots can provide students
with broad language information that human language partners may
lack. Given the fact that most EFL/ESL students and their peers are at
a similar target language proficiency level, learners may not be able to
provide extra language knowledge to their peers (Fryer et al., 2019). A
well designed chatbot, however, could provide extra information such
as a broad range of expressions, questions, and vocabulary. Third,
chatbots can play the role of a tireless assistant, freeing humans from
repetitive work (Fryer et al., 2019; Kim, 2018b) such as answering fre-
quently asked questions and sustaining language practice. Chatbots as
learning partners are willing to communicate with students endlessly,
which offered students the chance to continuously practise the new
language.
1.2 |Critical appraisal of using chatbots in
language learning
However, despite the potential of chatbots to reduce students' anxi-
ety (Ayedoun, Hayashi, & Seta, 2015, 2019; Bao, 2019) and engage
them in language learning (Ruan et al., 2019), the novelty effect of
chatbots has been mentioned as a possible reason why learners'
engagement and performance improvement are only short-term
(Fryer et al., 2019; Ayedoun et al., 2019). “The novelty effect”refers
to the newness of a technology to students, which disappears after
students become more familiar with the technology.
An additional worry that has been expressed about the use of
chatbots is their limited capabilities, despite the exponential increase
of chatbot implementation in educational contexts (Smutny &
Schreiberova, 2020). Designing intelligent dialogue in chatbots is chal-
lenging for software developers despite the advancement of artificial
intelligence (Brandtzaeg & Følstad, 2018). For instance, if students
misspell their inputs, they may receive irrelevant responses from the
chatbot. A chatbot with low intelligence cannot fulfil students'
requests and thereby may provide unrelated answers
(Haristiani, 2019; Lu et al., 2006). Students' interaction may be
restricted to the pre-set knowledge base (Grudin & Jacques, 2019).
Another limitation is the chatbot's inability to understand multiple
sentences at once (Kim, Cha, & Kim, 2019), which is unlike human-
human interaction in a real language learning context.
To more effectively implement chatbot use, it is crucial to know
how chatbots have been used for current language learning and what
improvements might be incorporated into future chatbot-supported
language learning environments.
1.3 |Rationale for the current review
Several recent articles have reviewed the use of chatbots in language
learning (Fryer et al., 2020; Haristiani, 2019; Kim et al., 2019).
Although these articles have undoubtedly increased our understand-
ing of chatbot use in language learning, they have mainly focused on
only one or two narrow aspects of chatbot use. Haristiani (2019), for
instance, reviewed the different types of chatbots used in language
learning, and found that Cleverbot was the main chatbot used. Kim
et al. (2019) similarly reviewed and reported on the different types of
chatbots used in language learning, and found that few chatbot pro-
grams allowed chatbots and humans to directly interact via voice rec-
ognition systems or texting for the purpose of learning foreign
languages. Fryer et al. (2020) reported on two current developers of
2HUANG ET AL.

chatbots (“Cleverbot”,“Mondly”), and provided some suggestions on
how the two chatbots could be structured to make the technology
more useful to foreign language learners.
Unlike the previous reviews, the current review goes beyond
merely reporting the specific types of chatbot employed in past empiri-
cal studies. It empirically examines the possible technological, pedagogi-
cal, and social affordances associated with chatbots in language
learning through the lens of the usefulness theoretical perspective
(Kirschner, Strijbos, Kreijns, & Beers, 2004, see following section for
detail). This could help educators better understand how chatbots have
actually been used in language learning, and their benefits or chal-
lenges, as well as suggestions to deal with these challenges.
1.4 |Usefulness theoretical framework
This paper examines the possible technological, pedagogical, and
social affordances associated with chatbots in language learning. This
is accomplished by taking a usefulness theoretical perspective
(Kirschner et al., 2004) to analyse the utility and usability of chatbots
for teaching and learning purposes in language learning contexts.
Usability is concerned with whether a system enables users to
accomplish a set of tasks in an easy and efficient way that satisfies
the user (Kirschner et al., 2004). Utility refers to the functionality of a
system—whether it provides the functions that users need. Taken
together, both the usability and utility of a system will determine how
useful the system is to the user (Kirschner et al., 2004).
Utility is determined by the pedagogical and social affordances of
the tool, whereas usability is determined by the tool's technological
affordances. Affordance can be defined as the action qualities of an
object or environment that enable people to physically behave in a
certain way, such as how a chair affords users the opportunity to sit
(Gibson, 1977).
In other words, technological, pedagogical, and social affordances
drive the design of the learning environment (Kirschner et al., 2004).
By leveraging these three types of affordances, we can examine the
usefulness of educational chatbots in language learning contexts
(Figure 1). Technological affordance refers to the design features of a
technology that enable students to accomplish the desired learning
tasks easily (Kirschner et al., 2004). Pedagogical affordances can be
defined as the characteristics of a tool that determine if and how a
particular learning behaviour could possibly be enacted within a given
context (Kirschner et al., 2004). For example, the characteristics of a
learning tool for teaching and learning activities can determine if and
how individual and group-based learning can take place. Social
affordances, as portrayed in Kirschner et al. (2004), are the functions
of a tool that facilitate social interaction among participants. One key
feature that can determine the extent of social interaction is the tech-
nological tool's ability to support social presence among the different
participants (Tu, 2000). Social presence can be defined as the feeling
that the interactants in an online space are real people (Garrison,
Anderson, & Archer, 1999). Hence, in this study, we characterize
social affordances as the potential of chatbots to promote social pres-
ence in communication with the learners.
1.5 |Research questions
This systematic review focused on the current use of chatbots in lan-
guage learning, with the main purpose of discovering the possible
pedagogical, technological and social affordances enabled by chatbots.
The following research questions were addressed:
Research question 1: In what contexts have chatbots been used
in language learning?
Research question 2: What are the technological affordances, if
any, of using chatbots in language learning?
Research question 3: What are the pedagogical affordances, if
any, of using chatbots in language learning?
Research question 4: What are the social affordances, if any, of
using chatbots in language learning?
Research question 5: What are the challenges, if any, of using
chatbots in language learning?
2|METHODOLOGY
2.1 |Literature search process
We conducted a systematic search within the following major aca-
demic databases of published educational research: all EBSCOhost
databases (e.g., ERIC), the Web of Science, Scopus, ProQuest. Addi-
tionally, we employed Google Scholar as a search engine to search for
potential sources that may not be included in the academic databases.
During the preliminary search, “chatbot* AND language learning”
were used as keywords in all the databases and search engine. As
studies could interchangeably apply “chatterbot OR conversational
agent OR pedagogical agent”instead of “chatbot”and “language
acquisition OR language teaching”instead of “language learning,”we
included these additional terms in the second round of search. Peer-
reviewed journal articles were first examined because these papers
had been evaluated by reviewers who were generally considered
experts in their respective disciplines (Korpershoek, Harms, de Boer,
van Kuijk, & Doolaard, 2016), thus ensuring some form of quality
check (Sulosaari, Suhonen, & Leino-Kilpi, 2011).
FIGURE 1 Conceptual framework from an affordance perspective
(adapted from Kirschner et al., 2004, p. 52)
HUANG ET AL.3

However, we also included conference proceedings (if any)
due to the emerging nature of this field to obtain the most up-to-
date information about the chatbots' implementation. In addition,
according to the Cochrane Handbook for Systematic Reviews of
Interventions, searching for conference proceedings is a highly
desirable practice because it can help capture as many studies as
possible, and minimize the risk of publication bias (Lefebvre
et al., 2021). Peer-reviewed journal articles, as well as conference
proceeding papers, were selected if they exhibited academic rig-
our in reporting empirical data on students' language learning
using chatbots. For example, articles must clearly report the types
of student outcomes that were examined. In addition, if the article
was a quasi-experiment, it must report the statistical values
(e.g., pvalue).
By the end of Oct 2020, two rounds of literature search had
yielded 2261 papers that met all of the inclusion criteria (Table 1).
Figure 2 presents the literature searching and selection process, which
was in accordance with PRISMA guidelines (Moher et al., 2009). After
deleting duplicates, reviewing abstracts, and reading full-text papers,
we identified 25 eligible studies for this review.
TABLE 1 Inclusion and exclusion criteria
Inclusion Exclusion
The study must be conducted
with a chatbot
Any study on other tools for
language learning (e.g., physical
robots, multimedia platforms).
The study must be conducted
in the field of language
learning.
Any study about computer
language learning (e.g., natural
language processing,
programming).
The study must include
empirical data.
Any study that proposes only the
design, development, or
evaluation of a chatbot but does
not describe in detail how the
chatbot was used to support
language learning.
The study must be conducted
in educational settings for
educational purposes.
Any study involving students as
participants that merely reports
about instruction activities. No
details were given about the
chatbot used.
The study must be written in
the English language.
Studies written in any other
language.
Records identied through database searching (n = 2,261)
EBSCOhost (n = 268)
Web of Science (n = 38)
Scopus (n = 67)
ProQuest (n = 122)
Google Scholar (n = 1 ,766)
Records after duplicates removed (n = 2,153)
Records screened (n = 2,153)
Records excluded after title and abstract
screening (n = 2,075)
not about chatbots (e.g., speech
recognition, fully immersive 3D
avatar-based game, and text mining)
not about language learning (e.g.,
computer language learning)
Full-text articles assessed for
eligibility (n = 78)
Full-text articles excluded (n = 53), for
the following reasons
not about educational chatbots for
language learning
lack of empirical data on students’
language learning (e.g., design and
development of chatbots, evaluation of
chatbot performance)
Studies identied for systematic
review (n = 25)
Identication Screening Eligibility Included
FIGURE 2 The process of
literature searching and selecting
4HUANG ET AL.

2.2 |Data analysis
To answer RQ1—in what contexts (e.g., country, language domain)
have chatbots been used in language learning—we analysed the
descriptions of the chatbot implementations in the 25 studies and
reported the relevant information such as the country or region where
the research took place, the language being taught, the educational
context, the chatbot interface design, and the learning mode between
the chatbot and students (see Table 2).
To answer RQ2, RQ3, and RQ5—what are the technological, and
pedagogical affordances, and the challenges of using chatbots in lan-
guage learning—we used the inductive grounded approach (Braun &
Clarke, 2006) to identify and categorize the major relevant themes. The
unit of analysis was each individual empirical study. The coding scheme
was not predetermined prior to our analysis but emerged inductively
and was continually refined through our interaction with the data. The
two examples below illustrate how the data were analysed and coded.
The first example was taken from the study by Ruan et al. (2019)
which reported the use of a voice-based chatbot called BookBuddy
that asked a child for basic information such as name, gender, and
interests (e.g., animals, gardens). Then the underlying recommendation
algorithm in the chatbot would find the most appropriate book in a
book database for the child to read. The example described here was
coded as the pedagogical activity of “providing recommendation”
because the most salient element appeared to be the chatbot using a
recommender system to suggest relevant books for a child.
The second example was taken from a study by Fryer et al.,
(2017), which demonstrated that students' interest in interacting with
a chatbot partner significantly declined over time due to a novelty
effect. This apparent novelty effect, however, did not occur when stu-
dents interacted with a human partner. The example described here
was coded as a challenge of using chatbot called “novelty effect”
because this was the explanation provided by the study researchers.
To answer RQ4—what are the social affordances of using chatbots
in language learning—we used Garrison's social presence framework
(Garrison, 2011; Garrison et al., 1999) to guide our initial analysis and
coding. According to Garrison (2011), the classification scheme for
social presence consists of three categories, namely interpersonal com-
munication, open communication, and cohesive communication. Inter-
personal communication (e.g., the expression of emotion, self-
disclosure, use of humour) creates an academic climate and sense of
belonging to students' purposeful communication. Open communica-
tion (e.g., asking questions, expressing agreement and appreciation) is
for a trustful learning environment that enables students to question
each other while protecting self-esteem and acceptance. Cohesive
communication (e.g., using vocatives, greetings and closures) is to build
students' group identify and sustain a collaborative online learning envi-
ronment. Although Garrison's framework was used a priori, we did not
forcefully impose any of the indicators onto our data corpus. During
the course of our analysis, we also allowed for new types of indicators
(if any) to emerge inductively during the coding process.
All articles were firstly read in its entirety and coded into themes
by the first author. To ensure the reliability of the data analysis, seven
studies (28% of all eligible studies) were randomly chosen and a
trained coder was involved to code the information. The inter-coder
agreement was 86%. Disagreements were resolved by discussion
between the first author and the trained coder until the consent was
reached.
3|RESULTS
3.1 |In what contexts have chatbots been used in
language learning?
Table 2 presents the contextual information of all 25 reviewed
studies.
3.1.1 | Geographic distribution
The geographic distribution of studies indicated that 18 studies were
conducted in Asia, five in North America, one in Ireland, and one in
Ukraine. One study (Yin & Satar, 2020) was considered cross-continental
because it was conducted in both China and the United Kingdom.
3.1.2 | Language
The results showed that English was the dominant language in the use
of chatbots for students' language learning. Among 23 studies involv-
ing chatbots used for English learning, three of them were for lan-
guage literature for native English speakers (Lin & Chang, 2020; Xu
et al., 2021; Xu & Warschauer, 2020); the other 20 studies involved
the teaching of English as a foreign language (EFL) or second language
(ESL). One study involved the teaching of Chinese as a second lan-
guage. One study involved the teaching of Irish as an endangered lan-
guage to native students in Ireland, where the national language was
changed to English.
3.1.3 | Domains
Chatbots were implemented to assist students' learning of speaking,
listening, reading, and writing skills. The learning topics covered
vocabulary, grammar (e.g., verb tense), academic pragmatics, and
meaning negotiation strategies (e.g., checking comprehension during
dialogue). For example, in one study, students conversed with a
chatbot on diverse topics (e.g., business, the environment) for more
than 10 min per session, as homework (Kim, 2018b).
3.1.4 | Educational settings
The use of chatbots for language learning was concentrated in higher
education. A total 19 of the 25 studies were conducted with university
HUANG ET AL.5

TABLE 2 Summary of 25 articles reviewed
First author (year) Country/region Language: Domain
Edu.
context Chatbot name Chatbot interface Development Capability
Learning
mode
Ayedoun et al. (2015) Japan EFL: Speaking HE Jack Web-based, human-like
avatar
Self-designed
system
TF, CD Individual
Ayedoun et al. (2019) Japan ESL: Speaking HE Peter Web-based, human-like
avatar
Self-designed
system
TF, CD Individual
Ayedoun (2020) Japan ESL: Speaking HE Peter Web-based, human-like
avatar
Self-designed
system
TF, CD Individual
Chen, Vicki Widarso, and
Sutrisno (2020)
Taiwan CSL: Vocabulary HE Xiaowen Mobile messenger, textual Self-designed
system
TF, CD Individual
Chiaráin (2016) Ireland Irish: Speaking SE Taidhgín Web-based, text-to-speech Self-designed
system
VC, UD Individual
Fryer (2017) Japan EFL: Speaking HE Cleverbot Web-based, speech-to-text Existing system VC, UD Individual
Fryer (2019) Japan EFL: Speaking HE Cleverbot Web-based, speech-to-text Existing system VC, UD Individual
Fryer (2020) Japan EFL: Speaking HE Cleverbot Web-based, speech-to-text Existing system VC, UD Individual
Gallacher, Thompson, and
Howarth (2018)
Japan EFL: Speaking HE Cleverbot Web-based, speech-to-text Existing system VC, UD Individual
Goda, Yamada, Matsukawa, Hata, and
Yasunami (2014)
Japan EFL: Speaking HE ELIZA Web-based Self-designed
system
TF, CD Individual
Hsu (2020) Taiwan EFL: Speaking HE n/a n/a n/a n/a Individual
Jia (2008) China EFL: Grammar SE CSIEC Web-based, textual and
auditory
Self-designed
system
VC, TF,
UD, CD
Individual
Jia et al. (2012) China EFL: Vocabulary SE CSIEC Web-based, textual and
auditory
Self-designed
system
TF, CD Individual
Kim (2016) Korea EFL: Negotiation of
meaning
HE Indigo Mobile messenger, auditory Existing system VC, UD Individual
Kim (2018a) Korea EFL: Vocabulary HE Elbot Mobile messenger, textual
and auditory
Existing system VC, UD Individual
Kim (2018b) Korea EFL: Listening &
Reading
HE Elbot Mobile messenger, textual
and auditory
Existing system VC, UD Individual
Kim et al. (2019) Korea EFL: Grammar HE Replika Mobile messenger, textual Existing system VC, UD Individual
Lin and Chang (2020) Canada English Writing HE DD Web-based, textual Self-designed
system
TF, CD Individual
Ruan, Willis (2019) United States EFL: Reading ECE BookBuddy Web-based, auditory Self-designed
system
TF, CD Individual
Tegos, Demetriadis, and Tsiatsos (2014) Ukraine EFL: Speaking HE MentorChat Web-based, textual Self-designed
system
TF, CD Group
Wang et al. (2017) China EFL: Grammar HE VILLAGE Web-based Self-designed
system
TF, CD Individual
6HUANG ET AL.

students, including undergraduate and postgraduate students. Four
studies involved elementary school students as the target learners; The
other three studies were conducted in a secondary education context.
3.1.5 | Chatbot interface design
The most common chatbot interface designs can be categorized into two
types: Web-based (Figure 3) and mobile messenger (Figure 4) interfaces.
When using Web-paged chatbots, students can access the conversation
via computers or tablets. Mobile messenger chatbots are integrated into
mobile instant messaging applications (e.g., Facebook Messenger,
WhatsApp, WeChat, and Line), through which students can talk with the
chatbots as if they were exchanging messages with a friend.
3.1.6 | Chatbot development and conversational
capability
Depending on the purpose of each individual study, researchers can
either use an existing conversational chatbot system (e.g., Cleverbot)
as a virtual companion to talk to students or design a task-focused
chatbot to perform specific learning activities. Virtual companion
chatbots provide open conversation with the user on any topic with
the primary purpose of engaging users in a continuous dialogue
(Grudin & Jacques, 2019). Students can direct the conversation topics
with this type of chatbot. Task-focused chatbots interact with a group
of target users on specific narrowed topics (Grudin & Jacques, 2019),
such as thesis writing (Lin & Chang, 2020) and restaurant reservation
(Ayedoun et al., 2015).”
3.1.7 | Learning modes
The majority of the chatbots were used for individual learning activities,
in which students communicated with one chatbot via an individual
channel and could not interact with each other synchronously in the
chatbot system. One chatbot, named MentorChat (Tegos et al., 2014),
communicated with a group of students in a discussion activity.
3.2 |What are the technological affordances, if
any, of using chatbots in language learning?
Content analysis of the current literature revealed three categories of
technological affordances: timeliness, ease of use, and personalization
(see Table 3).
3.2.1 | Timeliness
Students can receive immediate responses when they communicate
with chatbots. The accessibility of chatbots grants students the
TABLE 2 (Continued)
First author (year) Country/region Language: Domain
Edu.
context Chatbot name Chatbot interface Development Capability
Learning
mode
Xu (2020) United States English: Reading ECE n/a Auditory Self-designed
system
TF, CD Individual
Xu et al. (2021) United States English: Reading ECE n/a Auditory Self-designed
system
TF, CD Individual
Yang (2010) United States ESL: Speaking HE Dr Brown Web-based, human-like
avatar
Self-designed
system
TF, CD Individual
Yin (2020) China and United
Kingdom
EFL: Negotiation of
meaning
HE Tutor Mike,
Mitsuku
Web-based, human-like
avatar
Existing system VC, UD Individual
Abbreviations: CD, chatbot-driven interaction; CSL, Chinese as a second language; ECE, early children education; EFL, English as a foreign language; ESL, English as a second language; HE, higher education; SE,
secondary education; TF, task-focused chatbot; UD, user-driven interaction; VC, virtual companion chatbot.
HUANG ET AL.7

opportunity to learn the language at any time. For instance, the stu-
dents in Kim's (2018a) study practiced English with chatbots outside
class. Real-time interaction with chatbots can satisfy students' need
for self-learning pace (Chen et al., 2020) and offer students a sense of
authenticity in a native-speaking environment (Wang et al., 2017).
3.2.2 | Ease of use
Chatbots now are readily available in either web-page or mobile applica-
tion. Similar with browsing websites and using mobile applications, stu-
dents can interact with chatbots with ease. For example, students can
easily practice speaking with chatbots via sending text or voice messages
on a website (Fryer et al., 2017). Many chatbots now are available within
mobile instant messaging applications, through which students do not
need to download an extra application that may take up valuable space
on their phones. For example, Chen et al. (2020) integrated the chatbot
into LINE (i.e., a mobile instant messaging application), where students can
understand what they should do to connect with the chatbot without
instruction. The chatbot's ease-of-use interface has a positive effect on
students' perceived usefulness of language learning (Tegos et al., 2014).
3.2.3 | Personalization
Even when given a common topic to discuss with a chatbot, differ-
ent students can communicate with the chatbot differently, with dif-
ferent inputs. The ability of chatbots to respond with specific
information to students' previous utterances can personalize
students' learning. For example, students in Ruan and her colleagues'
(2019) study received recommended reading materials based on
their gender and interests by the chatbot BookBuddy. Similarly, Jia
et al. (2012) enabled a chatbot CSIEC to provide students English
chatting topics based on students' registration information, such as
the educational level and address.
3.3 |What are the pedagogical affordances, if any,
of using chatbots in language learning?
In this section, we analyse the pedagogical affordances of chatbots
at two levels: the specific pedagogical ways in which chatbots have
been used in language learning, and the effects of chatbot-
integrated language learning on students' behavioural and cognitive
outcomes.
The results indicated that chatbots have been employed in lan-
guage learning in five pedagogical ways: (a) as an interlocutor, (b) as
simulation, (c) to transmit information, (d) as a helpline, and
(e) providing recommendations. These are given in Table 4, along with
representative learning activities and examples. Among these,
chatbots are most commonly used as interlocutors to converse with
language learners.
3.3.1 | Interlocutor
This function emphasized the role of the chatbot as a learning com-
panion to assist students' language learning. Three subcategories of
FIGURE 3 Screenshot of Cleverbot
(an example of a web-based chatbot)
[Colour figure can be viewed at
wileyonlinelibrary.com]
8HUANG ET AL.

interlocution were revealed: (a) language knowledge practice activi-
ties, (b) learning skills facilitation activities, and (c) the coordination of
group discussion.
The first type of learning activity involved students interacting
with chatbots to facilitate the daily practice of targeted language
knowledge. Kim (2018b), for example, explored the use of chatbots in
TABLE 3 Technological affordances of chatbots in language learning
Affordance Example N
a
Sample studies
Timeliness The chatbot provided a real-time interaction with
students without time and space limitations.
25 Wang et al. (2017)
Xu (2020)
Ease of use Students accessed chatbots via computer, tablet, or
mobile without any technological difficulty.
10 Chen et al. (2020)
Lin and Chang (2020)
Personalization The chatting topics were chosen based on students
personalized information, such as gender, interest,
and educational levels.
6 Jia (2009)
Ruan et al. (2019)
a
The number of studies added up to more than 25 because there were several studies in which one chatbot was used in more than one particular way.
FIGURE 4 Screenshot of Replika (Kim, 2019) (an example of a messenger-based chatbot) [Colour figure can be viewed at
wileyonlinelibrary.com]
HUANG ET AL.9

an EFL course for freshmen students majoring in different subjects in
a Korean university; the chatbots were used to promote their vocabu-
lary learning. The students were randomly allocated to a treatment
group or a control group. Across an eight-week intervention period,
the students in the treatment group interacted with a messenger
chatbot for 10 min per week discussing diverse chat topics regarding
school life and movies; while no chatbot was used in the control
group. The students' pre- and post-test results revealed a significant
improvement only in the treatment group in terms of a change in their
vocabulary knowledge, specifically adjective and verb knowledge. In
the post-survey, students reported that they felt confident practicing
English vocabulary with the chatbot. In another example of a language
practice activity, 50 students in their first year of secondary school in
China were introduced to the CSIEC chatbot system to practise gram-
mar knowledge and sentence expression (Jia & Ruan, 2008). The
learning content in the CSIEC system was taken from English text-
books. The students interacted with the chatbot both during class
time and at home through gap-filling exercises. Once a student had
finished a practice session, the chatbot system rewarded the student
with a star badge as positive reinforcement. In the post-intervention
survey, the students reported that using the chatbot had benefited
their English learning. More than 75% of the students stated that they
wished to use the chatbot in the whole English instruction.
The second type of activity involved using chatbots to help stu-
dents learn certain skills such as critical thinking and negotiation skill.
An example of this can be found in Goda's study (2014), in which
students in the experimental group conversed with the Web-based
chatbot ELIZA for 10 min to prepare for a group discussion about the
topic “an ideal family,”while the control group searched for related
information online. During the pre-discussion preparation, ELIZA
required students to clarify their ideas or thoughts on the topic using
Socratic questions (e.g., “Why do you think that?”). The experimental
group showed a significant improvement in critical thinking, especially
in their awareness of critical thinking. For instance, students reported
that they were able to organize ideas well and engaged in difficult
problems. In another example, students with different levels of lan-
guage proficiency interacted with the voice-based chatbot Indigo over
their mobile phones. The students engaged with Indigo for 16 weeks,
and their chat scripts in the last session indicated an increasing fre-
quency of use of negotiation strategies compared with the first ses-
sion (Kim, 2016). Students at different language proficiency levels
showed improvement in different negotiation strategies: low-level
students demonstrated more repetition and reformulation skills to
overcome communication silence during chatting, while medium-level
and high-level students were more inclined to use confirmation check
strategies to comprehend the conversation.
The third type of activity involved using chatbots to coordinate
student online group discussions, where students were asked to inter-
act with one another synchronously. Tegos et al. (2014), for example,
explored the use of the dialogue-based chatbot MentorChat to trigger
student' utterances in group discussions and balance the conversation
between “weak”and “strong”students to foster peer interactions.
TABLE 4 Pedagogical uses of chatbots
Affordance Learning activities Example N Sample studies
Interlocutor Language knowledge practice Students interacted with chatbots to practice
discussing a specific topic (e.g., hotel room
bookings and daily life).
17 Ayedoun (2020)
Kim (2016)
Students used the chatbot to practice grammar in a
fill-in-the-blanks exercise.
1 Jia (2008)
Learning skills activity Students read story with being asked questions by
chatbots.
6 Xu et al. (2021)
Goda et al. (2014)
Group discussion coordination The chatbot distinguished between weak and strong
students, and supported students with individual
promptings.
1 Tegos et al. (2014)
Simulation Role-playing The chatbot played the role of a waiter or professor
in an authentic language environment simulation.
4 Ayedoun (2020)Yang
(2010)
Learning scenario representations Chatbots were used in parts of the simulation (e.g., in
a virtual real estate company office, a clothing
store, a shoe store) to provide learners with
opportunities to address specific language needs;
these were integrated into a larger virtual learning
environment.
1 Wang et al. (2017)
Transmissive Delivering well-targeted interventions The chatbot functioned as a co-teacher delivering
instructional materials.
3 Lin and Chang (2020)
Tegos et al. (2014)
Helpline Responding to requests for assistance Learners accessed chatbots for help when they
encountered language problems regarding the
learning content.
2 Wang et al. (2017)
Recommendation Providing level-appropriate learning
contents
The chatbot recommended a book based on
students' language levels.
1 Ruan, Willis (2019)
10 HUANG ET AL.

Once MentorChat identified “weak”students, who gave no response
to a given question, the system would direct the question again to
these students by mentioning their names, for instance, “Janna, what
can help you to block out negative thoughts?”(p. 78). The participat-
ing students reported that the chatbot helped them recall key con-
cepts learned in previous sessions.
3.3.2 | Simulating an authentic language
environment
This pedagogical approach emphasized the use of chatbots to simulate a
virtual target language speaking environment by two types of learning
activities: (a) role-play activity and (b) learning scenario representation.
Role-playing entails having chatbots perform real-life roles to simulate an
authentic communication environment of the targeted language. In a
study conducted by Ayedoun et al. (2015), for example, a Web-based
chatbot was presented as a native English-speaking waiter equipped with
verbal and non-verbal (i.e., facial expressions, head movements, and lip-
syncs) functions to simulate a restaurant setting. Five university students
played the role of customers, interacting with the chatbot individually.
An analysis of answers to the pre- and post-questionnaires suggested
students' rising self-confidence and growing desire to communicate in
English. Another example of a role play simulation activity was found
in Yang and Zapata-Rivera's (2010) study, in which a chatbot took
the role of a professor named Dr. Brown who responded to stu-
dents' request strategies. Fifteen students from a university in the
U.S. interacted with the chatbot system for 45 min during the inter-
vention. The results of the usability questionnaire indicated that
92% of the students agreed that communicating with the chatbot
motivated them to learn how to make requests in academic settings
with professors.
An example of a learning scenario representation was reported by
Wang et al. (2017). Within the virtual simulation, chatbots were built
into different scenarios that students could “visit”(e.g., a real estate
company office, stores, a supermarket, a hotel, and a restaurant). The
virtual learning environment immersed students in simulations of sce-
narios in which they had to tackle problems similar to those that they
might meet in reality. Students who experienced interacting with the
chatbots later reported they felt a sense of being physically present in
the language learning environment.
3.3.3 | Transmission of information
This function highlighted the use of chatbots as a channel to deliver
learning contents prepared beforehand by the course teacher. For
example, Lin and Chang (2020) explored the use of the chatbot DD to
deliver essay writing outline in two tutorial sessions, where the chatbot
introduced the features of a thesis statement. Chiaráin and Chas-
aide (2016) designed a chatbot Taidhgín as an Irish native speaker to
talk to secondary students. The topics (e.g., hobbies and holidays) were
associated with the curriculum for second level school oral examination.
A correction system was used in this chatbot to present the most com-
mon grammatical and orthographic errors made by students.
3.3.4 | Helpline
By trawling through enormous amounts of information in a database,
chatbots can perform the function of a helpline, providing students
with information about the learning content when queried by users.
Wang et al. (2017), for example, explored the integration of chatbots
in the Web-based virtual learning platform VILLAGE, in which stu-
dents were assigned to practice their grammar (e.g., using linking
verbs and constructing sentences using different verb tenses). Stu-
dents could access the chatbot for help whenever they encountered
problems with the learning activities.
3.3.5 | Providing recommendation
Chatbots can also automatically recommend learning materials
according to the students' prior utterances. For example, Ruan
et al. (2019) explored a voiced-based chatbot BookBuddy as a virtual
learning partner to facilitate children's reading comprehension. The
chatbot system collected children's information and analysed appro-
priate topics during the interaction and recommended books to chil-
dren from the database. The children reported that they enjoyed
speaking English with the chatbot and were highly engaged during the
interaction.
3.3.6 | Effects of using chatbots on students'
behavioural and cognitive outcomes
Chatbots are synchronous tools that support individual learning and have
been used with the aim of increasing student outcomes (Winkler &
Soellner, 2018). Fryer et al. (2020), however, pointed out that the limita-
tions of the technology itself (e.g., low accuracy of chatbots' responses)
may diminish the degree to which chatbots can help improve students'
performance. To robustly understand the effects of using chatbots on stu-
dents' language outcomes, we searched within the previous list of 25 arti-
cles using a more restrictive criteria to select eligible studies (Table 5).
Guided by the inclusion and exclusion criteria, we identified eight
experimental research studies (Table 6). Two of them examined stu-
dents' behavioural outcomes and the other six investigated students'
cognitive outcomes. Behavioural outcomes refer to students per-
forming learning tasks such as participating in discussions (Goda
et al., 2014), which can be assessed by observations (e.g., the number
of conversations). Cognitive outcomes refer to students' learning per-
formance concerning domain-specific knowledge, such as reading (Xu
et al., 2021), writing (Lin & Chang, 2020), vocabulary (Jia et al., 2012;
Kim, 2018b) and grammar (Kim, 2019).
Overall, previous studies exploring examining the effects of
chatbots on students' behavioural outcomes showed positive results
HUANG ET AL.11

when chatbots were used to buttress the learning content through
interaction with students. For example, Goda et al. (2014) used a
chatbot to prepare students' group discussion activity. All students
were given a discussion topic and 10 min to prepare. A chatbot was
used in experimental group to help students organize their ideas and
structure the discussion; students in control group were asked to sea-
rch information online. After 10-min preparation, students joined the
group discussion with their peers in the same condition. The number
of interactions in this peer group discussion was coded for both
groups. The results indicated that the number of conversation actions
in experimental group were higher than that of control group. How-
ever, it should be noted that conversation frequency is not equivalent
to communication quality, which the authors did not discuss. In
Kim's (2016) study, students were first grouped into three proficiency
levels (i.e., low, medium, and high levels, based on a TOEIC test).
TOEIC is a standardized English test that measured students' English
language skills required in the workplace. Next, students in each level
(i.e., low, medium, high) were randomly assigned into either an experi-
ment group (i.e., communicate with a chatbot) or a control group
(i.e., communicate with human peers). Results indicated that the low-
level students in the chatbot group performed more repetition and
reformulation strategies (e.g., repeating words or paraphrases from
previous interactions rather than paraphrasing the meanings) than the
non-chatbot group. In contrast, the medium-level and high-level stu-
dents requested clarifications and performed confirmation checks
more frequently during the conversation with the chatbot. There was
no significant difference in the performance of comprehension check
strategies between the chatbot groups and the control group.
Previous research, however, suggested mixed findings con-
cerning the effects of chatbot on students' cognitive outcomes. On
one hand, several studies (Kim, 2018a, 2018b reading; Xu
et al., 2021) reported no significant difference between participants
who used chatbots and those who did not. For example, Xu
et al. (2021) compared children's reading comprehension under a
chatbot-assisted conversation (experiment group), and a human con-
versation (control group). The chatbot was assigned to ask questions
toguidechildrenintheexperimentalgrouptounderstandthestory,
whereas children in control group were asked the same questions by
a human teacher. A post-hoc analysis on the comprehension scores
from experimental and control groups indicated that chatbot had a
similar effect (p=0.29) as a human teacher on facilitating children's
reading comprehension by asking guided questions. Kim (2018a and
2018b reading) integrated a chatbot as language practice partners
into university students' homework to engage students' out-of-class
vocabulary, and reading learning. Students interacted with the
chatbot “Elbot”on mobile phones via text or auditory messages. No
significant differences in students' vocabulary knowledge (adjective
use, noun use), and reading skill were reported between the experi-
mental groups (interacting with chatbot) and control groups
(received no treatment).
On the other hand, other studies (Jia et al., 2012; Kim, 2018b
listening, 2019; Lin & Chang, 2020) reported positive effects of
chatbot on students' language learning. For example, Jia
et al. (2012) designed specific dialogue scripts based on a course
syllabus. The students in the experimental group were required to
use the CSIEC chatbot to take one vocabulary assessment per week
to review their vocabulary knowledge through both closed ques-
tions and multiple-choice questions, whereas the students in the
control group did not complete any chatbot assessments. A post-
test on students' vocabulary acquisition indicated a significant dif-
ference in favour of the experimental group (p=0.044), with a
moderate effect size (g=0.417).
The experiment group students in Kim (2018b listening) carried
out conversations on topics ranging from business to the environment
using voice chats with the chatbot Elbot for a total of 20 sessions with
each session lasting more than 10 min over 16 weeks. Although both
the experimental and the control group received formal listening
instruction during the regular English teaching time period, the control
group received no treatment. Results showed that students using the
chatbot significantly outperformed the control group (p=0.013
)
in
their listening skills with a large effect size (g=0.752).
In another study, Kim et al. (2019) examined the effects of a
chatbot on Korean college students' English grammar skills. Students
in the experiment group conversed via text on mobile phones with
the chatbot Replika, which could ask them questions. The chatbot
conversations took place in 10 chat sessions, each session lasting
10 min, over a period of 16 weeks. Students in the control group con-
versed with a human partner. Student grammar outcomes were mea-
sured by a grammar test adapted from a standardized grammar test. A
TABLE 5 Criteria of studies evaluating the effects of chatbots on
student behavioural and cognitive outcomes
Inclusion Exclusion
The study must have an
experimental group (i.e.,
learning with chatbot) and a
control group (i.e., learning
without chatbot).
Any study that has no control
group.
The study must report
quantitative findings
regarding chatbots' impact on
students' language
performance for specific
language knowledge or the
obtainment of certain
language skills.
Any study that evaluates
students' anxiety, confidence,
task and course interest, and
learning attention.
Students' learning outcomes
must be assessed by objective
measurements such as test
scores, chat records, or the
number of conversations
observed.
Any study that relies only on self-
reported data, such as
students' questionnaires.
The study must provide
statistical data, such as mean,
standard deviations, or t-test
results, for both the
experimental group and the
control group.
Any study that merely reports
percentages of outcomes.
Any study that reports
incomplete pvalues.
Any study that reports data only
from the experimental groups.
12 HUANG ET AL.

TABLE 6 Experimental studies using chatbots
Study
Research design
Statistical analysis
(measures)
Outcomes
examined
Results
Experimental design
Sample
size Duration
Availability
of pre-test
a
pvalue
Hedge's
g
Goda et al.
(2014)
EG:31
CG:32
10 min No t-test (chatlog
observation)
Communication +(p<0.01) in favour of EG
with regard to number of
conversations increasement
in group discussion.
1.248 EG: Students talked with the chatbot ELIZA
regarding the topic of “an ideal family”for
10 min before participating in a group
discussion with peers.
CG: Students searched for relevant information
on the Internet.
*This study did not establish whether there was
an initial difference between the EG and CG.
Lin and
Chang
(2020)
EG:167
CG:190
50 min a
week *
2 weeks
No t-test (writing
assignment)
Essay outline
assignment
+(p<0.05) in favour of EG. 0.039 EG: Students interacted with the chatbot DD to
learn how to structure a thesis statement and
how to provide peer feedback to their
classmates' essay outline.
CG: No treatment was received.
*No initial difference between EG and CG.
Jia
(2012)
EG:47
CG:49
45 min a
week *
20 weeks
Yes t-test
(vocabulary test)
Vocabulary
acquisition
+(p<0.05) in favour of EG 0.417 EG: Students spent 45 min per week in a
multimedia computer lab using assessments on
CSIEC to review their vocabulary.
CG: No treatment was received.
*No initial difference between EG and CG.
Kim (2016) EG:63
CG:60
10 min a
week *
16 weeks
No t-test
(chatlog observation)
Confirmation
checks
+(p<0.05) in favour of
medium-level and high-level
of EG.
1.214 EG: Students downloaded the chatbot
application Indigo and interacted with it on
mobile phones. Students talked with the
chatbot via voice in weekly 10-min sessions
for 16 weeks. The conversation topics were
related to students' daily lives, such as school
life and classmates.
CG: Students chatted with student partners.
*This study did not establish whether there was
an initial difference between the EG and CG.
Comprehension
checks
No significant differences
(p>0.05) in all three levels
between EG and CG.
0.173
Clarification
requests
+(p<0.05) in favour of
medium-level and high of
EG.
0.831
Repetition +(p<0.05) in favour of low-
level and medium-level of
EG.
1.559
Reformulation +(p<0.05) in favour of low-
level of EG.
0.730
Kim (2018a) EG:24
CG:23
10 min a
week *
8 weeks
Yes t-test
(vocabulary quizzes)
Verb +(p<0.05) in favour of EG. 0.626 EG: Students talked with the chatbot Elbot via
text or voice in weekly 10-min sessions for
8 weeks on mobile phones. Chat topics varied
from school life to movies.
CG: No treatment was received.
*No initial difference between EG and CG.
Adjective No significant differences
(p>0.05) between EG and
CG.
0.593
(Continues)
HUANG ET AL.13

TABLE 6 (Continued)
Study
Research design
Statistical analysis
(measures)
Outcomes
examined
Results
Experimental design
Sample
size Duration
Availability
of pre-test
a
pvalue
Hedge's
g
Noun No significant differences
(p>0.05) between EG and
CG.
0.525
Kim (2018b) EG:24
CG:22
10 min a
week *
16 weeks
Yes t-test
(TOEIC listening and
reading test)
Listening +(p<0.05) in favour of EG. 0.752 EG: Students talked with the chatbot Elbot on
mobile phones as homework. The
conversations could be textual or auditory, and
took place in weekly 10-min sessions for
16 weeks. There were 20 chat sessions in
total. Chat topics varied from business to the
environment.
CG: No treatment was received.
*No initial difference between EG and CG.
Reading No significant differences
(p>0.05) between EG and
CG.
0.148
Kim et al.
(2019)
EG:36
CG:34
10 min a
week *
16 weeks
Yes t-test
(grammar test adapted
from a standardized
test)
Grammar +(p<0.05) in favour of EG. 0.482 EG: Students talked via text on mobile phones
with the chatbot Replika, which could ask
them questions. The conversations took place
in weekly 10-min sessions for 16 weeks.
CG: Students chatted with student partners.
*No initial difference between EG and CG.
Xu et al.
(2021)
EG:33
CG:31
20 min No Post-hoc analysis (story
comprehend-sion
quizzes)
Reading No significant differences
(p>0.05) between EG and
CG.
0.024 EG: Students read a story with being asked
questions from a voice-based chatbot to guided
their comprehension within 20 min.
CG: Students were asked the same questions
from a human teacher.
*No initial difference between EG and CG.
Abbreviations: CG, control group; EG, experimental group using chatbots.
a
“Availability of pre-test”refers to the pre-test for learning performance with objective measures (e.g., test scores), excluding the pre-test for students' perspectives on (e.g., interest in) language learning (e.g.,
Kim, 2016).
14 HUANG ET AL.

post-test indicated a significant difference in favour of the chatbot
group (p=0.046), with a moderate effect size (g=0.482).
The final study by Lin and Chang (2020) employed the chatbot
DD to introduce students the main elements of an argumentative
essay writing (e.g., statement, topic sentences, and conclusion) during
tutorial sessions in 2 weeks. Each tutorial class lasted 50 min. In the
experimental group, the chatbot DD was assigned to deliver the out-
line of essay statement in the first week, and then guide students in
providing peer feedback to their classmates' essay outline. Students in
the control group wrote the essay outline without interacting with the
chatbot. Students' achievement was measured by their essay outline
writing. Students learning with chatbot DD performed better than the
control group (p=0.027) with an almost negligible effect size
(g=0.039). The authors did not establish whether any significant dif-
ference existed between the experiment and control group students'
initial writing ability at the start of the study.
In summary, results from the eight experimental studies reveal
that chatbots can positively enhance students' language learning in
some topics such as grammar, listening, and writing. Chatbots do not
appear to improve students' reading comprehension. Results con-
cerning the effectiveness of chatbots in enhancing vocabulary learning
were mixed. However, because these results (reading and vocabulary)
were based on only two experimental studies each, it should be inter-
preted with caution. So far, no studies reported an adverse negative
effect of using chatbots on language student learning outcomes.
3.4 |What are the social affordances, if any, of
using chatbots in language learning?
We examined the social presence of chatbots in language learning via
focusing especially on the eligible studies' instructional design, the
analysis of students' discourse, and interviews with students about
their views on chatbot-supported language learning. As given in
Table 7, three categories of social presence were identified in
chatbot-supported language learning, including the interpersonal com-
munication (e.g., students' self-discourse), open communication
(e.g., continuing a thread, asking questions, expressing agreement) and
cohesion communication (e.g., using vocatives and greetings).
The findings suggested that interpersonal communication can be
established through students' self-disclosure with chatbots. For exam-
ple, Goda et al. (2014) reported that students demonstrated self-
disclosure when they discussed the essential factors of an ideal family
with a chatbot partner. The students expressed their personal opinions
about their own family, such as “my family members aren't friendly,”
and they also asked the chatbot partner questions, such as “Do you
have family?”Similarly, Xu and Warschauer (2020) reported students
shared personal experience when they were explaining their opinions
with the chatbot. The findings showed that it was possible to use a
chatbot as a learning partner to enhance social interaction through
exchanging self-disclosure information. In another study, analysis of the
conversation scripts by Ayedoun et al. (2020, p. 608) showed that the
chatbot was able to continue a thread with students by providing exam-
ples of possible responses when students faced difficulties to answer
the question, for example, “You may say, 'one beer please' to order a
beer.”Social presence can also be found in chatbots' use of greetings,
expressions of agreement with students' ideas, asking question, and voc-
atives. Greetings were used usually at the beginning of the conversation
as an ice-breaking activity. The chatbot DD in the study of Lin and
Chang (2020), p. 82 greeted students by saying “Hi, remember me? I
worked with you for your thesis statement. It's me DD!”When chatbots
expressed their agreements during a conversation, for example, “Good
job! You're correct!”, students perceived the chatbot as a patient,
friendly, and non-judgmental partner (Ruan et al., 2019). In terms of ask-
ing other students questions in an online language learning environment.
We found only one study (i.e., Tegos et al., 2014) that employed a
chatbot as a group discussion coordinator asking students questions,
which in turn helped increase students' actions of asking questions with
each other. The chatbot in this study also referred to students by names,
such as in the statement “Janna, what can help you to block out nega-
tive thoughts (p.78)? [Janna was the name of a student]”.
Embracing chatbots in language learning can be a way to encourage
an open learning climate of interpersonal communication, which can help
overcome students' nervousness about speaking the target language and
promote their willingness to communicate (Ayedoun et al., 2015), help
them better understand learning objectives and support them in collabo-
rative learning (Tegos et al., 2014), and strengthen their sense of social
presence within virtual language environments (Wang et al., 2017).
TABLE 7 Social presence classification and indicators of chatbot-supported language learning
Category Indicators Example N
Sample studies (first
author)
Interpersonal
communication
Self-disclosure Presenting details of personal life outside class, or
expressing vulnerability
2 Goda et al. (2014)
Xu (2020)
Open communication Expressing
agreement
Expressing agreement with others or with the content of
others' statements
4 Ruan et al. (2019)
Continuing a thread Using the reply function to continue a topic, rather than
starting a new thread
3 Ayedoun (2020)
Asking questions Asking questions of other students 1 Tegos et al. (2014)
Cohesion communication Salutations Using greetings for a purely social interaction 5 Lin and Chang (2020)
Vocatives Addressing or referring to participants by name 1 Tegos et al. (2014)
HUANG ET AL.15

However, the immature development of chatbot technology can
also lessen social presence in students' language learning. Fryer et al.
(2017) criticized chatbots for their inability to maintain students' lan-
guage learning interest; students' interest in speaking tasks with
chatbot partners dropped after the first task compared with speaking
to a human partner. Similarly, the participants in Hsu's (2020) study
who interacted face-to-face with human interlocutors perceived a
higher socialization than those who conversed with chatbot partners
in a second language context.
3.5 |What are the challenges, if any, of using
chatbots in language learning?
Despite the aforementioned technological, pedagogical, and
social affordances, researchers have noted substantial existing
challenges in implementing chatbots in language learning. Three
categories of challenges were summarized, namely chatbots' tech-
nological limitations, novelty effects, and students' cognitive load
limitations.
3.5.1 | Technological limitation of chatbot
Although chatbots can contribute to students' language learning, the
limitations of their technological capability cannot be overlooked. The
most frequently reported technological challenge was the perceived
unnaturalness of the computer-generated voice, which students con-
trasted with human voices (Goda et al., 2014; Tegos et al., 2014).
Failed communication was also found to happen when students
entered incomplete sentences (Yin & Satar, 2020) or when chatbots
responded with nonsense outputs (Fryer et al., 2019). Along with the
design of the chatbot interface, the lack of emotion and visible cues
from chatbots during interactions were found to diminish students'
positive affective states (e.g., interest) in relation to language learning
(Gallacher et al., 2018). Chatbots with limited artificial intelligence
could not decipher students' inputs that were out of their range. For
example, students' ideas headed in unpredictable directions as conver-
sations developed, and new topics introduced by students could not
be recognized by the chatbot system (Yang & Zapata-Rivera, 2010).
Due to chatbots' unnatural robotic voices and their inability to carry
on long conversations, chatbots appear to isolate learners from the
language learning environment.
3.5.2 | The novelty effect of language learning
A novelty effect arises when a new technology is introduced to stu-
dents, which may increase students' motivation or learning perfor-
mance due to the newness of the technology (Chen et al., 2016).
Fryer et al. (2017) demonstrated this effect with chatbots in a
16-week experimental study, in which students' interest in speaking
tasks declined after the first communication task with the chatbot.
The students viewed the chatbot as a novelty rather than a lasting
partner in daily language practice (Gallacher et al., 2018).
3.5.3 | Cognitive load aroused via chatbots
“Cognitive load”in this context refers to students' additional attention
or mental effort that needs to be exerted to perform a learning task
during the learning process. More specifically, the instructional design
of chatbot-supported activities determines how much mental effort
students are required to produce. Given that humans have limited
capacity of cognitive processing, the imposed cognitive load on stu-
dents influenced their learning performance (Sweller, 1988). For
example, the design of chatbot-supported learning with complex ele-
ments (e.g., voice and animation) can confuse students to allocate
attention and process task information. Kim (2016) reported that stu-
dents of medium and high language proficiency processed more inter-
actions with a voice-based chatbot than students with low
language proficiency, suggesting that the medium- and high-level stu-
dents benefit more from the voice-based chatbot than the low-level
students who were burdened with a higher cognitive load to process
the auditory information. In such a situation, the use of chatbots may
be a barrier to students' language learning. A higher extraneous
cognitive load could diminish students' learning outcomes (Fryer
et al., 2020).
4|DISCUSSION
This systematic review set out to identifying the usefulness of chatbots
in language learning, and the effects on students' learning outcomes
thereof. In this section, we first address main findings of five research
questions regarding the current use of chatbots in language learning, and
then propose several implications for future chatbot implementation
based on our findings. Figure 5 shows all identified affordances within
the usefulness framework. Finally, we suggest directions for further
research on chatbot-supported language learning.
4.1 |The current stage of chatbots in language
learning
The first research question identifies the current contexts where
chatbots have been used in language education. Higher education has
been the main setting where chatbots are used. The explanation for
this may be found in the growth of online learning and computer-
assisted learning in higher education. Chatbots embedded in either
webpages or instant messaging applications provide higher education
students convenient online access to learning language. The findings
also encourage educators to use chatbots for open conversation
(i.e., discussing any topic with students), mainly because of chatbots'
capability of keeping a conversation going (Grudin & Jacques, 2019)
that helps increase students' willingness to communicate in specific
16 HUANG ET AL.

languages (Ayedoun et al., 2019). Additionally, educators can use
task-focused chatbots to assist students in learning specific procedure
such as the outline of a thesis (Lin & Chang, 2020).
4.2 |The technological affordances of chatbots in
language learning
Evidence addressing the second research question indicates that
chatbots can promote students' communication in target languages
through three technological affordances: timeliness, personalization,
and ease of use. Students can practice their language knowledge with-
out the temporal and geographical limitations of having a human
learning partner. Unlike human partners, a chatbot can provide imme-
diate response tirelessly (Brandtzaeg & Følstad, 2017). Students can
receive personalized language learning materials based on their previ-
ous interaction with chatbots. Chatbots embedded into webpages,
tablets, or mobile instant messaging applications can help facilitate
easier-to-use student-chatbot interaction. Chatbots have achieved
the “fundamental benefit of using online technologies”in an educa-
tional environment (Bower, 2017) because they can provide students
access to learning resources. These findings should encourage educa-
tors to use chatbots to fill the role of a tireless learning companion
who can be contacted for domain-specific conversation in any acces-
sible device.
4.3 |Five pedagogical affordances of chatbots in
language learning
The present review also identified five particular pedagogical uses for
chatbots in language learning, namely, interlocution, simulation, trans-
mission, helpline, and recommendation. The findings show chatbots
are most commonly used as a role of interlocuter to communicate
with students. We summarized practical examples for each category
to give educators ideas for using chatbots in their own teaching con-
texts (see Table 3). Although these pedagogical uses of chatbots are
associated with different learning objectives in each study, the over-
arching educational purpose of chatbot is to promote students'
interaction during their language learning process. This can be
explained by Moore's (1989) three types of interaction in online learn-
ing, namely student–student, student-teacher, and student-content
interaction. Educators can employ a chatbot to be a knowledgeable
friend accessible to students (i.e., student–student interaction), a vir-
tual tutor providing guidance and recommendation (i.e., student-
teacher interaction), and to deliver language learning contents
(i.e., student-content interaction) in a simulated language learning
scenario.
However, our understanding of the effects of chatbot on student
outcomes is still limited. In this review, only eight eligible studies could
be found that specifically examined the possible effects of chatbots
on students' behavioural and cognitive outcomes. Two of these stud-
ies failed to establish whether there was an initial difference between
the control and experimental groups (see Table 5). The equivalence of
students and teachers before the intervention should be considered
in future research design and data analysis. In addition, the measure-
ment of behavioural engagement by previous studies so far has
focused mainly on computing the number of student-chatbot interac-
tion. Counting the number of student-chatbot interaction does not
provide us with an indication of the quality of student language learn-
ing. Interaction oriented to students' cognitive outcomes should be
more characterized by the qualitative nature of the interaction and
less by quantitative measures (Garrison & Cleveland-Innes, 2005).
4.4 |Social presence in chatbot-supported
language learning
The findings of the fourth research question can advance our theoret-
ical understanding of social presence in chatbot-supported language
learning. The answer to this research question shows that chatbots
can encourage students' online social presence through supporting
student self-disclosure, continuing a friendly thread, expressing agree-
ment, and addressing students by names. Chatbot self-disclosure can
promote students' self-disclosure in turn, and encourage students to
provide longer responses and express their feelings on target topics.
Chatbot self-disclosure can also exert a positive effect on improving
participants' perceived intimacy and enjoyment with the chatbot (Lee,
FIGURE 5 Identified technological,
pedagogical, and social affordances of
chatbots in language learning
HUANG ET AL.17

Yamashita, Huang, & Fu, 2020). Educators can assign chatbot to a
social member who can communicate with students in the target lan-
guage in a warm and friendly way. The study of De Gennaro,
Krumhuber, and Lucas (2020) suggests that using an empathic chatbot
has a mitigating impact on students' social exclusion; and this can help
develop an open and interpersonal communication atmosphere within
the language learning environment.
4.5 |Current challenges of using chatbots in
language learning and potential solutions
The fifth research question reveals three challenges of using
chatbots in language education, including the technological limita-
tions of chatbots, novelty effects and cognitive load during stu-
dents' learning process. Studies reported technological limitations
such as narrow database support and unnatural robotic voices.
With regard to tackling the current technological challenges,
teachers can take a leadership role in determining how chatbots
can be best used to help achieve learning outcomes. Teachers can
determine how best to use chatbots in their current state of tech-
nological development, thereby mitigating their limitations. For
example, Fryer and Carpenter (2006) argued that using chatbots is
more appropriate for advanced language learners than beginners
because inaccurate word input cannot be analysed by the system
and may give students disappointing responses. In contrast, De
Gasperis and Florio (2012) successfully used a restricted chatbot to
correct learners' spelling errors, demonstrating that it is possible to
transform a technical limitation into a benefit. Similarly, teachers
can exploit the restrictedness of chatbot conversations to check
beginners' factual knowledge, such as with vocabulary memoriza-
tion. Chatbots with narrow functionalities or a limited database
may be more acceptable to students for learning factual knowledge
than for conceptual knowledge (Huang et al., 2019). As for
advanced language learners, teachers can set rules of communica-
tion with chatbots to help students understand the chatbots' capa-
bilities and limitations. For example, if help is needed during a
conversation, students can be asked to type a particular word or
symbol (e.g., “helpline”) to activate the helpline function without
interrupting the ongoing interaction. For the future implementa-
tion of chatbots in language learning, the current stage of techno-
logical development should be considered by educators.
To mitigate the novelty effect, delivering a workshop prior to
the first lesson can prepare students by giving them prior experience
on chatbot-integrated learning. As suggested by Fryer et al. (2020),
students' cognitive processing (i.e., how students process new infor-
mation in an environment) can be enhanced by using Mayer's (2017)
principles of multimedia use for learning. For instance, students' lan-
guage learning can be more efficient if the texts in chatbots are
presented in the form of a conversation and humanlike gestures are
employed in the agent. Additionally, the integration of quick buttons
can make chatbots easier to use and allow students to choose learn-
ing resources by a click, which helps enhance the interactivity
between chatbot and students and further engaged students
(Sundar, 2012).
4.6 |Suggestions for future research
First, the era of mass adoption of chatbots in language education has
not yet arrived. Currently, we do not have enough empirical evidence
of whether using chatbots is beneficial for language learners across all
age ranges. To further investigate the validity of using chatbots in lan-
guage learning, research involving education levels other than univer-
sity (e.g., primary and secondary school) ought to be conducted in the
future to fill the gaps in our knowledge.
Second, no studies in the current review were undertaken longi-
tudinally. The intervention times in the studies examined lasted from
several minutes to one semester, which could have led to a novelty
effect. To evaluate the long-term effects of chatbots on students' lan-
guage learning, studies lasting two or more semesters should be con-
sidered to see if students' interest or motivation of interacting with
chatbots will change overtime. Researchers are encouraged to mea-
sure both students' behavioural and cognitive outcomes. Other vari-
ables, such as students' technology literacy and learning adaptability,
the design of chatbots' interface, and different target languages can
be evaluated in future empirical research.
Third, the majority of previous studies on chatbot-supported lan-
guage learning conducted measurements using self-reported question-
naires, which insufficiently presented the potential effects of using
chatbots on students' language achievement. As Fryer et al. (2020)
suggested, observed variables such as students' achievement and class-
room observations should be included in future research to validate the
integration of educational chatbots in language learning. Researchers
are advised to include objective measurements in future studies.
Fourth, the majority of previous experimental research on chatbot-
supported language learning focused on the different effects between
chatbots and human partners. Few research evaluated the use of
chatbots compared with other equivalent tools (e.g., students in control
group searching information online; Goda et al., 2014). Additionally, given
that current use of chatbots has been tied closely with online learn-
ing, computer-assisted learning, and mobile learning, these learning
conditions also include the presences of other techniques (e.g., 3D
learning platforms; Wang et al., 2017). Therefore, it is vital to con-
sider whether any increase in students' performance and engage-
ment is aroused by the chatbots only or by the combinations of
tools. Researchers are encouraged to follow up along these lines to
provide educators with empirical-based evidence in making appro-
priate use of chatbots in the future.
Finally, few studies reported on teachers' perceptions surround-
ing the use of chatbots in language teaching activities. Teachers may
be sidelined as chatbot designers due to the extra effort required to
create specific chatbots for target learners (Nghi et al., 2019). It is
challenging to satisfy all students' learning expectations with just
one type of chatbot because different students may want to talk
about different topics (Fryer & Carpenter, 2006). Future studies
18 HUANG ET AL.

could explore teachers' perceptions of the chatbots implementation
in language learning.
5|LIMITATION
Several limitations of this systematic review must be acknowl-
edged. First, details about the implementation of chatbots were
not always clearly reported in previous studies. This resulted in
our excluding such studies in the present review. This limits the
findings of our review to only studies which described in detail
how chatbots were used to support language learning. We
encourage researchers to document the details of their chatbot
activities in future research. Second, the findings of this review
are limited to chatbot empirical studies that were reported in
English. Possible pedagogical activities related to the use of
chatbots published in other languages were excluded. Third, there
were only eight experimental studies identified using a more strin-
gent criteria that reported the effects of using chatbots on stu-
dents' behavioural and learning outcomes. Therefore, conclusions
about the effects of chatbot use on student learning outcomes
should be viewed with caution given the small number of experi-
mental studies.
6|CONCLUSION
The past two decades have demonstrated the potential of using
educational chatbots for language learning. Yet, to date, the effects
of chatbots on students' language learning are still generally under-
researched. In this study, we found that chatbots offer diverse tech-
nological affordances in facilitating students' completion of language
learning tasks. We also identified five pedagogical uses of chatbots
in language learning and the effects on students' learning outcomes
thereof. The findings suggested that educational chatbots can foster
students' language learning via interaction activities underpinned by
intended learning objectives. Using chatbots in language learning
can offer social affordances such as self-disclosure and agreement
expression among students via the establishment of an open and
cohesive learning community. Several challenges of using chatbots
in language learning were found as well, including technological limi-
tations, the novelty effect, and cognitive load.
The current review contributes to the literature in four specific
ways: (a) First, it goes beyond merely reporting the different types
of chatbot use, and instead examines the possible technological,
pedagogical, and social affordances associated with chatbots in lan-
guage learning. This is accomplished by taking a usefulness theoreti-
cal perspective (Kirschner et al., 2004) to analyse the utility and
usability of chatbots for teaching and learning purposes in language
learning contexts. (b) Second, this review helps educators better
understand how chatbots have actually been used in language learn-
ing, and their benefits or advantages (if any). (c) Third, this review
identifies several challenges related to chatbot use, and provides
possible solutions to address them. (d) Finally, this review proposes
several directions for future research that can advance our under-
standing of chatbot use in language learning.
ACKNOWLEDGEMENTS
This study was supported by a grant from the University of Hong
Kong Teaching Development Grant 2019 (Project reference no: 730).
PEER REVIEW
The peer review history for this article is available at https://publons.
com/publon/10.1111/jcal.12610.
DATA AVAILABILITY STATEMENT
Data are available in the published articles.
ORCID
Weijiao Huang https://orcid.org/0000-0002-2095-1204
Khe Foon Hew https://orcid.org/0000-0003-4149-533X
REFERENCES
Ayedoun, E., Hayashi, Y., & Seta, K. (2015). A conversational agent to
encourage willingness to communicate in the context of English
as a foreign language. Procedia Computer Science,60, 1433–1442.
Ayedoun, E., Hayashi, Y., & Seta, K. (2019). Adding communicative and
affective strategies to an embodied conversational agent to enhance
second language learners' willingness to communicate. International
Journal of Artificial Intelligence in Education,29(1), 29–57. https://doi.
org/10.1007/s40593-018-0171-6
Bao, M. (2019). Can home use of speech-enabled artificial intelligence mit-
igate foreign language anxiety—Investigation of a concept. Arab World
English Journal,5,28–40.
Bower, M. (2017). Design of Technology-Enhanced Learning: Integrating
Research and Practice. Emerald Group Publishing.
Brandtzaeg, P. B., & Følstad, A. (2017). Why people use Chatbots. In I.
Kompatsiaris, et al. (Eds.), Internet Science. INSCI 2017 (Vol. 10673).
Springer. https://doi.org/10.1007/978-3-319-70284-1_30
Brandtzaeg, P. B., & Følstad, A. (2018). Chatbots: Changing user needs and
motivations. Interactions,25(5), 38–43.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qual-
itative Research in Psychology,3(2), 77–101.
Carayannopoulos, S. (2018). Using chatbots to aid transition. The Interna-
tional Journal of Information and Learning Technology,35(2), 118–129.
Chen, H.-L., Vicki Widarso, G., & Sutrisno, H. (2020). A ChatBot for learn-
ing Chinese: Learning achievement and technology acceptance. Journal
of Educational Computing Research,58(6), 1161–1189.
Chen, J. A., Tutwiler, M. S., Metcalf, S. J., Kamarainen, A., Grotzer, T., &
Dede, C. (2016). A multi-user virtual environment to support students'
self-efficacy and interest in science: A latent growth model analysis.
Learning and Instruction,41,11–22.
Chiaráin, N. N., & Chasaide, A. N. (2016). Chatbot technology with syn-
thetic voices in the acquisition of an endangered language: Motivation,
development and evaluation of a platform for Irish. Proceedings of the
Tenth International Conference on Language Resources and Evalua-
tion (LREC'16) (pp. 3429-3435).
De Gasperis, G., & Florio, N. (2012). Learning to read/type a second
language in a chatbot enhanced environment. In International Work-
shop on Evidence-Based Technology Enhanced Learning (pp. 47–56).
Springer.
De Gennaro, M., Krumhuber, E. G., & Lucas, G. (2020). Effectiveness of an
empathic chatbot in combating adverse effects of social exclusion on
mood. Frontiers in Psychology,10, 3061.
HUANG ET AL.19

Fryer, L., & Carpenter, R. (2006). Bots as language learning tools. Language
Learning & Technology,10(3), 8–14.
Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., & Sherlock, Z. (2017).
Stimulating and sustaining interest in a language course: An experi-
mental comparison of Chatbot and Human task partners. Computers in
Human Behavior,75, 461–468.
Fryer, L. K., Coniam, D., Carpenter, R., & L
apus¸neanu, D. (2020). Bots for
language learning now: Current and future directions. Language Learn-
ing & Technology, 24(2), 8–22. Retrieved from http://hdl.handle.net/
10125/44719
Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners:
Connecting learning experiences, interest and competence. Computers
in Human Behavior,93, 279–289.
Gallacher, A., Thompson, A., & Howarth, M. (2018). “My robot is an
idiot!”–Students' perceptions of AI in the L2 classroom. In P. Taalas, J.
Jalkanen, L. Bradley, & S. Thouësny (Eds.), Future-Proof CALL: Language
Learning as Exploration and Encounters: Short Papers from EUROCALL
(pp. 70–76). Research-publishing.net.
Garrison, D. R. (2011). E-learning in the 21st Century: A Framework for
Research and Practice. Taylor & Francis.
Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in a
text-based environment: Computer conferencing in higher education.
The Internet and Higher Education,2(2–3), 87–105.
Garrison, D. R., & Cleveland-Innes, M. (2005). Facilitating cognitive pres-
ence in online learning: Interaction is not enough. The American Journal
of Distance Education,19(3), 133–148.
Gibson, J. J. (1977). The theory of affordances. In R. Shaw & J. Bransford
(Eds.), Perceiving, Acting, and Knowing: Toward an Ecological Psychology
(pp. 67–82). Erlbaum.
Goda, Y., Yamada, M., Matsukawa, H., Hata, K., & Yasunami, S. (2014).
Conversation with a chatbot before an online EFL group discussion
and the effects on critical thinking. The Journal of Information and Sys-
tems in Education,13(1), 1–7.
Grudin, J., & Jacques, R. (2019). Chatbots, humbots, and the quest for arti-
ficial general intelligence. In Proceedings of the 2019 CHI Conference on
Human Factors in Computing Systems (pp. 1–11). ACM.
Haristiani, N. (2019). Artificial intelligence (AI) chatbot as language learning
medium: An inquiry. Journal of Physics: Conference Series,1387(1), 2020.
Hsu, L. (2020). To CALL or not to CALL: Empirical evidence from neurosci-
ence. Computer Assisted Language Learning,1–24.
Jia, J., Chen, Y., Ding, Z., & Ruan, M. (2012). Effects of a vocabulary acqui-
sition and assessment system on students' performance in a blended
learning class for English subject. Computers & Education,58(1), 63–76.
Jia, J., & Ruan, M. (2008). Use chatbot CSIEC to facilitate the individual
learning in English instruction: A case study. In Proceedings of the Inter-
national Conference on Intelligent Tutoring Systems (pp. 706–708).
Springer.
Kim, N.-Y. (2016). Effects of voice chat on EFL learners' speaking ability
according to proficiency levels. Multimedia-Assisted Language Learning,
19(4), 63–88.
Kim, N.-Y. (2018a). Chatbots and Korean EFL students' English vocabulary
learning. Journal of Digital Convergence,16(2), 1–7.
Kim, N.-Y. (2018b). A study on chatbots for developing Korean college stu-
dents' English listening and reading skills. Journal of Digital Conver-
gence,16(8), 19–26.
Kim, N.-Y., Cha, Y., & Kim, H.-S. (2019). Future English learning: Chatbots
and artificial intelligence. Multimedia-Assisted Language Learning,22(3),
32–53.
Kirschner, P., Strijbos, J.-W., Kreijns, K., & Beers, P. J. (2004). Designing
electronic collaborative learning environments. Educational Technology
Research and Development,52(3), 47–66.
Korpershoek, H., Harms, T., de Boer, H., van Kuijk, M., & Doolaard, S.
(2016). A meta-analysis of the effects of classroom management strat-
egies and classroom management programs on students' academic,
behavioral, emotional, and motivational outcomes. Review of Educa-
tional Research,86(3), 643–680.
Lee, Y.-C., Yamashita, N., Huang, Y., & Fu, W. (2020). "I hear you, I feel
you": Encouraging deep self-disclosure through a chatbot. In Proceed-
ings of the 2020 CHI Conference on Human Factors in Computing Sys-
tems (pp. 1–12). ACM.
Lefebvre, C., Glanville, J., Briscoe, S., Littlewood, A., Marshall, C.,
Metzendorf, M.-I., Noel-Storr, A., Rader, T., Shokraneh, F.,
Thomas, J., & Wieland, L. S. (2021). Chapter 4: Searching for and
selecting studies. In H. JPT, J. Thomas, J. Chandler, M. Cumpston, T. Li,
M. J. Page, & V. A. Welch (Eds.), Cochrane Handbook for Systematic
Reviews of Interventions (Version 6.2 ). Cochrane www.training.
cochrane.org/handbook
Lin, M. P.-C., & Chang, D. (2020). Enhancing post-secondary writers' writ-
ing skills with a chatbot. Journal of Educational Technology & Society,
23(1), 78–92.
Lu, C. H., Chiou, G. F., Day, M. Y., Ong, C. S., & Hsu, W. L. (2006). Using
instant messaging to provide an intelligent learning environment. In
Proceedings of the International Conference on Intelligent Tutoring Sys-
tems (pp. 575–583). Springer.
Mayer, R. E. (2017). Using multimedia for e-learning. Journal of Computer
Assisted Learning,33(5), 403–423.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, P. (2009).
Preferred reporting items for systematic reviews and meta-analyses:
The PRISMA statement. PLoS Medicine,6(7), e1000097.
Moore, M. G. (1989). Three types of interaction. American Journal of
Distance Education,3(2), 1–7.
Moore, M. G. (1993). Theory of transactional distance. In D. Keegan (Ed.),
Theoretical Principles of Distance Education. Routledge.
Ruan,S.,Willis,A.,Xu,Q.,Davis,G.M.,Jiang,L.,Brunskill,E.,&
Landay, J. A. (2019). Bookbuddy: Turning digital materials into
interactive foreign language lessons through a voice chatbot. In
Proceedings of the Sixth (2019) ACM Conference on Learning Scale
(pp. 1–4). ACM.
Schmulian, A., & Coetzee, S. A. (2019). The development of messenger
bots for teaching and learning and accounting students' experience of
the use thereof. British Journal of Educational Technology,50(5), 2751–
2777.
Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of
educational chatbots for the Facebook messenger. Computers & Educa-
tion,151, 862.
Sulosaari, V., Suhonen, R., & Leino-Kilpi, H. (2011). An integrative review
of the literature on registered nurses' medication competence. Journal
of Clinical Nursing,20(3–4), 464–478.
Sundar, S. S. (2012). Social psychology of interactivity in human-website
interaction. In Oxford Handbook of Internet Psychology. Oxford Univer-
sity Press.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learn-
ing. Cognitive Science,12(2), 257–285.
Tegos, S., Demetriadis, S., & Karakostas, A. (2015). Promoting academically
productive talk with conversational agent interventions in collabora-
tive learning settings. Computers & Education,87, 309–325.
Tegos, S., Demetriadis, S., & Tsiatsos, T. (2014). A configurable conversa-
tional agent to trigger students' productive dialogue: A pilot study in
the CALL domain. International Journal of Artificial Intelligence in Educa-
tion,24(1), 62–91.
Tu, C. H. (2000). On-line learning migration: From social learning theory to
social presence theory in a CMC environment. Journal of Network and
Computer Applications,23,27–37. https://doi.org/10.1006/jnca.1999.
0099
Wang, Y. F., Petrina, S., & Feng, F. (2017). VILLAGE—Virtual immersive lan-
guage learning and gaming environment: Immersion and presence.
British Journal of Educational Technology,48(2), 431–450. https://doi.
org/10.1111/bjet.12388
20 HUANG ET AL.

Weizenbaum, J. (1966). ELIZA—A computer program for the study of natu-
ral language communication between man and machine. Communica-
tions of the ACM,9(1), 36–45.
Winkler, R., & Soellner, M. (2018). Unleashing the potential of chatbots in
education: A state-of-the-art analysis. In Academy of Management
Annual Meeting Proceedings. Academy of Management.
Xu, Y., Wang, D., Collins, P., Lee, H., & Warschauer, M. (2021). Same benefits, dif-
ferent communication patterns: Comparing children's reading with a conver-
sational agent vs. a human partner. Computers & Education,161, 4059.
Xu, Y., & Warschauer, M. (2020). Exploring young children's engagement
in joint reading with a conversational agent. In Proceedings of the Inter-
action Design and Children Conference (pp. 216–228). ACM.
Yang, H.-C., & Zapata-Rivera, D. (2010). Interlanguage pragmatics with a
pedagogical agent: The request game. Computer Assisted Language
Learning,23(5), 395–412.
Yin, Q., & Satar, M. (2020). English as a foreign language learner interac-
tions with chatbots: Negotiation for meaning. International Online Jour-
nal of Education and Teaching,7(2), 390–410.
How to cite this article: Huang, W., Hew, K. F., & Fryer, L. K.
(2021). Chatbots for language learning—Are they really useful?
A systematic review of chatbot-supported language learning.
Journal of Computer Assisted Learning,1–21. https://doi.org/
10.1111/jcal.12610
HUANG ET AL.21