ArticlePDF Available

Abstract

Modern Computer Assisted Language Learning (CALL) systems use speech recognition to give students the opportunity to build up their spoken language skills through interactive practice with a mechanical partner. Besides the obvious benefits that these systems can offer, e.g. flexible and inexpensive learning, user interaction in this context can often be problematic. In this article, the authors introduce a parallel layer of feedback in a CALL application, which can monitor interaction, report errors and provide advice and suggestions to students. This mechanism combines knowledge accumulated from four different inputs in order to decide on appropriate feedback, which can be customized and adapted in terms of phrasing, style and language. The authors report the results from experiments conducted at six lower secondary classrooms in German-speaking Switzerland with and without this mechanism. After analyzing approximately 13,000 spoken interactions it can be reasonably argued that their parallel feedback mechanism in L2 actually does help students during interaction and contributes as a motivation factor.
DOI: 10.4018/IJCALLT.2017070101

Volume 7 • Issue 3 • July-September 2017
Copyright © 2017, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.



Nikos Tsourakis, University of Geneva, Geneva, Switzerland
Claudia Baur, University of Geneva, Geneva, Switzerland
Manny Rayner, University of Geneva, Geneva, Switzerland

Modern Computer Assisted Language Learning (CALL) systems use speech recognition to give
students the opportunity to build up their spoken language skills through interactive practice with
a mechanical partner. Besides the obvious benefits that these systems can offer, e.g. flexible and
inexpensive learning, user interaction in this context can often be problematic. In this article, the
authors introduce a parallel layer of feedback in a CALL application, which can monitor interaction,
report errors and provide advice and suggestions to students. This mechanism combines knowledge
accumulated from four different inputs in order to decide on appropriate feedback, which can be
customized and adapted in terms of phrasing, style and language. The authors report the results from
experiments conducted at six lower secondary classrooms in German-speaking Switzerland with
and without this mechanism. After analyzing approximately 13,000 spoken interactions it can be
reasonably argued that their parallel feedback mechanism in L2 actually does help students during
interaction and contributes as a motivation factor.

CALL, Children, Feedback, German, Knowledge-Based

There is a growing interest in Computer Assisted Language Learning (CALL) applications. A major
problem for many beginner students is the difficulty of finding conversation partners; given their
minimal skills, they feel diffident about asking native speakers to talk with them. Some researchers
in the sociolinguistic tradition, e.g. (Kubota & Lin, 2009), describe this in terms of difficulty in
establishing a valid L2 identity, an analysis which, as they argue, is particularly apt for immigrant
learners. Without conversation practice, it is impossible to “automatize” (Luu, 2016), and convert
the conscious declarative knowledge acquired from classroom or textbook work into unconscious
procedural skills. Even a simple dialogue system may however tangibly help the beginner learner
progress to a level where they are confident enough to engage native speakers in conversation.
As in typical classroom courses, CALL programs need to report upon each interaction; in
particular provide useful feedback to students. In the realm of language learning the role of feedback is
1

Volume 7 • Issue 3 • July-September 2017
2
still very controversial; a double-edged sword according to (Kluger & DeNisi), as whether to correct,
what to correct, how to correct, and when to correct. Methodologists suggest for example that teachers
need to focus their attention on a few error types rather than try to address all the errors (Ellis, 2009).
Corrective feedback has been studied extensively in typical educational settings and has been reported
to provide minimal effects in the field of First Language Acquisition (FLA). Conversely, in the field
of Second Language Acquisition (SLA) researchers appear to agree concerning the significance of
its role in the process of SLA (El Tatawy, 2002).
Besides educational feedback, the interaction in the context of a CALL program involves a
software system that imposes several technical challenges per se. Software installation, microphone
configuration, learning how to use the interface, etc., are typical impediments. An important design
rule of a graphical user interface is visibility, so that all possible actions can be made visible and,
therefore, easily discoverable (Norman, 2010). In this way the system can often be learned through
exploration. Visual user interfaces have also significantly benefited from the introduction of the
Windows, Menus, Icons and Pointers (WIMP) widgets that offered a unified interaction scheme. A
new visual interface can rely on the knowledge accumulated over many years, so that users do not
need to learn new ways of doing things. For example, the ‘×’ in the upper right or left corner of a
window is a global signifier for closing the current window. Unfortunately, we lack such signifiers
in interfaces that combine visual interaction and speech. For example, it is not obvious what a user
can say or do to abort an ongoing speech recognition action. As pointed out in (Tsourakis, 2014)
multimodal interfaces that normally lack this type of functionality suffer.
Interaction can become even more problematic if the student is overwhelmed by the functionalities
the system offers. A typical example is the increasingly common trend in CALL applications of
incorporating gamification techniques in order to increase student motivation (Baur, 2013). The
main point of these kinds of systems is to give the student an opportunity to accelerate their language
development by accumulating more “flying hours” of active second language use. Computers are,
unfortunately, less interesting conversational partners than live human beings; if the interaction
becomes too much like a mechanical drill exercise, students will easily become bored and leave.
Turning the system into some kind of game may redress the balance. Videogames are notoriously
addictive, and there has recently been a great deal of interest in trying to exploit this fact to “gamify”
software applications (Werbach & Hunter, 2012). Gamified applications are based on elements like
scores, badges and leaderboards, but the rules that govern interaction may not always be transparent
to end-users.
For the reasons outlined above we decided to implement a feedback module and embed it in
CALL-SLT, a computer assisted language learning system (Rayner et al., 2014). The CALL-SLT
tool offers students of English the opportunity to train their receptive and productive language skills
in various conversational dialogue scenarios (Baur, 2015). The interactive element of this approach
allows the students not only to exercise their knowledge of grammar and vocabulary, but more
importantly their real-life conversational skills. The aim of creating the module is threefold: (1) expose
system functionalities during interaction, (2) help students overcome certain interaction problems and
(3) increase their motivation during the gameplay. The current interface already provides different
kind of feedback, e.g. reporting failed recognition, informing on total score, etc. The question that
immediately arises is why we need to introduce an extra feedback mechanism. Based on several user
studies with CALL-SLT (Rayner et al., 2010), (Bouillon et al., 2011), (Rayner & Tsourakis, 2013),
we identified the following obstacles when using the system:
Users may be overwhelmed by too much feedback, leading to information overload;
Users may not be aware of certain system functionalities;
Users may not understand what triggered certain events;
It is difficult to unify the feedback messages in terms of phrasing, style and language.
18 more pages are available in the full version of this
document, which may be purchased using the "Add to Cart"
button on the product's webpage:
www.igi-global.com/article/constructing-knowledge-based-
feedback-in-the-context-of-an-interactive-spoken-call-
application/190910?camid=4v1
This title is available in InfoSci-Journals, InfoSci-Journal
Disciplines Library Science, Information Studies, and
Education, InfoSci-Select, InfoSci-Educational Leadership,
Administration, and Technologies eJournal Collection,
InfoSci-Select, InfoSci-Select, InfoSci-Select. Recommend
this product to your librarian:
www.igi-global.com/e-resources/library-
recommendation/?id=2
Related Content
Vocabulary Learning and Consolidation with Mobile Application
Dongshuo Wang, Bin Zou and Minjie Xing (2014). International Journal of Computer-
Assisted Language Learning and Teaching (pp. 101-112).
www.igi-global.com/article/vocabulary-learning-and-consolidation-with-
mobile-application/109833?camid=4v1a
Digital Classrooms Equipped with Tablet Computers as the Students'
Learning Companions
(2015). Intelligent Web-Based English Instruction in Middle Schools (pp. 236-266).
www.igi-global.com/chapter/digital-classrooms-equipped-with-tablet-
computers-as-the-students-learning-companions/119321?camid=4v1a
Chinese Students' Perceptions of Using Mobile Devices for English Learning
Bin Zou and Xinxin Yan (2014). International Journal of Computer-Assisted Language
Learning and Teaching (pp. 20-33).
www.igi-global.com/article/chinese-students-perceptions-of-using-mobile-
devices-for-english-learning/117192?camid=4v1a
The Role of Teachers and Their Beliefs in Implementing Technology-
Mediated Language Learning: Implications for Teacher Development and
Research
Geoff Lawrence (2014). International Journal of Computer-Assisted Language
Learning and Teaching (pp. 59-75).
www.igi-global.com/article/the-role-of-teachers-and-their-beliefs-in-
implementing-technology-mediated-language-learning/122842?camid=4v1a
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
This paper presents two new corpus-based studies of feedback in the domain of teaching Spanish as a foreign language, concentrating on the type and frequency of different feedback moves, as well as what happens in the moves that follow the feedback. In particular, as well as looking at positive feedback, it concentrates on two general kinds of negative feedback strategies: (1) Giving-Answer Strategies (GAS), where the teacher directly gives the desired target form or indicates the location of the error, and (2) Prompting-Answer Strategies (PAS), where the teacher pushes the student less directly to notice and repair their own error. Investigating the GAS/PAS distinction sheds light on the relative importance for Intelligent Computer-Assisted Language Learning (ICALL) systems of knowledge construction from interaction, which many believe is crucial for effective learning from ITS. The main finding here is that, although GAS occur more frequently than PAS in both corpora, it is the PAS that are more effective, in terms of eliciting explicit repairs by the students. The first study takes place in a classroom context, whereas the second, smaller, study looks at tutorial inter- actions. This makes it possible to investigate the extent to which the mode of interaction influences the frequency and effectiveness of feedback moves, as well as to look at how concepts such as "wait time" are relevant to explain moves that are ineffective. The paper concludes by using these results to make recommendations about how to choose appropriate feed- back moves in ICALL systems.
Article
Full-text available
With their playful game-oriented nature, virtual learning environments can form constructive ecosocial settings for enacting game-driven collaborative tasks that have real pedagogical, learning, and teaching implications. Even though multiple studies have been undertaken in virtual learning environments, there is a dearth of studies on the constellation of complex interrelated skills and knowledge that second or foreign language educators integrate in actual situated contexts to design and offer task-oriented game-driven learning activities. Building on Compton's (2009) framework, this study investigates the complex set of technological, pedagogical, and evaluation skills that an expert Austrian educator, Franziska integrated to design a multifaceted game-oriented plot in the virtual village of Chatterdale in OpenSim in order to engage thirteen-year-old Austrian and Norwegian high school students in oral interactions during three slightly different task-oriented quests to solve an alien mystery. Data analysis of an interview, reflective comments on a wiki, and follow-up emails indicate that the design and integration of an epistemic game in praxis forms a collaborative endeavor involving the integration of a set of complex and multifaceted sets of skills and knowledge. The study investigates the skills that emerged during two stages, the planning, preparation, and student training stage and integration stage. The findings of this study can be used to broaden the pedagogical discussion on the skills and knowledge that second or foreign language teachers need to acquire and apply to design successful playful task-driven learning quests in virtual learning environments.
Conference Paper
Full-text available
The European 'Lifelong Learning Programme' (LLP) project 'Games Online for Basic Language learning' (GOBL) aimed to provide youths and adults wishing to improve their basic language skills access to materials for the development of communicative proficiency in Dutch, French, and English through web-based mini-games. These mini-games were tested in four countries: The Netherlands (Dutch), Belgium (French), United Kingdom and South-Africa (English). Four types of mini-games were developed, and in two of them users can use 'automatic speech recognition' (ASR) to support spoken interaction. In the current paper we will focus on the English versions of these two games that were tested in the United Kingdom and South-Africa. The analyses that are presented in this paper were conducted to determine what users' perceptions are about mini-games with and without speech input and ASR and which aspects of the speech-enhanced games are strongly related to each other.
Thesis
Full-text available
Communicative second language skills are becoming increasingly important in today’s fast-moving and global world. There are a number of approaches to attain these competences, such as conversation trainings with native speakers, language stays abroad, and traditional language courses. Whereas the first two approaches are time- and cost-intensive, the third alternative is the most commonly used approach to Second Language Acquisition for children of school age. However, one big challenge of this latter approach is the lack of time for conversational training during limited teaching hours. A promising approach to tackle this problem is Computer Assisted Language Learning (CALL). The central objective of this thesis is to evaluate the potential of CALL in the Swiss education system. For this purpose we developed an interactive speech-enabled English dialogue course (CALL-SLT) for Germanophone lower secondary school students that is customized to the content taught in traditional school classes. In a large-scale experiment – including 15 school classes – we evaluated whether the regular use of CALL-SLT helped students improve their communicative second language skills and whether the integration of CALL in traditional teaching was well received by both students and teachers. To assess these questions, both objective and subjective measures were taken into account, such as written placement tests, recordings of spoken system interactions, and various questionnaires providing information on participants’ personal backgrounds, and students’ and teachers’ qualitative feedback on CALL-SLT. Results of this experiment support our hypothesis that students who regularly trained their competences with CALL-SLT significantly improved their post-experimental competences. We demonstrated that the frequent use of CALL-SLT resulted in statistically significant improvement in second language (L2) oral competences, such as pronunciation, fluency, and the correct use of vocabulary in communicative L2 speech. With regard to written L2 competences we showed that orally acquired vocabulary competences could be transferred to written language by demonstrating that highly active CALL-SLT users statistically significantly improved their post-experimental written vocabulary competences. Qualitative feedback from students and teachers also suggested that CALL-SLT was a well-received supplement to traditional teaching methods. The evaluation of the potential of gamification elements, such as scores and badges, as a motivation enhancer could not be proven in this study. However, although gamification did not prove to enhance students’ motivation from an objective point of view, subjective user feedback indicated that the gamified version of CALL-SLT was preferred by a majority of all participants.
Conference Paper
Full-text available
In this paper we report on the ASR-based CALL system DISCO: Development and Integration of Speech technology into COurseware for language learning. The DISCO system automatically detects pronunciation and grammar errors in Dutch L2 speaking and generates appropriate, detailed feedback on the errors detected. We briefly introduce DISCO and present the results of a first evaluation of the complete system.
Article
Commercial off-the-shelf, or vernacular digital games have been studied in foreign language (FL) pedagogy. However, few have examined how a game design interacts with pedagogical mediation. This study explores the interplay of game design and pedagogical mediation for learners of Japanese. Learners played a vernacular game in Japanese and used activity worksheets and a reference list. Findings from the vocabulary tests and questionnaires show that materials can focus learner attention on particular vocabulary, but that vocabulary functionality also plays a role in learning. In-game vocabulary was identified as primary or secondary based on its necessity for successful game completion. When both types of vocabulary were combined, results showed good retention between the posttest and delayed test. When separated, the primary vocabulary was retained well, but the secondary vocabulary was not retained. This confirms that in-game vocabulary functionality impacts learning and implicates the careful design of supplemental materials to guide FL learners' attention.
Article
This paper proposes a sentence stress feedback system in which sentence stress prediction, detection, and feedback provision models are combined. This system provides non-native learners with feedback on sentence stress errors so that they can improve their English rhythm and fluency in a self-study setting. The sentence stress feedback system was devised to predict and detect the sentence stress of any practice sentence. The accuracy of the prediction and detection models was 96.6% and 84.1%, respectively. The stress feedback provision model offers positive or negative stress feedback for each spoken word by comparing the probability of the predicted stress pattern with that of the detected stress pattern. In an experiment that evaluated the educational effect of the proposed system incorporated in our CALL system, significant improvements in accentedness and rhythm were seen with the students who trained with our system but not with those in the control group.
Article
This study presents the results of applying automated speech scoring technology to English spoken responses provided by non-native children in the context of an English proficiency assessment for middle school students. The assessment contains three diverse task types designed to measure a student's English communication skills, and an automated scoring system was used to extract features and build scoring models for each task. The results show that the automated scores have a correlation of r = 0.70 with human scores for the Read Aloud task, which matches the human-human agreement level. For the two tasks involving spontaneous speech, the automated scores obtain correlations of r = 0.62 and r = 0.63 with human scores, which represents a drop of 0.08 - 0.09 from the humanhuman agreement level. When all 5 scores from the assessment for a given student are aggregated, the automated speaker-level scores show a correlation of r = 0.78 with human scores, compared to a human-human correlation of r = 0.90. The challenges of using automated spoken language assessment for children are discussed, and directions for future improvements are proposed.