Conference PaperPDF Available

Playing around Minimal Pairs to improve pronunciation training

Authors:
Cristian Tejedor-García at Radboud University
Cristian Tejedor-García
Valentín Cardeñoso-Payo at Universidad de Valladolid
Valentín Cardeñoso-Payo
Enrique Cámara-Arenas at Universidad de Valladolid
Enrique Cámara-Arenas
César González-Ferreras
César González-Ferreras
César González-Ferreras
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Show all 5 authorsHide
Download full-text PDF
Read full-text
Download citation

Abstract

Computer Assisted Pronunciation Training (CAPT) apps are becoming widespread to aid learning new languages. However, they are still highly criticized for the lack of the unreplaceable need of direct feedback from a human expert. The combination of the right learning methodology with a gamification design strategy can, nevertheless, increase engagement and provide adequate feedback while keeping users active and comfortable. In this paper, we introduce the second generation of a serious game[1] designed to aid pronunciation training for non-native students of English, Spanish or Chinese. The design of the new version of the game supports a learning methodology which is based in the combination of three different learning strategies: exposure, discriminations and pronunciation[2]. In exposure mode, players are helped to become familiar with the sounds of sequences of minimal pairs or trios, selected by a native linguist and presented at random. When in discrimination mode, users test their ability to discriminate between the phonetics of minimal pairs. They listen to the sound of one of the words in the pair and have to choose the right word on screen. In pronunciation mode, finally, subjects are asked to separately read aloud (and record) both words of each round of minimal pairs lists. Native pronunciation of a word can be played as many times as a user needs. When the test word is correctly uttered by the user, the corresponding icon changes its base colour to green, and gets disabled as a positive feedback message appears. Otherwise, a message with the recognized words appears on the graphical interface and a non-positive feedback message is presented. The word changes its base colour to red and gets disabled after five failures. Speech is recorded and played using commercial off-the-shelf ASR and TTS. Our game adapts to the player as a function of right and wrong answers. Users collect points to reach a " phonetic level " and obtain different achievements, in order to encourage their engagement. There are different language dependent leaderboards based on points too, to increase the desire to play. Sharing results in social networks is another option that is under way. From a pedagogical point of view, the use of Minimal Pairs[3] favours users awareness on the potential risks of producing wrong meanings when the correct phonemes are not properly realized. The discrimination of the words that make up a minimal pair is a challenging task for the ASR, since the phonetic distance between each couple of words can be really small, although clearly perceptible for a native speaker. To be efficient, minimal pairs lists are to be selected by expert linguists for each language. Real use data acquisition and processing is still ongoing , but preliminary results are promising and show that this learning and gaming strategy provides measurable improvement of learners' pronunciation. The app offers an enjoying opportunity for anywhere anytime self-learning, and a tool for teachers to design challenging games.
Content uploaded by Cristian Tejedor-García
Author content
All content in this area was uploaded by Cristian Tejedor-García on Nov 11, 2015
Content may be subject to copyright.
Playing around Minimal Pairs to improve pronunciation training
Cristian Tejedor-García, Valentín Cardeñoso-Payo, Enrique Cámara-Arenas, César González-
Ferreras and David Escudero-Mancebo
Department of Computer Science. Universidad de Valladolid. cristian.tejedor@alumnos.uva.es
Computer Assisted Pronunciation Training (CAPT) apps are becoming widespread to aid
learning new languages. However, they are still highly criticized for the lack of the unreplaceable
need of direct feedback from a human expert. The combination of the right learning methodology
with a gamification design strategy can, nevertheless, increase engagement and provide adequate
feedback while keeping users active and comfortable.
In this paper, we introduce the second generation of a serious game[1] designed to aid
pronunciation training for non-native students of English, Spanish or Chinese. The design of the new
version of the game supports a learning methodology which is based in the combination of three
different learning strategies: exposure, discriminations and pronunciation[2]. In exposure mode,
players are helped to become familiar with the sounds of sequences of minimal pairs or trios, selected
by a native linguist and presented at random. When in discrimination mode, users test their ability to
discriminate between the phonetics of minimal pairs. They listen to the sound of one of the words in
the pair and have to choose the right word on screen. In pronunciation mode, finally, subjects are
asked to separately read aloud (and record) both words of each round of minimal pairs lists. Native
pronunciation of a word can be played as many times as a user needs. When the test word is correctly
uttered by the user, the corresponding icon changes its base colour to green, and gets disabled as a
positive feedback message appears. Otherwise, a message with the recognized words appears on the
graphical interface and a non-positive feedback message is presented. The word changes its base
colour to red and gets disabled after five failures. Speech is recorded and played using commercial
off-the-shelf ASR and TTS.
Our game adapts to the player as a function of right and wrong answers. Users collect points to
reach a “phonetic level” and obtain different achievements, in order to encourage their engagement.
There are different language dependent leaderboards based on points too, to increase the desire to
play. Sharing results in social networks is another option that is under way.
From a pedagogical point of view, the use of Minimal Pairs[3] favours users awareness on the
potential risks of producing wrong meanings when the correct phonemes are not properly realized.
The discrimination of the words that make up a minimal pair is a challenging task for the ASR, since
the phonetic distance between each couple of words can be really small, although clearly perceptible
for a native speaker. To be efficient, minimal pairs lists are to be selected by expert linguists for each
language.
Real use data acquisition and processing is still on-going, but preliminary results are promising
and show that this learning and gaming strategy provides measurable improvement of learners’
pronunciation. The app offers an enjoying opportunity for anywhere anytime self-learning, and a tool
for teachers to design challenging games.
References
[1] David Escudero-Mancebo and Enrique Cámara-Arenas and Cristian Tejedor-García and César
González-Ferreras and Valentín Cardeñoso-Payo, Implementation and Test of a Serious Game
Based on Minimal Pairs for Pronunciation Training, SLaTE 2015, pages 125-130.
[2] E. Cámara-Arenas, Native Cardinality: on teaching American English vowels to Spanish
students, S. de Publicaciones de la Universidad de Valladolid, Ed., 2012.
[3] M. Celce-Murcia, D. M. Brinton, and J. M. Goodwin, Teaching pronunciation: A reference for
teachers of English to speakers of other languages. Cambridge University Press, 1996.
... Este experimento fue el primero en intentar responder a las preguntas de investigación RQ2 (e Issue 2.1, Issue 2.2, Issue 2.3) y RQ3, junto a la pregunta de investigacion RQ1 (e Issue 1.1), tratando aspectos relativos a los objetivos RO1, RO2, RO3 y RO4. Los principales resultados han sido publicados en [18], [19], [20], [21], [22]. En la Sección 7.3 se encuentra la descripción completa de dicho experimento. ...
... Finally, results from the questionnaire provided at the end of the experiment. The most important results are reported in the following subsections and discussed in Chapter 8. Performance-related results have been partially published in [18], [19], [20]; whereas results related to the gamification elements included in the CAPT system have been published in [21], [22]. ...
... 21: Correlation between the software and human raters post-test scores of the English Vowels prototype, adapted from[23]. Learners belong to the experimental group. ...
Design and Evaluation of Mobile Computer-Assisted Pronunciation Training Tools for Second Language Learning
Thesis
Full-text available
  • Sep 2020
The quality of speech technology (automatic speech recognition, ASR, and text–to–speech, TTS) has considerably improved and, consequently, an increasing number of computer-assisted pronunciation (CAPT) tools has included it. However, pronunciation is one area of teaching that has not been developed enough since there is scarce empirical evidence assessing the effectiveness of tools and games that include speech technology in the field of pronunciation training and teaching. This PhD thesis addresses the design and validation of an innovative CAPT system for smart devices for training second language (L2) pronunciation. Particularly, it aims to improve learner's L2 pronunciation at the segmental level with a specific set of methodological choices, such as learner's first and second language connection (L1–L2), minimal pairs, a training cycle of exposure–perception–production, individualistic and social approaches, and the inclusion of ASR and TTS technology. The experimental research conducted applying these methodological choices with real users validates the efficiency of the CAPT prototypes developed for the four main experiments of this dissertation. Data is automatically gathered by the CAPT systems to give an immediate specific feedback to users and to analyze all results. The protocols, metrics, algorithms, and methods necessary to statistically analyze and discuss the results are also detailed. The two main L2 tested during the experimental procedure are American English and Spanish. The different CAPT prototypes designed and validated in this thesis, and the methodological choices that they implement, allow to accurately measuring the relative pronunciation improvement of the individuals who trained with them. Both rater's subjective scores and CAPT's objective scores show a strong correlation, being useful in the future to be able to assess a large amount of data and reducing human costs. Results also show an intensive practice supported by a significant number of activities carried out. In the case of the controlled experiments, students who worked with the CAPT tool achieved better pronunciation improvement values than their peers in the traditional in-classroom instruction group. In the case of the challenge-based CAPT learning game proposed, the most active players in the competition kept on playing until the end and achieved significant pronunciation improvement results.
View
... In particular, game-based results of the Alpha experiment have been partially published in [15]. Performance-related results of the TipTopTalk! prototype have been partially published in [16,20,21]; whereas results related to the gamification elements included in the CAPT tool have been published in [22,23]. Regarding the latest game-based prototype, COP, the main results about performance, motivation, and pronunciation improvement have been published in [17] (Journal Citation Reports, JCR Q1). ...
Design and Evaluation of Mobile Computer-Assisted Pronunciation Training Tools for Second Language Learning: a Ph.D. Thesis Overview
Conference Paper
  • Mar 2021
Recent advances on speech technologies (automatic speech recognition, ASR, and text-to-speech, TTS, synthesis) have led to their integration in computer-assisted pronunciation training (CAPT) tools. However, pronunciation is an area of teaching that has not been developed enough since there is scarce empirical evidence assessing the effectiveness of CAPT tools and games that include ASR/TTS. In this manuscript, we summarize the findings presented in Cristian Tejedor-García's Ph.D. Thesis (University of Valladolid, 2020). In particular, this dissertation addresses the design and validation of an innovative CAPT system for smart devices for training second language (L2) pronunciation at the segmental level with a specific set of methodological choices, such as the inclusion of ASR/TTS technologies with minimal pairs, learner's native-foreign language connection, a training cycle of exposure-perception-production, and individual/social approaches. The experimental research conducted applying these methodological choices with real users validates the efficiency of the CAPT prototypes developed for the four main experiments of this dissertation about English and Spanish as L2. We were able to accurately measure the relative pronunciation improvement of the individuals who trained with them. Expert raters on phonetics' subjective scores and CAPT's objective scores showed a strong correlation, being useful in the future to be able to assess a large amount of data and reducing human costs.
View
... We have also reflected on the degree of engagement generated by the tool [12], [13]. However, our findings in relation to the actual teaching efficiency of the system have been less conclusive: the introduction of corrective feedback [14], [15] allowed us to confirm that there was pronunciation improvement among users after the first few turns, while protracted use of the tool seemed to invariably lead to stagnation. An extra complication concerning the assessment of pronunciation improvement among users had to do with the freedom of movement granted to them and, therefore, with the already mentioned lack of control on the part of the system. ...
Assessing Pronunciation Improvement in Students of English Using a Controlled Computer-Assisted Pronunciation Tool
Article
  • Mar 2020
Over the last few years, we have witnessed a growing interest in computer-assisted pronunciation training (CAPT) tools and the commercial success of foreign language teaching applications that incorporate speech synthesis and automatic speech recognition technologies. However, empirical evidence supporting the pedagogical effectiveness of these systems remains scarce. In this study, a minimal-pair based CAPT tool that implements exposure—perception—production cycles and provides automatic feedback to learners is tested for effectiveness in training adult native Spanish users (English level B1—B2) in the production of a set of difficult English sounds. Working under controlled conditions, a group of users took a pronunciation test before and after using the tool. Test results were considered against those of an in-classroom group who followed similar training within the traditional classroom setting. Results show a significant pronunciation improvement among the learners who used the CAPT tool, as well as a correlation between human rater's assessment of post-tests and automatic CAPT assessment of users.
View
... This application represents an evolution of previous serious games [4,5,6] designed for pronunciation training of L2 by non-native. All of them rely on the minimal pairs methodology [7] and are within the context of research projects related to the development and testing of software tools and games for foreign language learning (TIN2014-59852-R and VA050G18). ...
Japañol: a mobile application to help improving Spanish pronunciation by Japanese native speakers
Conference Paper
Full-text available
  • Nov 2018
In this document, we describe the mobile application Japañol 1 , a learning tool which helps pronunciation training of Spanish as a foreign language (L2) at a segmental level. The tool has been specifically designed to be used by native Japanese people , and implies a branch of a previous CAPT gamified tool TipTopTalk!. In this case, a predefined cycle of actions related to exposure, discrimination and production is presented to the user, always under the minimal-pairs approach to pronunciation training. It incorporates freely available ASR and TTS and provides feedback to the user by means of short video tutorials, to reinforce learning progression.
View
... We were able to assess user's pronunciation level in a L2, that is, those with a certified higher level consistently reached better scores in the game [15]. Besides, we also have discussed that the introduction of corrective feedback [16] allowed us to confirm that there was pronunciation improvement among users after the first stages of use. However, continued use of the tool seemed to invariably lead to stagnation. ...
Improving Pronunciation of Spanish as a Foreign Language for L1 Japanese Speakers with Japañol CAPT Tool
Conference Paper
Full-text available
  • Nov 2018
Availability and usability of mobile smart devices and speech technologies ease the development of language learning applications, although many of them do not include pronunciation practice and improvement. A key to success is to choose the correct methodology and provide a sound experimental validation assessment of their pedagogical effectiveness. In this work we present an empirical evaluation of Japañol, an application designed to improve pronunciation of Spanish as a foreign language targeted to Japanese people. A structured sequence of lessons and a quality assessment of pronunciations before and after completion of the activities provide experimental data about learning dynamics and level of improvement. Explanations have been included as corrective feedback, comprising textual and audiovisual material to explain and illustrate the correct articulation of the sounds. Pre-test and post-test utterances were evaluated and scored by native experts and the ASR, showing a correlation over 0.86 between both predictions. Sounds [s], [fl], [ɾ] and [s], [fɾ], [θ] explain the most frequent failures for discrimination and production, respectively, which can be exploited to plan future versions of the tool, including gamified ones. Final automatic scores provided by the application highly correlate (r>0.91) to expert evaluation and a significant pronunciation improvement can be measured.
View
... Three main elements are involved in our system, an Android client application, an own web server and external services provided by Google. See references [13][12] [11] for more specific details. Figure 1 represents the conceptual architecture of the Android client application. The Control module includes the application's business logic. ...
TipTopTalk! Mobile application for speech training using minimal pairs and gamification
Conference Paper
Full-text available
  • Nov 2016
This demonstration describes the TipTopTalk! mobile application , a serious game for foreign language (L2) pronunciation training, based on the minimal-pairs technique. Multiple Spoken Language Technologies (SLT) such as speech recognition and text-to-speech conversion are integrated in our system. User's interaction consists in a sequence of challenges along time, for instance exposure, discrimination and production exercises. The application implements gamification resources with the aim of promoting continued practice. A specific feedback is also given to the user in order to avoid the performance drop detected after the protracted use of the tool. The application can be used in different languages , such as Spanish, Portuguese (European and Brazilian), English, Chinese, and German.
View
... While such tools undoubtedly engage users in learning-oriented practice, there have been very few attempts to objectively assess the actual improvement attained by them [2] [3]. In the present paper, we show the performance results of users of TipTopTalk! [4] [5], a serious game designed for L2 pronunciation training and testing. ...
Measuring pronunciation improvement in users of CAPT tool TipTopTalk!
Conference Paper
Full-text available
  • Sep 2016
We present a L2 pronunciation training serious game based on the minimal-pairs technique, incorporating sequences of exposure, discrimination and production, and using text-to-speech and speech recognition systems. We have measured the quality of users' production during a period of time in order to assess improvement after using the application. Substantial improvement is found among users with poorer initial performance levels. The program's gamification resources manage to engage a high percentage of users. A need is felt to include feedback for users in future versions with the purpose of increasing their performance and avoiding the performance drop detected after protracted use of the tool.
View
Acquisition d'une langue étrangère et didactique de la phonétique anglaise en France
Thesis
  • Nov 2019
De nombreuses études ont démontré que les apprenants français de l’anglais peinent à acquérir une bonne prononciation de cette langue. Leur maitrise des segments, mais aussi celle de la prosodie de l’anglais reste le plus souvent très fragile, malgré un apprentissage précoce de la langue en milieu scolaire puisque les élèves français débutent l’apprentissage de l’anglais dès le CP. Ceci est surprenant puisque le Ministère indique dans ses directives au niveau national vouloir rendre prioritaire l’enseignement de l’oral dans les classes de langues. La littérature ainsi qu’une étude menée dans le cadre de cette thèse suggèrent que ce faible niveau serait dû à un enseignement dont la composante phonétique serait quasi-absente, notamment en raison d’une non-formation des professeurs en didactique de la prononciation. Ainsi, nous avons souhaité mener une étude expérimentale dont le but était de voir s’il était possible d’intégrer cette composante au programme efficacement et de manière à la fois utile pour les élèves et pour l’enseignante. Pour cela, deux classes de seconde orléanaises ont été sélectionnées. L’une d’entre elle a suivi à raison d’une heure par semaine un entrainement renforcé en phonétique pendant 10 semaines, et l’autre non. Trois sessions d’évaluation réparties sur 22 semaines ont permis d’établir la progression de la qualité d’anglais oral des élèves des deux classes, et donc de tester l’efficacité de l’entrainement prodigué. Les résultats suggèrent que l’entrainement proposé aux élèves de la classe testée leur a permis d’améliorer significativement leur prononciation de l’anglais, tandis que l’enseignement classique qu’ont suivi les élèves de la classe témoin n’a pas rendu possible une telle progression.
View
View
Design and Evaluation of a Mobile Application for Second Language Pronunciation Training based on Minimal Pairs
Conference Paper
Full-text available
  • Nov 2018
A correct pronunciation is crucial to grasp an adequate communication ability in a foreign language. Nevertheless, traditional foreign language learning systems usually focus on the development of linguistic competencies related to grammar or lexicon. The quality of spoken language technologies (speech synthesis and recognition) has noticeably improved and, as a consequence, an increasing number of second language learning tools have included it. Serious games show great potential as educational resources. Their social versions encourage competitive and colabora-tive learning scenarios for individuals and groups. The proposal of this thesis project focuses on developing a system for improving foreign language pronunciation. A serious game to be run on a mobile platform is proposed, and a learning method which makes use of speech technology and is designed to guide and motivate users to a continuous practice over time, in a controlled way.
View
View
Arenas and Cristian Tejedor-García and César González-Ferreras and Valentín Cardeñoso-Payo, Implementation and Test of a Serious Game Based on Minimal Pairs for Pronunciation Training
  • Jan 2015
  • 125-130
  • David Escudero
  • Mancebo
  • Enrique Cámara
David Escudero-Mancebo and Enrique Cámara-Arenas and Cristian Tejedor-García and César González-Ferreras and Valentín Cardeñoso-Payo, Implementation and Test of a Serious Game Based on Minimal Pairs for Pronunciation Training, SLaTE 2015, pages 125-130.
Native Cardinality: on teaching American English vowels to Spanish students
  • Jan 2012
  • E Cámara-Arenas
E. Cámara-Arenas, Native Cardinality: on teaching American English vowels to Spanish students, S. de Publicaciones de la Universidad de Valladolid, Ed., 2012.