ArticlePDF Available

Using Educational Games for Sign Language Learning -A SignWriting Learning Game: Case Study

  • The National Higher Engineering School of Tunis, University of Tunis, Tunisia
  • Arab League Educational, Cultural and Scientific Organization (ALESCO), Tunis, Tunisia

Abstract and Figures

Apart from being used as a means of entertainment, computer games have been adopted for a long time as a valuable tool for learning. Computer games can offer many learning benefits to students since they can consume their attention and increase their motivation and engagement which can then lead to stimulate learning. However, most of the research to date on educational computer games, in particular learning versions of existing computer games, focused only on learner with typical development. Rather less is known about designing educational games for learners with special needs. The current research presents the results of a pilot study. The principal aim of this pilot study is to examine the interest of learners with hearing impairments in using an educational game for learning the sign language notation system SignWriting. The results found indicated that, overall, the application is useful, enjoyable and easy to use: the game can stimulate the students' interest in learning such notations.
Content may be subject to copyright.
Bouzid, Y., Khenissi, M. A., Essalmi, F., & Jemni, M. (2016). Using Educational Games for Sign Language Learning - A
SignWriting Learning Game: Case Study. Educational Technology & Society, 19 (1), 129141.
ISSN 1436-4522 (online) and 1176-3647 (print). This article of the Journal of Educational Technology & Society is available under Creat ive Commons CC-BY-ND-NC
3.0 license (https://creativecommons.o rg/licenses/by-nc-nd/3.0/). For further queries, please contact Journal Editors at
Using Educational Games for Sign Language Learning - A SignWriting
Learning Game: Case Study
Yosra Bouzid*, Mohamed Ali Khenissi, Fathi Essalmi and Mohamed Jemni
Research Laboratory of Technologies of Information and Communication & Electrical Engineering (LaTICE),
University of Tunis, Tunisia // // // //
*Corresponding author
Apart from being used as a means of entertainment, computer games have been adopted for a long time as a
valuable tool for learning. Computer games can offer many learning benefits to students since they can consume
their attention and increase their motivation and engagement which can then lead to stimulate learning.
However, most of the research to date on educational computer games, in particular learning versions of existing
computer games, focused only on learner with typical development. Rather less is known about designing
educational games for learners with special needs. The current research presents the results of a pilot study. The
principal aim of this pilot study is to examine the interest of learners with hearing impairments in using an
educational game for learning the sign language notation system SignWriting. The results found indicated that,
overall, the application is useful, enjoyable and easy to use: the game can stimulate the students’ interest in
learning such notations.
People with hearing disabilities, Avatar technology, Deaf education, Educational games, Technology-assisted
As for any other group, education is critical to expanding the life prospects of people with hearing disabilities since it
helps them to gain knowledge they need to succeed in today’s communities and the world of information and
communication. Unfortunately, despite serious efforts to date, many deaf students continue to experience difficulties
in achieving normative standards of literacy. The most recent available data provided by the World Federation of
Deaf indicate that the enrollment rate and literacy achievement of deaf children is far below the average for the
population at large and that there is at least 80 % of the world’s 70 million deaf people are illiterate or semi-literate.
How literacy is taught, utilized, and potentially mastered is, in fact, one of the most discussed, researched, and highly
contentious topics in the field of deaf education and deaf studies (Harris & Marschark, 2011). Among the most
notable results which have been drawn by different researchers in this field is that high levels of illiteracy and low
academic achievements among these hearing impaired students are associated to the discrepancy between their
incomplete spoken language system, and the demands of writing and reading through a speech-based system (Geers
& Hayes, 2011; Nussbaum et al., 2012; Zamfirov & Saeva, 2013). It is important to know that reading and writing
require two related capabilities; firstly you must be familiar with a language and secondly, you must understand the
mapping between that language and the printed word (Chamberlain & Mayberry, 2000). Deaf and Hard of Hearing
learners (DHH) are disadvantaged on both counts. For example, learning to read requires DHH learners to learn the
mapping between the spoken language and the printed words, and this is not easy for them because they do not have
access to phonological code and many do not know the language well since it is often considered as a second
language for them (Goldin-Meadow & Mayberry, 2001).
Actually, providing these learners with a suitable writing form in their first language could be a help to them. It has
been demonstrated that an appropriate sign language written form can offer deaf learners the possibility to process
written linguistic information provided in a syntactic structure that reflects the structure of the corresponding sign
language. Moreover, such written form can be very useful to improve the ability of these signers to comprehend and
acquire the written versions of oral languages (Vendrame et al., 2013; Guimarães et al., 2014). At present,
SignWriting (SW) is already one of the best known writing systems which are currently being used as an educational
tool in several pilot projects around the world. This system deemed adequate to transcribe the visual nature of
signing through highly iconic symbols and its practical usage in literacy education has had a great impact on
the education of deaf children (Kato, 2008). Nonetheless, it’s fair to say that a training to learn to interpret such static
transcriptions is needed for novice readers, who are accustomed to the use of their preferred language in a visual-
gestural modality.
Fortunately, the emergence of Information and Communication Technology (ICT) has brought new hopes and
opportunities for these learners. The rapid evolution of a vast range of newer digital technologies has made the role
of such assistive technologies, in educating deaf children, even more crucial than ever before. For instance, the
adoption of visual forms of ICTs (e.g., immersive multimedia, 3D animation, virtual reality and video conferencing)
within educational settings can facilitate the acquisition and absorption of knowledge, increase learner motivation
and engagement, and enhance teacher training (Hameed, 2007; Haddad & Jurich, 2002; Passey et al., 2004).
Another promising and interesting alternative to reinforce learning experience, today, is the use of educational
games. Indeed, it has been proved by experience that the usage of educational games can consume the attention of
learners and increase their motivation and engagement which can then lead to stimulate learning (Bourgonjon et al.,
2011; McClarty et al., 2012). Educational games can increase creativity, improve self-confidence and provide visual,
tactile and intellectual stimulation (Griffiths, 2002). However, it should be noted that most of the research to date on
educational games focused only on learner with typical development. Rather less is known about designing
educational games for learners with special needs, including the Deaf and Hard of Hearing (DHH).
In the current research, we are interested in combining these two approaches to benefit from an effective and useful
educational game geared toward the DHH learners. Indeed, we believe that the involvement of ICT, especially
virtual avatar technology, into a learning game could be a suitable solution to stimulate the learning interest of these
disabled students and make the learning process more meaningful and enjoyable. The proposed game, namely
MemoSign, aims essentially to foster and promote the vocabulary acquisition for DHH learners in both signed and
spoken languages. More specifically, MemoSign would offer an additional support to learn the sign language written
form, SignWriting (Sutton, 2015), and this by rendering its notations’ content in visual-gestural modality through a
3D signing avatar. We will seek, through the present study, to examine the deaf learners’ interest in using such
educational game for learning SignWriting notations and new vocabularies. In particular, we want to know whether
the game is helpful and appreciated for them. Thus, the research question of the present study is the following: What
is the students’ opinion regarding the usability of the MemoSign game?
The remainder of this paper is organized as follows: the next section starts by highlighting the main benefits of using
avatar technology and computer games within the educational process. The section 3 contextualizes the contribution
by reporting related works. Section 4 presents the new educational game for DHH learners. Section 5 describes the
evaluation method and the empirical results. Finally, section 6 concludes the paper with a summary of the work and
future research directions.
Literature review
Today, the emergence of assistive technologies has opened great opportunities for improving the learning
performance of students with hearing disabilities. These technological tools have a great potential to enhance the
quality of education and this by adopting appropriate techniques suited to the learners’ abilities. We will address in
this section a particular view to the benefits of using avatar technology and computer games within education.
The educational benefits of avatar technology
For students with severe hearing disabilities, the use of computer animated avatars within educational contexts is
proving to be successful and holds particular promise. The 3D characters can act as a powerful communication
medium for deaf learners to display knowledge in sign language and make instructional materials completely
accessible to them (Vesel, 2005; Adamo-Villani & Hayward, 2010; Kipp et al., 2011, Jaballah & Jemni, 2013).
Besides, by appearing on screen as embodied entities, whether humans, or anthropomorphized characters and
animals, these graphical entities can increase effectively learners’ attention and motivate them to keep interacting
with the content presented (Mahmood & Ferneley, 2006; Deuchar & Nodder, 2003). It may seem, at the first glance
that recording a real signer with a video camera would be a reasonable approach, but digitized video lacks the
flexibility of avatar animation systems. In fact, 3D animation systems provide a low-cost and effective means for
adding sign language translation to any type of media because animation is actually a much easier type of data than
video to store. Signing avatars can be produced anywhere at any time with a relatively low cost in order to be easily
integrated in any educational application.
The educational benefits of computer games
Despite the controversy surrounding the usefulness of computer games in encouraging learning (Barlett et al., 2009;
Prot et al., 2012), a large amount of evidence proves that such games could be an effective way to impart knowledge
and provide personalized learning opportunities for students. Many researchers working in the field of game studies
argue that besides being a strong motivational attractive, the usage of such software category as learning objects can
amplify the students’ potential of exploration and imagination, providing moment recreation to didactics, involving
investigation, reflection and learning (Mitchell & Savill-Smith, 2004; Egenfeldt-Nielsen, 2007; Silveira et al., 2011).
In this sense, Papastergiou (2009) and Gee (2006) claimed that computer games are hugely successful in engaging
and motivating learners to spend more time and effort on problem solving and learning skills. Nevertheless, it is
important to point out that designing and developing educational games that could effectively support the process of
learning, need to satisfy the following criteria: the educational games should be designed properly, meet the abilities
of the students, integrated with curriculum and classroom activities, and monitored by parents or teachers
(Bourgonjon et al., 2011).
Related work
Despite the effectiveness of computer games in enhancing learners’ motivation and engaging them in learning, the
majority of the practice and research around the use of these games has been undertaken only with students with
typical development. Designing appropriate educational games for learners with special needs is unfortunately less
known. We will present in this section a brief overview of existing works that address the development of computer
games for the DHH learners.
CopyCat is an interactive educational video game to develop American Sign Language skills (ASL) in younger
children. Using gesture recognition techniques, CopyCat allows deaf children to communicate with the computer
using ASL and encourages them to practice signing in an enjoyable way (Henderson et al., 2005). CopyCat refers
here to Iris, the main character of the game. Iris is a white cat whose kittens are hiding in the backyard. The player’s
role is to help Iris find her kittens by signing a phrase such as “Black kitten under the chair.The game interface
includes a tutorial video demonstrating the correct signs, live video (providing input to the gesture recognition
system and feedback to the child via the interface), and Iris the cat, to execute the child’s instructions. It should be
noted that, before playing the game, the child must wear colored gloves with wrist-mounted accelerometers and sit in
front of the computer equipped with a video camera for the computer vision recognition system.
Sign my World
Sign my World is a mobile video game developed by the Seek and Sign research project to aid deaf children to learn
the Australian Sign Language (Auslan). The game aims to support deaf children to be familiar with the appearance of
common nouns and verb signs. The game interface has a 2D cartoon like style and bright colors. It began as a single
sample area (a bedroom) containing a number of interactive objects. When an interactive object is clicked, an image
and word are displayed, as on a flash card; followed by the video of the Auslan sign for that object. This is intended
to allow the child to make associations between the object and the sign (Korte et al., 2012).
Virtual Sign Game
Virtual Sign Game is a didactic game developed by the Virtual Sign Project with the collaboration with the research
group GILT (Graphics, interaction & learning technologies). The main goal of this game is to facilitate the learning
of the Portuguese sign language and to improve the dexterity of those who already know it (Escudeiro et al., 2014).
In Virtual Sign game, the player controls a synthetic character that interacts with various objects and non-player
characters with the aim of collecting several gestures from the Portuguese Sign Language. These gestures can then be
represented by the character in order to provide the user a chance to visualize and train the various existing gestures.
To improve the interaction between the player and the main character, a VirtualSign Translator has been connected to
the game using a translator Kinect and gloves. The VirtualSign application detects and translates the gestures that the
user makes, saving the one with higher probability of success.
To the best of our knowledge, there have been no studies evaluating the impact of using such games on learning sign
language for DHH learners. Besides, none of these games are available on the web and thus do not provide a social
experience for their players. Indeed, due to the use of special equipment and non-standard ways in which
applications are developed, the majority of these educational tools is limited to academic and research environments.
MemoSign: A new learning game for deaf learners
In order to overcome the serious difficulties they face in acquiring academic skills in spoken language and helping
them to adopt an appropriate manner for transcribing their first language in a written form, we propose in the present
section a learning game suitable to the learning needs of deaf and hearing impaired students. The learning game is
called MemoSign; it federates the use of a learning version of the Memory Match Game, avatar technology, and the
sign language writing system SignWriting (Figure 1).
Figure 1. The combination of avatar technology and computer game to learn SignWriting
Learning version of Memory Match Game (LMMG)
The original version of Memory Match Game (Zwick & Paterson, 1993) consists of several cards that have pictures
on one side. The number of cards is always even. Typically same picture is printed on two cards. All of the cards are
mixed up and laid face down on a surface. The game is designed for a single player, although the two player mode is
available. In each turn, the player selects a card to flip it over. If the next card selected by the player matches the first
card, both cards disappear from the surface. The objective of the game is to turn over pairs of matching cards with an
aim to get rid of all cards in the least possible trials. In the original version of the Memory Match Game, all cards
hold only visual information. However, in the learning version of this game (Khenissi et al., 2014), which is
available online at:, other types of information has been
added. Precisely, it uses eight types of pair of cards: Visual Visual; Visual Word; Visual Sound; Word Word;
Word Sound; Sound Sound; Calculates Calculates and Calculates Sound.
Synthesizing SignWriting notations
SignWriting (SW) is a writing formalism developed at the University of Copenhagen by Valerie Sutton in 1974 with
the intention to record signed languages for research purposes. Over the years, such formalism has evolved with the
aid of many deaf people and it has proven to be effective and easy to use in their daily lives since it makes it possible
to read, write and type any signed language by hand or by computer. This led to SignWriting being widely accepted
by different deaf communities worldwide and making its way into education (Flood, 2002). Nowadays, more than
fifteen countries (including France, Italy, USA, Portugal, Canada, Brazil, Germany, Tunisia, Jordan) have their own
SW dictionary which is used to aid literacy. Figure 2 shows an example of SignWriting notation.
Figure 2. The SignWriting notation of the sign “house” in Tunisian Sign Language
One of the most important features that make Sutton system very promising compared with other existing notation
systems, is that it can express by itself any signed sequence, without further annotations in a written vocal language
(Borgia et al., 2012). Signed words written in SignWriting are created by compounding symbols of a highly pictorial
design for hand shapes, palm orientation, body parts, facial expressions, contacts, and finger movements (Figure 2).
However, despite this featural script closely visually resembles the concrete signs, a training to learn to interpret its
static transcriptions is needed for novice users who are accustomed to the use of their sign language in a visual-
gestural modality. The bi-dimensional representation of such notations may inadvertently create confusion and
ambiguity to them since the four-dimensional nature of signing (three-dimensions of space and one of time) cannot
be fully reflected into a symbolic transcription. To offer a closed signing for such notation and make its content
completely accessible to deaf readers, an avatar based system, called tuniSigner (Bouzid & Jemni, 2013), was used to
synthesize sign language animations from transcribed gestures. A 3D human character, as shown in Figure 3,
displays and interprets the SignWriting notation content in natural and comprehensible movements.
Figure 3. The interpretation of the sign “house” via tuniSigner
tuniSigner system includes essentially three main stages to render sign language animated sequences (Figure 4). At
the first stage, the system starts by parsing the SWML file which is an XML-based encoding of SW notations. All
information associated with the input SWML file is processed in order to identify the significant features which
compose the internal structure of the sign. At the second stage, the system provides an explicit linguistic description
incorporating all phonetic details needed to describe the avatar motion in the form of postural and transitional
segments. At the third stage, the system ensures the conversion of the obtained sign description to SML (Sign
Modeling Language), a skeletal representation of sign language gestures which has been developed especially for
WebSign application (Jemni & Elghoul, 2007). SML can describe any signed utterance in terms of translation or
Euler rotation of a group of joints, such as the neck, wrist, hand fingers, eyebrows, eyelids, jaw and so on. Such
script is interpreted then automatically by an animation engine to generate the corresponding animations (Bouzid &
Jemni, 2014a; Bouzid & Jemni, 2014b).
Figure 4. An overview of tuniSigner system’s architecture
MemoSign game
MemoSign is an educational game designed to foster and promote the vocabulary acquisition of any written
language, both spoken and signed. More specifically, MemoSign offers an additional support to learn the sign
language notation system, SignWriting, by rendering its notations content in visual-gestural modality through a 3D
signing avatar. MemoSign exists in two versions dedicated to learning the vocabularies of English-American Sign
Language and Arabic-Tunisian Sign Language. These two instances are available online at the following links: (a), (b) With this
game, not only we are promoting the knowledge for deaf and hearing impaired learners, but we are also encouraging
other hearing people to learn sign language and its written form SignWriting in order to become able to better
understand this community. Indeed, the fact the game would be available online meant that it could also be used by
other users interested in learning signed languages.
MemoSign is an adaptation of LMMG, it excludes the type of cards that hold voice contents since the learners to
whom this game was essentially designed are learners with hearing impairments, and defines three pair of cards
which are:
Word - SignWriting: The first card holds written information, whilst the second card holds the translation of the
word in SignWriting (SW). In this case, the learner must know the meaning of the word. After that, he/she has to
seek for the correspondence of the card in SignWriting, and then select these two cards if they are similar.
Visual - SignWriting: The first card holds visual content, whilst the second card holds SW information. In this
case, the learner must find the relationship between the visual and SW content. Particularly, he/she has to see the
visual card and read the SW information on the second card, and then select these two cards if they are similar.
Calculates - SignWriting: The first card holds simple math problem content, whilst the second card holds SW
content. In this case, the learner must find the relationship between the result of the calculation and the result
translated on SW. Exactly, he/she has to complete the calculation on the first card and memorize the result. After
that, he/she has to look for the card that holds the result of the calculation translated in SW notation.
When the player flips a card that holds a SignWriting notation, a 3D virtual signer starts the interpretation of its
notation content in visual-gestural modality. The incorporation of a virtual avatar in the MemoSign game could be
particularly beneficial for two main reasons. In the one hand, a signing avatar could offer a support for players to
understand and grasp the SignWriting notation content, and this by displaying and interpreting the transcribed
gestures in natural and comprehensible movements. In the other hand, rendering an animated human like character
could certainly increase and promote the engagement, fun and motivation of disabled learners. As mentioned by
(Peterson, 2005), avatars allow the user to take on a visible persona within a virtual world, affording them the
opportunity to engage in surreal and imaginary experiences that transcend the actual world in which they live. Hence,
avatar technology can give the DHH learners the opportunity to master content in a way that meets their needs and to
practice skills until they experience success. Figure 5 presents a screenshot of MemoSign game, it shows that a
learner has selected randomly two cards: the first one holds visual information whilst the second card holds
SignWriting notation with a 3D avatar that interprets the transcribed gestures in natural movements.
Figure 5. Screenshot of MemoSign game
The pilot study: Methodology
We seek, through this pilot study to examine the deaf learners’ interest in using the educational game MemoSign for
learning SignWriting notations and new vocabularies. In particular, we want to know whether the learners enjoy the
game, think it might be useful and learn new vocabularies.
The evaluation was performed in the Tunisian Association for deaf people (ATAS) over three sessions on separate
days. ATAS which is located in Ezzahra, a region in northern Tunisia, offers specialist educational services for
children with hearing impairments basing on the use of SignWriting system. The sample consisted of 9 deaf learners,
7 boys and 2 girls, aged 9 to 16 years old. The demographic characteristics of these participants are shown in Table 1.
These characteristics include age, gender, degree of hearing loss and first language.
Table 1. Demographic data of the participants
Demographic characteristics
Percentage (%)
Degree of hearing
First language
The experimental procedure consisted of three parts: a preparatory phase, a learning phase, and an analysis phase.
Phase 1: Training session
In the preparation phase, two experts provided a short training session on how playing the game to participants.
Given the unique needs of these children, the training session included the intervention of a teacher and a sign
language interpreter who play the role of intermediaries between experts and deaf participants for translating spoken
language to sign language and vice versa. Besides, the experts have coordinated with the teacher about the
vocabularies that will be presented in the MemoSign game.
Phase 2: Participants play the game
The experts arranged three learning sessions with the association ATAS. During each session, each participant was
invited to play the game, while the experts observed their reaction and behavior throughout the process. The types of
cards used in the game are (Visual, SignWriting) and (Word, SignWriting).
Phase 3: Collecting and analyzing data
The third phase consists of collecting and analyzing qualitative data from the experiment. The methods chosen for
that purpose were observations and interviews.
Indeed, the playing and training sessions were video-recorded. Two video cameras were used; the first one is a facial
camera with screen recorder that records the participants’ facial reactions in parallel of his/her interactions with the
game, while the second one was a side camera that records all participants’ actions during the game-play. Figure 6
shows the participants while playing the MemoSign game. At the end of study, the video recordings will pass to an
expert to analyze and note observations using an observation sheet. The observation sheet is composed of 3 sections:
difficulty of using the game, engagement and fun, behavior during the game. Every section had a free space in which
the expert could take notes about the participants while playing MemoSign.
On the other side, each participant was requested to answer questions in the form of an interview after finishing
playing. The interview is monitored by the experts with the help of the teacher and her assistant.
Figure 6. The deaf participants while playing MemoSign
The data for this study were gathered by means of a questionnaire. The questionnaire included four major sections
derived from two validated questionnaires which are: The technology acceptance model questionnaire (Davis, 1989)
and USE (Usefulness, Satisfaction and Ease) questionnaire (Lund, 2001). The first and the second sections of the
questionnaire were related respectively to the Usefulness (U) and Ease of Use (EOU) of the learning game. While the
third and fourth sections examine the users’ Satisfaction (S) and Ease of Learning (EOL). The questions that belong
to U, EOU, EOL, S, are cited in the Table 2. All sections are based on a 3 point Lickert scale, ranging from 1
(“agree) to 3 (disagree).
Table 2. The questions that belong to U, EOU, EOL and S
Questionnaire items
It is useful
It meets my needs
It helps me to concentrate on my study
Ease of use
It is simple to use
I can use it successfully every time
It is user friendly
Ease of learning
It helps me to learn vocabulary easily
I easily remember what I have learnt
It helps me to understand the SW notations
I am satisfied with it
I feel I need to have it
It is fun to use it
The answers given by the participants in the questionnaire were treated as descriptive statistics. The obtained results
are presented in Table 3.
Table 3. Descriptive statistics of participants’ answers to the questionnaire
Ease of use
Ease of learning
Table 3 reveals that the participants’ respondents generally hold positive attitudes towards using MemoSign game.
The noticeable finding from this result is related basically to EOU and EOL items. In fact, all participants declared
that the level of difficulty of the game was low and that game rules were easy to follow. The simplicity of the
educational game encourages and motivates them to use it every time. Likewise, all participants claimed that playing
MemoSign game was generally fun and could help them learn and retain new vocabularies in both spoken and signed
languages more easily. However, although the respondents’ views are generally positive, participants were not totally
satisfied. We can see that only 77.77% of them stated that are satisfied with the game and 22.22% were not satisfied.
More importantly, only 88.88% of participants think about the usefulness and the effectiveness of the game in
supporting their vocabulary building. Those participants stated that the use of the educational game would make
lessons more interesting than traditional teaching method, because it keeps them concentrated and engaged in the
learning process. However, the rest of participants are doubtful whether the game can enhance their vocabulary
learning. In general, this group of participants includes those aged 14 years old and over.
Based on data gathered from the video recordings and experts observations, additional analyses have been
performed. For example, the duration of the training session, the duration of the playing sessions and the duration of
the interview were analyzed using descriptive statistics and summarized in Table 4.
Table 4. Descriptive statics of the duration of the training session, the duration of the playing sessions and the
duration of the interview
Duration Number of
participants Minimum
(minute) Maximum
(minute) Mean
(minute) Std. deviation
Training session
Playing session (Day 1)
Playing session (Day 2)
Playing session (Day 3)
Table 4 shows that the mean of the training session is greater than those obtained in the three playing sessions, this
could be explained by the fact that DHH learners have adequate time to assimilate the game rules and then apply
them. The second observation derived from the Table 4 is that the means of the playing durations decreased from a
session to another. For example, the mean of the playing duration on training session is 5.7 minutes whilst the mean
of the playing duration on session 3 is 4.64 minutes. These findings indicate that participants become skillful on
game playing due to its simplicity.
The analysis of the participants’ reactions during the game-play (filmed by two video cameras) which was done by
the expert is presented in Table 5.
Table 5. Result of evaluation of each participant
Difficulty of using the game
Engagement and fun
Behavior during the game
Participant 1 x x
He tries to mimic the avatar
movements during the game-play
Participant 2 x x
He smiles when making a correct
matching and nerves when making
false trials
Participant 3 x x
She is so exciting when playing the
Participant 4
Very engaged
Participant 5
Participant 6
Very engaged
Participant 7 x x
He tries to emulate the avatar
movements during the game-play
Participant 8 x x
Any noticeable remark, he is very
Participant 9 x x
He is so exciting when playing the
As shown in Table 5, the expert noted that all participants, whatever their degree of hearing loss, handled the game
interface without difficulties. This proves that MemoSign is easy to use and it suits all students with a hearing loss.
Besides, the expert observed that the majority of participants tries to mimic the avatar movements during the game-
play, and they succeeded to do this correctly. This proves that the incorporation of the signing avatar in the game
helped them to understand correctly the transcribed notations, and thus facilitate the learning process. Moreover, the
expert noticed that the participants’ behavior during the game was very good. He found that the majority of
participants was having fun while playing the game.
Discussion and conclusion
Given their specific disability, learners with hearing impairments were often denied from using computer game for
learning purposes like their hearing peers. There have been, even timidly, few attempts that focusing on developing
educational games geared towards this community. The ultimate aim of the current research is to examine the deaf
learners’ interest in using MemoSign, an educational game for learning SignWriting notations and new vocabularies.
The results of the present pilot study revealed that the game: (a) is easy, entertaining and pleasant; (b) it is well liked
and accepted by students as a more satisfying and pleasurable teaching method than the traditional lessons; (c) it
offers an innovative approach to learning SignWriting notations; and thus can constitute a useful tool for teaching the
vocabularies of signed and spoken language.
Participants’ comments on the learning game during the interview were very encouraging. Most participants wish
that they have more opportunities to use the MemoSign game as a learning tool during the session of learning
vocabulary. They also wish that the learning game will be available on the social network Facebook. These findings
provide evidence that students had strong motivation to use this game to learn. More importantly, participants found
that the use of this educational game is more satisfying than traditional classroom lessons, since it can raise their
stimulus and desire to acquire new vocabularies in an enjoyable way. For example, some participants admitted they
gained a lot from the repetition of selecting words in the game. Regarding the opinion of the teacher and her assistant
about the game, it was quite positive, as they considered it to be friendly and easy-to-use. Indeed, this result implies
that MemoSign game could offer a possibility to complement traditional instructional strategies for reinforcing the
students’ lessons.
The results of the expert’s analysis of video recordings support the ones obtained in the interview. During the game,
participants were very engaged and fully involved. Particularly, they showed a great interest and enthusiasm for
simulating the avatar interpretation. Clearly, this finding proves the usefulness of avatar technology in supporting the
learning process for these learners and making the learning process easier. Hence, regarding the research question
(What is the students’ opinion regarding the usability of the MemoSign game?), it was concluded that DHH learners
were quite positive toward using the new educational game.
It is important to stress that MemoSign game exists in two versions dedicated to learning the vocabularies of
English-American Sign Language and Arabic-Tunisian Sign Language. A mobile version of this game exists too in
order to complement the existing Browser Game. Indeed, providing a learning version of the game on mobile devices
for deaf gamers offers unique opportunities to deliver learning content in authentic learning situations (De Jong et al.,
The present study also has some limitations. It was designed to be preliminary and the sample is too small to offer
useful data. In order to further generalize and strengthen the validity of the results, we aim to conduct more
experiments and classroom field studies. Further, we intend to set up an interactive game generator to allow teachers,
desiring to develop computer-based educational materials which target DHH learners, to personalize the different
features of the game and this by specifying the target words, SW notations, images and SL animations.
The authors would like to thank the Tunisian Association to help Deaf people (ATAS) for supporting this work,
especially Mrs Fouzia Mekni and Miss Wafa Laajili.
Adamo-Villani, N., & Hayward, K. (2010). Signing Avatars. In S. Hai-Jew (Eds.), Virtual Immersive and 3D Learning Spaces:
Emerging Technologies and Trends (pp. 249-267). Hershey, PA: IGI Global Publishing.
Borgia, F., Bianchini, C. S., Dalle, P., & De Marsico, M. (2012). Resource production of written forms of Sign Languages by a
user-centered editor, SWift (SignWriting improved fast transcriber). In Proceedings of the Language Resources Evaluation
Conference (LREC) 2012 (pp. 3779-3784). Istanbul, Turkey: European Language Resources Association.
Bourgonjon, J., Valcke, M., Soetaert, R., de Wever, B., & Schellens, T. (2011). Parental acceptance of digital game-based
learning. Computers & Education, 57(1), 14341444.
Bouzid, Y., & Jemni, M. (2013). An Avatar based approach for automatically interpreting a sign language notation. In
Proceedings of the 13th IEEE International Conference on Advanced Learning Technologies (ICALT) (pp. 92-94).
Bouzid, Y., & Jemni, M. (2014a). tuniSigner: A Virtual Interpreter to learn SignWriting. In Proceedings of the 14th IEEE
International Conference on Advanced Learning Technologies (ICALT) (pp. 601-605). doi:10.1109/ICALT.2014.176
Bouzid, Y., & Jemni, M. (2014b). A Virtual signer to interpret SignWriting. In K. Miesenberger et al. (Eds.), Proceedings of the
14th International Conference on Computer Helping People with Special needs (ICCHP) (pp. 458-465). doi:10.1007/978-3-319-
Barlett, C. P., Anderson, C. A., & Swing, E. L. (2009). Video game effects confirmed, suspected, and speculative: A Review of
the evidence. Simulation & Gaming, 40(3), 377-403.
Chamberlain, C., & Mayberry, R. I. (2000). Theorizing about the relationship between American Sign Language and reading. In
C. Chamberlain, J. Morford, & R. I. Mayberry (Eds.), Language Acquisition by Eye (pp. 221-260). Mahwah, NJ: Lawrence
Erlbaum and Associates.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly,
De Jong, T., Specht, M., & Koper, R. (2010). A Study of contextualised mobile information delivery for language learning.
Educational Technology & Society, 13(3), 110125.
Deuchar, S., & Nodder, C. (2003). The Impact of avatars and 3D virtual world creation on learning. In S. Mann & A. Williamson
(Eds.), Proceedings of the 16th Annual Conference of the National Advisory Committee on Computing Qualifications (NACCQ)
(pp. 255-258). Hamilton, New Zealand: NACCQ.
Egenfeldt-Nielsen, S. (2007). Third generation educational use of computer games. Educational Multimedia and Hypermedia,
16(3), 263-281.
Escudeiro, P., Escudeiro, N., Reis, R., Barbosa, M., Bidarra, J., Baltasar, A. B., Rodrigues, P., Lopes, J., & Norberto, M. (2014).
Virtual sign game-Learning sign language. In A. Zaharim, K. Sopian, & V. Vodoyozov (Eds.), Proceedings of the 5th
International Conference on Education and Educational Technologies (ICEET) (pp. 29-33). Kuala Lumpur, Malaysia: WSEAS
Flood, C. M. (2002). How do deaf and hard of hearing students experience learning to write using SignWriting, a way to read and
write signs? Retrieved from
Gee, J. P. (2006). Are video games good for learning? Nordic Journal of Digital Literacy, 1(3), 172182.
Geers, A. E., & Hayes, H. (2011). Reading, writing, and phonological processing skills of adolescents with 10 or more years of
cochlear implant experience. Ear and Hearing, 32(1), 49S59S.
Goldin-Meadow, S., & Mayberry, R. I. (2001). How do profoundly deaf children learn to read? Learning Disabilities Research
and Practice (Special issues: Emergent and early literacy: Current status and research directions), 16(4), 221-228.
Guimarães, C., Guardezi, F., & Fermandes, S. (2014). Sign language writing acquisition Technology for a writing system. In
47th Hawaii International Conference on System Sciences (HICSS) (pp. 120-129). doi: 10.1109/HICSS.2014.23
Griffiths, M. (2002). The Educational benefits of videogaming. Education and Health, 20(3), 47-51.
Haddad, W., & Jurich, S. (2002). ICT for education: Potential and potency. In W. Haddad, & A. Drexler (Eds.), Technologies for
Education: Potentials, Parameters, and Prospects (pp. 28-40). Washington, DC: UNESCO and Academy for Educational
Hameed, A. (2007, October). Information and communication technologies as a new learning tool for the deaf. Paper presented at
the World Congress on Engineering and Computer Science 2007, San Francisco, CA.
Harris, M., & Marschark, M. (2011). Literacy in the classroom and beyond. Journal of Deaf Studies and Deaf Education, 16(1), 1.
Henderson, V., Lee, S., Brashear H., Hamilton H., Starner T., & Hamilton S. (2005). Development of an American sign language
game for deaf children. In Proceedings of the 2005 Conference on Interaction Design and Children (pp. 70-79). New York, NY:
Jabballah, K., & Jemni, M. (2013). A Review on 3D signing avatars: Benefits, uses and challenges. International Journal of
Multimedia Data Engineering and Management (IJMDEM), 4(1), 21-45.
Jemni, M., & Elghoul, O. (2007). An Avatar based approach for automatic interpretation of text to sign language. In Proceedings
of the 9th European Conference for the Advancement of the Assistive Technologies in Europe (pp. 78-81). San Sebastian, Spain:
Kato, M. (2008). A Study of notation and sign writing systems for the deaf. Intercultural Communication Studies, 17(4), 97-114.
Kipp, M., Heloir, A., & Nguyen, Q. (2011). Sign language avatars: Animation and comprehensibility. In H. H. Vilhjálmsson et al.
(Eds.), Intelligent Virtual Agents (pp. 113-126). doi:10.1007/978-3-642-23974-8_13
Khenissi, M. A., Essalmi, F., & Jemni, M. (2014). A Learning version of memory match game. In Proceedings of the 14th IEEE
International Conference on Advanced Learning Technologies (ICALT) (pp. 209-210). doi:10.1109/ICALT.2014.67
Korte, J., Potter, L.E., & Nielsen, S. (2012). Designing a mobile video game to help young deaf children learn Auslan. In
Proceedings of the 26th Annual BCS Interaction Specialist Group Conference on People and Computers (BCS-HCI) (pp. 345-
350). Swinton, UK: British Computer Society.
Lund, A. M. (2001). Measuring usability with the USE questionnaire. Usability interface, 8(2), 3-6.
Mahmood, K., & Ferneley, E. (2006). The Use of animated agents in e-learning environments: An Exploratory, interpretive case
study. Research in Learning Technology, 14(2), 153-168.
McClarty, K. L., Orr, A., Frey, P. M., Dolan, R. P., Vassileva, V., & McVay, A. (2012). A Literature review of gaming in education
(Research report). Iowa City, IA: Pearson.
Mitchell, A., & Savill-Smith, C. (2004). The Use of computer and video games for learning: A Review of the literature. London,
UK: Learning and Skills Development Agency.
Nussbaum, D., Waddy-Smith, B., & Doyle, J. (2012). Students who are deaf and hard of hearing and use sign language:
Considerations and strategies for developing spoken language and literacy skills. Seminars in Speech and Language, 33(4), 310-
Papastergiou, M. (2009). Digital game-based learning in high school computer science education: Impact on educational
effectiveness and student motivation. Computers & Education, 52(1), 1-12.
Passey, D., Rogers, C., Machell, J., & McHugh, G. (2004). The Motivational effect of ICT on pupils. Retrieved from
Peterson, M. (2005). Learning interaction in an avatar-based virtual environment: A Preliminary study. PacCALL Journal, 1(1).
Prot, S., McDonald, K. A., Anderson, C. A., & Gentile, D. A. (2012). Video games: Good, bad or other? Peadiatric Clinics of
North America, 59(3), 647-658.
Silveira, I. F, De Araújo Jr., C. F., Da Veiga, J. S., Bezerra, L. N. M., & Kasperavicius, L. C. C. (2011). Building computer games
as effective learning tools for digital nativesand similars. Issues in Informing Science & Information Technology, 8, 77-91.
Sutton, V. (2015). History of SignWriting. Retrieved from
Vendrame, M., Cutica, I., & Bucciarelli, M. (2013, July). A Written version of sign language can enhance signing deaf
individuals’ comprehension and learning from texts. Paper presented at the 35th Annual Cognitive Science Conference (CogSci),
Berlin, Germany.
Vesel, J. (2005). Signing science. Learning & leading with technology, 32(8), 30-31.
Zamfirov, M., & Saeva, S. (2013). Computer enhanced English language tool for students with hearing loss A Bulgarian
study. Journal of Educational Technology & Society, 16(3), 259-273.
Zwick, U., & Paterson, M. S. (1993). The Memory game. Theoretical Computer Science, 110, 169-196.
... Much research has been conducted on the usage of physical educational games in science education [27,39,8]. [23] developed an educational card game, then measured its impact on students' performance. They found that there is a significant difference between students' test results before and after playing this card game. ...
... Selain digunakan sebagai sarana hiburan, game komputer telah diadopsi untuk waktu yang lama sebagai alat yang bernilai untuk belajar. Dalam papernya, Bouzid menyatakan, Permainan komputer dapat menawarkan banyak manfaat belajar bagi siswa karena game dapat mengkonsumsi perhatian siswa dan meningkatkan motivasi dan keterlibatan mereka yang kemudian dapat mendorong pembelajaran [13]. Dalam papernya, [14] menyatakan bahwa lingkungan mengambil efek dan meningkatkan potensi siswa belajar, terutama lingkungan berbasis permainan. ...
Full-text available
Sekolah Luar Biasa (SLB) Yayasan Asuh Anak-Anak Tuna(YAAT) Surakarta is a school that focuses on educating children with deaf impairment and mentally retarded disabilities in Surakarta. Less interest in learning and the lack of alternative learning media are problems that arise in the school, one of which is regarding the learning of hijaiyah letters. The purpose of this research is to provide a solution for schools to deal with existing problems by developing educational games to increase interest in learning for deaf and mentally retarded students and to become an alternative learning medium for teachers to students. This game was designed using Construct 2 software and for its manufacture using several steps, such as Analysis, Application Design, Application Implementation, and Testing. This game is made as easy as possible so that students can easily use it. The built application consists of several menus, namely materials, playing and quizzes. After the application has been demonstrated, the game is played by students alternately, then students and teachers are given a questionnaire to test the performance and usability of the game. The results of the blackbox test showed that the application was in accordance with what was expected by the application developer, and the results of the User Acceptance Test obtained an average value of 91.908%, indicating that this game makes it easier for teachers to convey material and provides convenience for students in understanding learning material.
... Beberapa penelitian lain menunjukkan game edukasi telah dikembangkan serta efektif untuk pembelajaran bagi siswa dengan beberapa keterbatasan disabilitas, seperti tunagrahita [21]- [23] dan tuna rungu wicara [24], [25]. Hal ini menunjukkan bahwa pembelajaran bahasa inggris untuk siswa tuna rungu wicara juga dapat dilakukan dengan menggunakan game edukasi. ...
Full-text available
Sekolah Luar Biasa (SLB) B (“B” untuk Kategori Tuna Rungu dan Wicara) Yakut Purwokerto adalah sebuah sekolah untuk siswa tuna rungu wicara yang terletak di daerah Kranji, Banyumas, Jawa Tengah. Berdasarkan observasi dan wawancara yang telah dilakukan kepada pihak sekolah, terdapat permasalahan utama di SLB B Yakut Purwokerto, yaitu di bidang pendidikan dan pembelajaran yang menjadi sebuah kesulitan bagi guru-guru di SLB tersebut dikarenakan pembelajaran digunakan untuk anak tuna rungu wicara, tetapi media pembelajaran yang tersedia hanya buku maupun modul standar yang menyulitkan dalam pembelajaran. Solusi yang dapat diberikan kepada SLB B Yakut Purwokerto sekaligus tujuan dari kegiatan pengabdian masyarakat ini adalah membangun dan memberikan media pembelajaran alternatif berupa game edukasi interaktif yang dapat digunakan oleh siswa tuna rungu wicara di SLB tersebut. Metode yang dilakukan dalam kegiatan ini secara umum terbagi menjadi beberapa tahapan, yaitu : focus discussion group dengan pihak sekolah, pembuatan game edukasi, pelatihan game edukasi kepada guru, implementasi game edukasi kepada siswa tuna rungu wicara, serta monitoring dan evaluasi keberlanjutan. Hasil dari kegiatan pengabdian ini adalah game yang dapat digunakan oleh guru dalam mengajar siswa SLB B Yakut Purwokerto sebagai media alternatif dalam belajar. Berdasarkan hasil evaluasi menggunakan kuesioner, didapatkan nilai akhir 86,25% dengan indicator kategori “Sangat Baik”, yang menunjukkan bahwa game edukasi ini dapat digunakan untuk proses belajar mengajar di SLB B Yakut Purwokerto.
Adding game elements to higher education is an increasingly common practice. As a result, many recent empirical studies focus on studying the effectiveness of gamified or game-based educational experiences. The findings of these studies are very diverse, showing both positive and negative effects, and thus calling for comparative meta-studies. In this paper we review and analyze different studies, aiming to summarise and evaluate controlled experiments conducted within different scientific disciplines. We focus on the clarity of non-experimental conditions’ descriptions and show that in most cases (a) educational methods used in control groups’ activities are poorly described, (b) educational materials used in control groups’ activities are often unclear, and (c) the starting conditions are unclear. We also noticed that studies in the fields of computer science and engineering, in general, report results more clearly than in other fields. Based on the above finding, we conclude with a few recommendations for the execution of future empirical studies of games in education for the sake of allowing a more structured comparison.KeywordGamificationGame-based learningLiterature-review
Conference Paper
Full-text available
Sign language is the most commonly used communication medium for people who are deaf and mute. By using body movements, they can convey meaning through visual cues, not vocal cues. This type of language is learned almost exdusively by people who are deaf or dumb and those who interact with them on a daily basis. Most people do not learn it because sign language is not part of basic education, and the lack of need to use it, especially from a young age. Most of the spoken languages in the world usually have sign language variants according to the language of their respective countries. In this study, a mobile application is developed for children to learn Indonesian sign language where the process model is designed using use case diagrams, and the database design is modeled with class diagrams. As for the implementation, the Android Studio software is used for the application, and the MySQLdatabase is for database storage.
The purpose of this research was to determine the opinions of hearing-impaired students about the digital teaching material prepared for subtraction, which requires decimal breaking. To achieve this goal, a digital teaching material that includes decimal breaking operations was developed. The case study method, a qualitative research method, was used to determine the opinions about the developed material. Four secondary school students, two primary school students, one secondary school mathematics teacher, and one primary school mathematics teacher participated in the study during the 2020-2021 academic years. Documents, interview forms, system records, observations, and video recordings were used as data collection tools to contribute to the research. The data obtained from the data collection tools were analyzed using the techniques commonly used in qualitative research, including content analysis and descriptive statistics. The results of the research showed that the developed digital teaching material has a positive impact on students' learning, the absence of distracting elements on the material reduces students' distraction, and students showed interest and success in using the material. The teachers' opinions supported the students' opinions and once again demonstrated the effectiveness of the digital teaching material in the education of hearing-impaired students.
Full-text available
This study presents a qualitative approach to exploring classroom behaviour using dramaturgical analysis of student interactions in relation with, and as mediated through, a gesture-based gaming software among L2 learners of English at two international branch campuses in the Arabian Gulf where face-to-face interactions between unrelated members of the opposite sex are generally discouraged. We investigated whether Avatar Kinect might provide a safe way for young males and females to interact while discussing social issues in a composition course. Data were collected through personal observation and survey. Five key themes emerged from the study. First, some participants chose to perform at front stage and others chose to remain back stage. Second, front stage participants chose avatars with gender and skin colour similar to themselves. Third, all participants appeared to be engaged in the interactive role play processes and with one another. Fourth, front stage actors appeared to act without inhibition. Finally, all participants expressed frustration with technology shortcomings.
Full-text available
Este estudo realiza um levantamento sobre pesquisas que abordam jogos voltados para o ensino de línguas de sinais ou de línguas orais escritas para surdos com objetivo de mapear e caracterizar os jogos que têm sido utilizados em contextos educacionais. Realizou-se uma revisão sistemática da literatura nas bases Education Resources Information Center (ERIC), Web of Science e Scopus, utilizando como termos de busca as palavras: game, deaf e sign language. Na primeira busca, foram encontrados um total de 530 artigos. A seleção com base nos critérios de inclusão e exclusão resultou na análise de 31 trabalhos. A maioria dos jogos utilizados nos trabalhos são digitais, e um terço deles faz uso de algum dispositivo eletrônico para reconhecimento de gestos ou sensores. Os jogos para ensino de língua de sinais ou bilíngues representam a maioria dos estudos selecionados, porém observamos que boa parte ensina apenas o alfabeto manual ou sinais isolados das línguas de sinais. Poucos jogos fizeram o uso das línguas de sinais em situações que simulem um contexto real de comunicação.
Full-text available
There is increasing interest in the use of animated agents in e-learning environments. However, empirical investigations of their use in online education are limited. Our aim is to provide an empirically based framework for the development and evaluation of animated agents in e-learning environments. Findings suggest a number of challenges, including the multiple dialogue models that animated agents will need to accommodate, the diverse range of roles that pedagogical animated agents can usefully support, the dichotomous relationship that emerges between these roles and that of the lecturer, and student perception of the degree of autonomy that can be afforded to animated agents.
Conference Paper
Full-text available
In the absence of a standardized writing system to transcribe their native sign language, deaf signers cannot communicate between each other in their own language except face-to-face. They can’t leave messages, read books take class notes and send email in sign language. Certainly, being able to read and write their own language would bring to these signers the same advantage that writing systems for spoken languages bring to speakers. SignWriting system seems at present the most appropriate method that could meet the deaf needs than other existing notations, as it was intended as an everyday tool for reading and writing. However, such script requires a training to learn to interpret the proposed transcriptions. In this paper, we present an avatar-based system named, tuniSigner, able to display and interpret automatically sign language transcriptions, in the well known SignWriting system. Showing how the actual gestures should be performed in virtual reality would be very useful to signers.
This paper illustrates the results of teaching English to students with hearing loss using a computer program developed for learners defined as either hard of hearing or deaf. The research takes place at one of the three specialized schools in Bulgaria – “Prof. Dr. Decho Denev” Special Secondary School with Kindergarten for Children with Hearing Loss, located in Sofia City. The software was utilized for one academic year by students ranging from sixteen to twenty-three years of age. The program includes two video screens; one for lip-reading English phrases and one for interpreting them into Bulgarian Sign language with sentences classified into topics and pictures representing each phrase.
The chapter focuses on signing avatars and their potential to improve deaf education. In sections 1 and 2, the authors give an overview of what signing avatars are and the benefits of using animated characters for deaf education. In section 3, they explain how signing avatars are created. In particular, in subsection 3.1, they describe different types of 3D models and skeletal deformation systems, and in subsection 3.2 the authors discuss a variety of methods used to animate manual and non-manual signs. In section 4 they report the state of the art in signing avatars' research and development and we discuss existing limitations and future trends. Section 5 includes a case study on the production of the signing avatars for SMILE™ and Mathsigner™ Conclusive remarks are presented in section 6.
Conference Paper
Learning vocabulary is a very important part of learning a new language. It is vitally important for beginners and children to learn the keywords needed to understand what they hear and read, and to express their own ideas. Despite this importance, vocabulary acquisition stills the hardest part of the language and faces the problem of low motivation of learners. One of the effective ways of raising motivation and increase learners' desire to learn is to use learning games. This paper describes a learning version of memory match game specifically designed for second language acquisition. The objective of this learning game is to render the learning experience with more fun, to engage learners and to keep them motivated.