ArticlePDF Available

Abstract and Figures

- The goal of this article is to provide insight into the design, development and evaluation of a game-based language assessment tool for tablets. The overall objective was to create a mobile application for practitioners in kindergarten, enabling these practitioners to gather relevant language data from children aged three to five. An automated analysis of the data provides information about the child’s language skills. If a need for support in language development can be found in the data, the application offers further possibilities how stakeholders can support a children’s learning process for the specific field of language. The main criteria for a successful development were the effective collection of relevant data, high usability for practitioners and appropriate usage of a game-based mobile application for young children.
Content may be subject to copyright.
PaperMobile Game-Based Language Assessment
Mobile Game-Based Language Assessment
Jan Delcker (*)
University of Mannheim, Mannheim, Germany
Dirk Ifenthaler
University of Mannheim, Mannheim, Germany
Curtin University, Perth, Australia
AbstractThe goal of this article is to provide insight into the design, devel-
opment and evaluation of a game-based language assessment tool for tablets. The
overall objective was to create a mobile application for practitioners in kinder-
garten, enabling these practitioners to gather relevant language data from children
aged three to five. An automated analysis of the data provides information about
the child’s language skills. If a need for support in language development can be
found in the data, the application offers further possibilities how stakeholders can
support a children’s learning process for the specific field of language. The main
criteria for a successful development were the effective collection of relevant
data, high usability for practitioners and appropriate usage of a game-based mo-
bile application for young children.
Keywords—Mobile tool, game-based assessment, language development, eye-
tracking, tablet
1 Introduction
Approximately one third of children enrolling in primary schools in Germany have
an migration background [1]. The presence of a migration background is often linked
to a language level which is less developed than the language level of same-aged chil-
dren of German origin. When children grow up with one language at home (e.g., Turk-
ish or Russian) and they get in contact with a second language in kindergarten, they are
thought to be less efficient in the second language they are learning. Research shows
that a correlation between language skills and the language spoken first cannot be made
that easily. “The quality, frequency and type of language used in the home” [2] are main
factors for the different development of language skills. If a child has a lot of experience
and skills in his or her primary language, the child is more likely to develop higher
skills in a second language [3]. Other important factors for the development of language
skills are maternal characteristics [4] and the language used in the pre-schools and kin-
dergartens the children are attending [5].
One of the main goals of the research project was the development of a spoken lan-
guage level assessment tool for pre-elementary school children, aged three to five. The
iJET Vol. 15, No. 3, 2020
PaperMobile Game-Based Language Assessment
instrument should be able to collect reliable language data from the target group. From
reliable data researchers can produce consistent, bias-free findings if the data is inter-
preted based on enough evidence and theoretical support. [6]. Regarding the developed
instrument, statements about the language skills of an assessed child can be made. On
the individual level, the tool can be used to identify educational needs of a child [7].
As a second goal, practitioners and parents are being provided with facilities, training
methods and concepts to foster the child’s further language development, based on the
results of the assessment. In this way, participating families and institution can directly
benefit from the use of the instrument [8]. The instrument is therefore designed to be
more than a mere tool to gather data, hence, creating value for all stakeholders.
On a systemic level, the tool can be used to find specific differences regarding mi-
gration background, gender or age. Reaching this third goal enables stakeholders to
develop systematic approaches to support groups of children with mutual development
To reach the above mentioned goals, the instrument has to fulfill a number of specific
requirements [9]. The collection of relevant data has to be parsimonious and the tool
has to be easy to use for practitioners. Relevant assessment data in the context of spoken
language assessment are statements made by the assessed user of a test instrument. To
be able to assess those statements in the most efficient way, they should be recorded.
The advantage of a recording over a live test is the fact that the data can be assessed by
more than one time or by more than one person [10]. The quality of recorded speech
data and language data in general is highly depended on the situation it is recorded in.
Evidence based interpretations of language skills are relying on data that is representing
real-life speech situations. The data has to be collected in situations which are as close
to real-life situations as possible, with the least artificial distortions or disturbance pos-
sible. The avoidance of interferences to the speech situation is of special importance to
the target group. Because the data is collected from young children, the natural speech
situation is especially susceptible to derangement.
The ease of use or usability of the tool is important to increase both the amount of
data gathered and the quality of the data. If the tool is easy to use, practitioners will use
it more often and therefor gather more data. Additionally, a tool that is easy to use
allows practitioners on concentrating on other important aspects, for example creating
a stimulating text surrounding. A proper test setup results in a higher quality of gathered
Another important factor to reach the intended goals is the creation of an instrument
that is engaging and interesting for children. A test tool that is fun to use increases the
chances to create a natural speaking situation and support the gathering of reliable and
valid data. The creation of a child-friendly, fun instrument helps to gain attention for
the instrument and might reduce the need to advertise the instrument to stakeholders
such as daycare centers, kindergartens and pre-schools. As a result, the amount of data
that is available for analysis will increase.
PaperMobile Game-Based Language Assessment
2 Design
The assessment of language skills, especially the ones of bilingual children, is a com-
plex task [11]. One of the main issues is “the lack of (culturally) appropriate instru-
ments” [12] that allow for the effective collection of reliable data. Given the goals,
requirements and the target group presented in the preceding paragraph, the design of
the instrument is based on reflections regarding a technology and game-based approach.
A combination of those two aspects covers the main challenges of the research project,
which made a team of professionals from the fields of linguistics, technology and edu-
cation necessary for conducting the project [13].
Modern mobile technology like smartphones or tablets can be used as low priced
recording tools that offer the recording quality necessary to create the language data for
evaluation [14]. Besides the high functionality as recording devices, an internet con-
nection makes it easy to collect the data, helping to prepare it for further analysis.
Additionally, an instrument that uses mobile technology is independent of a labora-
tory setting to gather data. This fact helps to work towards the goal of a test situation
placed in a natural speech environment.
From a research-economic and research-administration perspective, designing a dig-
ital instrument rather than a paper-based requires a higher investment at the start of the
project, for example the acquisition of equipment such as smartphones and tablets or
financing the coding of a digital instrument. In the long-run, those expenses can be
compensated, because the distribution of a digital instrument is very cost-efficient. In
addition, changing the instrument in the future does not require the printing and distri-
bution of new questionnaires or guidelines. While outdated versions of paper-based
instruments have to be thrown away, digital instruments can be brought to the newest
version by simply installing the updated version on any device in use.
While the technology based approach supports the goal to collect reliable spoken
language data of high quality, the game-based approach to the instrument helps to create
situations, in which this data is generated [15]. The target group of the research project,
children in kindergarten and pre-school aged 3-5, spends a considerable amount of their
time playing. Playing games and playful interactions with the environment are consid-
ered as of “vital importance for the healthy development of children” [16]. Because
playing is a natural action for children and part of their daily activities, the statements
made while playing can be considered as natural speech behavior or natural language
Playing games as a resource of natural language and using technology to record lan-
guage can be combined in form of a video game. Being “interactive, simulated systems
that are rule based, responsive, challenging, cumulative and inviting” [17], games offer
a wide range of possibilities to create an assessment instrument for the language skills
of children [18]. Initiating situations in which language is used to let the player interact
with the game (and vice versa) captures the interactive and responsive characteristics
of a game. Because the creator of the game is creating a simulated version of a situation,
the creator has full control of the situation, how a story is developing or which reactions
are caused by a specific action of the player. By designing a challenging and inviting
game, the concentration of a player can be focused on specific tasks as well as the skills
iJET Vol. 15, No. 3, 2020
PaperMobile Game-Based Language Assessment
needed to succeed. As an assessment tool, a game offers the opportunity to measure and
analyze those skills.
The instrument designed and developed for this project comprises of two applica-
tions working in conjunction with each other, combining the aspects of a mobile, game-
based instrument. The first application, called Wuschel App, is running on an iPad. Its
main function is the collection of speech data from a participating child by playing a
game. The second application is running on an iPhone or an iPod. The application is
used by a practitioner to navigate through the game. The application is called Practi-
tioners App. Although the instrument is presented in the form of a game, its main pur-
pose is the assessment of a child’s language skills and not entertainment. In the next
chapter, both applications and how they interact with each other is explained more spe-
2.1 Game concept and graphics
The game is designed as an adventure game. It is set in a magical forest. The main
characters are Rita the Witch, Wuschel the Dog, Willy the Gnome and a mean dragon,
archenemy of Rita. Throughout the game it is the child’s task to protect Rita from the
Dragon and to help her get specific objects. Rita is characterized as clumsy and she
tends to lose important wizard items, for example her wand or her broom. The dog
Wuschel is presented as her closest friend. With the help of the child, Wuschel is able
to find the lost objects for Rita. Willy the Gnome is pranking Rita and Willy in a playful
way, hiding objects from them. The graphics and the style of the game can be described
as comic-like (see Figure 1). The drawing style of the characters and the setting is very
child-friendly. By using bright and colorful patterns, the atmosphere is cheerful and
stimulating. The game is constructed as a series of scenes, comparable to the pages of
a children’s book. In each scene, the child is presented with a specific task. After the
situation of a sequence gets resolved, the next sequence can be started.
Fig. 1. Wuschel App
PaperMobile Game-Based Language Assessment
2.2 Tasks and controls
In most cases, the objective of a specific scene is to find an object on the screen, for
example the witch’s wand. The tasks are presented by Wuschel by asking for the ob-
jects: “It looks like Rita lost her wand. Can you tell me where the wand is please?” The
tasks are resolved when the player gives the right answer, for example: “The wand is
on the kitchen table”. If the right answer is given, Wuschel answers by saying in the
sense of “Oh, I see it, it is over there!”, walking to the objective and taking it. If the
players answer is incorrect or if it is unclear, Wuschel restates the initial question: “I’m
not quite sure what you mean, can you repeat that please? Where can I find the wand?”.
One of the key aspects of the game is the fact that it is not actually speech-controlled.
The scenes are pre-programmed and Wuschel will always find the objects at the end of
the scene.
Hence, the children are not playing the game alone but are being supervised by a
practitioner sitting in the background who is controlling the game with the Practi-
tioner’s App (see Figure 2). With the application a scene can be started, Wuschel’s
questions can be initialized and a scene can be finished. In this way, an adult keeps full
control of the game, adapting the speed to the skills of the child playing the game.
Because all the scenes are pre-scripted and Wuschel always finds the objectives or so-
lutions, the flow of the story is not getting disrupted, which is preventing frustration
and demotivation of the players.
Fig. 2. Practitioner's App
iJET Vol. 15, No. 3, 2020
PaperMobile Game-Based Language Assessment
In the current state of the game, two types of tasks are implemented. The type is the
localization of an object, for example the witch’s wand as described in the preceded
paragraph. The second type is the characterization of movement in form and direction.
For the second type of task the player has to either describe a movement that was pre-
sented on the screen (e.g., “Willy fell from the tree!) or formulate a movement as a
command for Wuschel (e.g., “You have to jump from stone to stone!”). The two types
of tasks have been chosen because they create language data for prepositions. For lin-
guist, those prepositions are of great significance, especially against the backdrop of
bilingual children. Bilingual children might tend to mix prepositions from different lan-
2.3 Data collection
In preparation for the assessment, a legal guardian of the tested child has to complete
an informed consent form, after being informed of the process of the assessment as
well. Additionally, the legal guardians are asked to provide demographic data of their
child (for example the age of the child and how long it has been speaking German).
Each of the questionnaires and the informed consent forms contain a unique QR-Code
and an 18-figure digital key. With the help of this key, consent form and personal data
can be linked after the anonymization the data. This allows for the deletion of data if
the legal guardians or the child decide to drop out of the research process at a later time
(compliance with EU GDPR).
With the QR-Code, the personal data can be linked to the records of the Wuschel
App. While the child is playing the game, the statements of the child are being recorded
on the iPad. Each statement is saved as a separate file. After the completion of the game,
the records are saved on a server, as soon as the iPad gets connected to a wireless net-
work. The records are automatically linked to the provided biographic data of the as-
sessed child via the digital key. In this way, the statements of a child can be matched
with the biographical data without having to use the real name of the child. The safety
and protection of the children’s biographical and speech data is of upmost importance
throughout the project.
2.4 Data preprocessing and analysis
After the recordings are saved on the server, instructed linguists transliterate the spo-
ken language data into written form. The transcribed data can automatically be analyzed
with linguist processing software that has been developed especially for this project. In
the future, suggestions for treatment options or teaching programs will be provided by
the application if practitioners want to receive them. In addition, automated transcrip-
tion of speech to text is currently in development.
PaperMobile Game-Based Language Assessment
3 Usability of the Instrument
Usability is defined as the “extent to which a system, product or service can be used
by specified users to achieve specified goals with effectiveness, efficiency and satisfac-
tion in a specified context of use” (ISO 9241-11:2018). It is a “quality attribute that
assesses how easy user interfaces can be used” [19]. In the case of the developed in-
strument, the concept of usability has to be applied to a range of different aspects. (I)
From a technological perspective, the instrument has to be working as intended. Tech-
nological functionality consists of the ability to easily install the software on the neces-
sary hardware (iPad, iPod, iPhone) and to run it on those devices without crashes, in-
dependent of the version of the operating system. The usability is high, if the connection
between the Wuschel App and the Practitioners App is working as intended and the
data is successfully stored on the servers. (II) The operability of the Practitioners App
is another part of the usability of the instrument. Practitioners have to be able to control
the instrument from the application. The design of the application has to be clear and
simple. All buttons have to be self-explaining. This way, practitioners can control the
instrument independent of their ICT skills. The ease of use of the Practitioners App was
of special interest for the design and development of the instrument. The instrument
relies on practitioners being able to use the application to gather relevant data. (III) To
ensure a high usability of the instrument, the Wuschel App has to be engaging and
playful for the children. The development of a natural language situation is supported
by the creation of an interesting story and appealing characters. Statements made in
those situations are considered of more value by linguists, because they are closer to
real-life language situations. Additionally, the instrument should be fun to use for all
The technical functionality of the instrument was tested before the other parts of the
usability were reviewed. The instrument was run on the largest possible number of dif-
ferent devices and devices combinations to ensure flawless interaction between the
Practitioners and the Wuschel App. The correct storing of language data on the servers
was tested in the same way.
The usability of the Wuschel App was analyzed in great detail through multiple sur-
veys. Initially, an eye-tracking survey was planned and conducted with children in a
kindergarten. In a second step, students took part in an eye-tracking survey and a think-
aloud study.
3.1 Increasing usability through game design
During the design of the game, the priority was on the assessment focus. To ensure
the gathering of relevant language data, the child has to have the impression of actually
controlling the game with his or her voice. Most children, even at a young age are fa-
miliar with the touch sensitive displays of smartphones and tablets. One of the biggest
challenges was to convince the children to use their voice and not their hands to control
and navigate through the game. This challenge was even bigger, because the game is
not controlled by the child at all but by a practitioner. Several design steps were imple-
mented to ensure the child would have the impression it would control the game with
iJET Vol. 15, No. 3, 2020
PaperMobile Game-Based Language Assessment
his or her voice. In the first scenes of the game, in which the characters are introduced,
the dog Wuschel shows the playing child how the interaction between the player and
the dog is working. The witch Rita is giving instructions to the dog with her voice and
Wuschel is following those instructions. Wuschel is underlining how he is reacting to
Rita’s spoken commands: “I am a good dog; I am always doing what I am told to do.”
As a second step, Wuschel is talking directly to child: “You should try it to. Tell me to
dance and I will dance!”. This is the first event in which the dog is directly responding
to a command of the child. To further deepen the child’s understanding of the control
it is having over the dog, the player has to help the dog to get from one side of a small
river to the other by jumping over from one stone to the other. At this event, Wuschel
is clarifying that he can only hear, but not see the child and that the child has to give
vocal commands. Wuschel is not reacting to gestures or interactions with the touch
Apart from the desired behavior of the player to use voice commands to control the
game, the assessed children were assumed to provide better data, if they would have
fun playing the game. This requires the creation of challenging, yet solvable tasks.
Willy the Gnome is used at different parts of the game to help the child understand what
the solution to a problem is or to point out the location of an item.
3.2 Usability testing of the wuschel app
A game on a mobile device can only be interesting and engaging, if the player can
see and understand everything that is happening on the screen. Hence, two eye-tracking
surveys were conducted to ensure a clear visualization of the delivered content of the
Wuschel App. Eye-tracking helps to understand at which part of a screen a person is
looking at a specific moment. Although focusing on a part of the screen and under-
standing what is happening on the screen is not the same thing, research suggests that
eye-tracking data can be used to make a connection between watching and compre-
hending [20, 21]. While conducting an eye-tracking survey, a special camera, the so-
called eye-tracker, is recording the eye movement of a user. With the help of specialized
software, the eye-movements of the user are synchronized to the events on a screen.
After the data collection, researchers are able to investigate on which parts of the screen
the user was looking at. In the next step, statements about the visibility of objects and
events on the screen can be made. The visibility of objects and events on the screen is
vital to the usability of the instrument. A child can only make statements about objects
and events in the game if it is able to see and recognize them. The eye-tracking studies
were therefor used to identify critical events in the game which would have to be im-
proved to increase the usability of the instrument.
The first eye-tracking study was conducted with five children in a kindergarten. The
test environment was very close to the environment in which the instrument will be
used in the future. A child is playing the game while a practitioner is controlling the
game from the distance. During the study, a researcher took the role of a practitioner.
A pre-school teacher was also present to make the child more comfortable. Soon after
the eye-tracking study started, problems with the eye-tracking hardware became obvi-
ous. The tablets on which the Wuschel App was running had to be mounted on a special
PaperMobile Game-Based Language Assessment
mobile device stand so the eye-tracker could make reliable recordings. This setup,
which is normally used for mature participants of eye-tracking studies proved to be
inapt for the young participants. The angle and the distance between the eye-tracker and
the participant, the two most factors for reliable eye-tracking measurement, couldn’t be
held to the extent necessary. As a result, the collection of reliable eye-tracking data
wasn’t possible with children.
As an addition to the eye-tracking study at the kindergarten, a second eye-tracking
study was performed with 20 adult students between 22 and 30 years. Although students
are not the original target group of the instrument, eye-tracking could still be used to
identify critical events in the Wuschel App. In addition, a think aloud study was con-
ducted to gather information about the users impression of the Wuschel App. This in-
formation was used to identify weak points of the application which weren’t of visual
The analysis of the eye-tracking data consists of an alignment between eye-tracking
data and important objects. If the user is asked for information about a certain object on
the screen, the focus of the user should be on that object shortly after the question is
asked. For example, if the user is asked to find the witch’s hat, the focal point should
be on the area of the screen, where the hat is shown. If the focal point is not on the part
of the screen where the hat is located, the user either can’t see the hat or the hat cannot
be identified as such.
During both the study with the children and the adult students all stationary objects
and most moving objects were identified and denoted in the proper way. This suggests
the assumption, that the visualization and the visibility of those objects was sufficient
and would guarantee a high usability of the Wuschel App. To increase the usability
further, the eye-tracking data of the objects which weren’t identified correctly were
observed in more detail. Two types of moving objects proved to cause difficulties for
the user. The first type of objects which were hard to identify were those falling from
the outside of the screen. The second type of objects were those which moved across
from on one side of the screen and disappeared on the other. In both cases, the eye-
tracking data showed the user was looking at the objects when the first part of the move-
ment (e.g., the entrance to the screen) already passed. It can be assumed that a longer
screen time, caused by a slower movement, would increase the chance of the user fo-
cusing on the object. As a result, the movement speed of these objects was decreased
in future versions of the application, further improving the usability of the whole in-
In the think-aloud study, the students described the overall design of the game as
interesting and child friendly. All the instructions given by the dog were described as
understandable, both in volume and presentation. The critical events identified with the
eye-tracking study came also up in the think aloud study. Some students said, the move-
ments were too fast and created a blur that made it hard to see and name the objects.
The length of the story was criticized by some of the participants. One run-through
takes around 30-40minutes, which was considered as to long for small children between
three and five years.
Parallel to the eye-tracking study and the think-aloud study, the instrument was
tested with 25 children in different pre-schools. The test was used to observe the overall
iJET Vol. 15, No. 3, 2020
PaperMobile Game-Based Language Assessment
interaction between the children and the game. The goal was to identify parts of the
game in which children might be afraid or bored. Additionally, tasks which are to dif-
ficult should be found.
Overall, the children played the game with great interest. They showed excitement,
moving close to the screen, paying close attention to the events, as well as expressing a
range of emotions from laughter and joy to tension, but no fear or boredom. Although
the run-through can be considered as long, no child decided to stop playing and every-
one completed the game. The different tests conducted to survey the Wuschel App sug-
gest that the usability of the application is very high. Weak points, such as the move-
ment speed of objects, could be identified and improved.
4 Discussion and Future Research Aspects
At the current state of the research process, the declared research goals have only
been reached partially. The design and development of a mobile, game-based instru-
ment, that is capable of collecting reliable data can be considered as completed. The
language data is reliably collected and of high quality. From a technological perspec-
tive, both applications are running without bugs and crashes. On the server-side, the
language data is automatically stored and connected to the anonymized demographic
data of the assessed children. Children’s joy in playing the game and interacting with
the characters on screen could be observed in the participating children.
Two main aspects will be in the focus of the future research process. In a first step,
the automated transcription of the spoken language data has to be refined. Right now,
training a transcription software to handle the data is one of the possible solutions for
this task. Only after a reliable and efficient method of turning the language into an an-
alyzable format is found, the linguist experts of the research group will be able to study
and interpret the data using quantitative and qualitative methods. Furthermore, addi-
tional tasks will be added to the Wuschel App. On a systemic level, the influence of
demographic variables on the development of spoken language skills can be explored.
With those findings, programs can be developed to foster the development of second
language children.
The second step of future research will be the provision of trainings, exercises and
reading material once a certain need for support can be identified for a child. Using the
existing system, feedback can anonymously be given to legal guardians and/or practi-
tioners in the institutions.
Right now, one of the most critical issues is the distribution of the instrument to
institutions for children aged three to five throughout Germany. Only if this distribution
can be assured, the database used for future analysis will be filled with sufficient data.
As of now, the applications are not publicly available. More important, most institutions
do not possess the tools (i.e., iPad, iPhone, iPad) to use the applications. One idea to
overcome those limitations is the creation of a mobile language lab. This lab consists
of a suitcase which equipped with the necessary tools to conduct the assessment: an
iPad/iPod which carry the language assessment instrument and the paper-based demo-
graphic questionnaire as well as information material and the informed consent form.
PaperMobile Game-Based Language Assessment
Institutions and practitioners will be able to order the suitcase and conduct the assess-
ment after the shipping has arrived.
Once the limitations and future research goals have been tackled, the instrument of-
fers an innovative way of reliably collecting spoken language data from children aged
3-5. Due to the implemented option of getting feedback about the language develop-
ment of an assessed child, the instrument offers practical use for all stakeholders. In-
stead of being an instrument to collect data it can function as a tool to support children,
legal guardians and practitioners to help understand the language development of chil-
dren and furthermore foster their language learning progress.
5 Acknowledgement
The authors acknowledge the financial support by the Daimler and Benz Foundation.
6 References
[1] S. Bundesamt. (2018, 7.1.). Number of persons with migration background in Germany in
2017, by age (in 1,000). Available:
[2] B. A. Collins, "Dual language development of Latino children: Effect of instructional pro-
gram type and the home and school language environment," Early Childhood Research
Quarterly, vol. 29, pp. 389-397, 2014/07/01/ 2014.
[3] R. L. Sparks, J. Patton, L. Ganschow, N. Humbach, and J. Javorsky, "Early first-language
reading and spelling skills predict later second-language reading and spelling skills," Journal
of Educational Psychology, vol. 100, pp. 162-174, 2008.
[4] C. S. Hammer, E. Komaroff, B. L. Rodriguez, L. M. Lopez, S. E. Scarpino, and B. Goldstein,
"Predicting SpanishEnglish Bilingual Children’s Language Abilities," Journal of Speech,
Language, and Hearing Research, vol. 55, pp. 1251-1264, 2012.
[5] M. M. Páez, P. O. Tabors, and L. M. López, "Dual language and literacy development of
Spanish-speaking preschool children," Journal of Applied Developmental Psychology, vol.
28, pp. 85-102, 2007/03/01/ 2007.
[6] R. Akbari, "Validity in Language Testing," in The Cambridge Guide to Second Language
Assessment, C. Coombe, P. Davidson, B. O'Sullivan, and S. Stoynoff, Eds., ed: Cambridge
University Press, 2012.
[7] A. K. Moeller, J. W. Creswell, and N. Saville, Second language assessment and mixed meth-
ods research. Cambridge: Cambridge University Press, 2016.
[8] A. Drigas, G. Kokkalia, and A. Economou, "Mobile Learning for Preschool Education,"
International Journal of Interactive Mobile Technologies (iJIM), vol. 10, p. 57, 2016.
[9] L. F. Bachman, Statistical analyses for language assessment, 1. publ. ed. Cambridge [u.a.]:
Cambridge Univ. Press, 2004.
[10] B. O'Sullivan, "Assessing Speaking," in The Cambridge Guide to Second Language Assess-
ment, C. Coombe, P. Davidson, B. O'Sullivan, and S. Stoynoff, Eds., ed: Cambridge Uni-
versity Press, 2012.
iJET Vol. 15, No. 3, 2020
PaperMobile Game-Based Language Assessment
[11] A. Green, Exploring Language Assessment and Testing: Language in Action: Taylor &
Francis, 2013.
[12] T. Boerma and E. Blom, "Assessment of bilingual children: What if testing both languages
is not possible?" Journal of Communication Disorders, vol. 66, pp. 65-76, 2017/03/01/ 2017.
[13] M. Shuib, A. Abdullah, S. N. Azizan, and T. Gunasegaran, "Designing an Intelligent Mobile
Learning Tool for Grammar Learning (i-MoL)," International Journal of Interactive Mobile
Technologies (iJIM), vol. 9, p. 41, 2015.
[14] J. Delcker, A. Honal, and D. Ifenthaler, "Mobile Device Usage in Higher Education," in
Digital Technologies: Sustainable Innovations for Improving Teaching and Learning, D. G.
Sampson, D. Ifenthaler, J. M. Spector, and P. IsaÌas, Eds., ed, 2018, pp. 45-56. https://
[15] C. S. Loh, Y. Sheng, and D. Ifenthaler, "Serious games analytics: Theoretical framework,"
in Serious games analytics. methodologies for performance measurement, assessment, and
improvement, C. S. Loh, Y. Sheng, and D. Ifenthaler, Eds., ed New York, NY: Springer,
2015, pp. 3-29.
[16] D. G. Burac, "The Playful Behavior in Swimming and Its Interferences in 1-3 Years Child's
Development," Procedia - Social and Behavioral Sciences, vol. 180, pp. 1229-1234,
2015/05/05/ 2015.
[17] R. E. Mayer, Computer Games for Learning: An Evidence-Based Approach: MIT Press,
[18] X. Ge and D. Ifenthaler, "Designing engaging educational games and assessing engagement
in game-based learning," in Handbook of research on serious games for educational appli-
cations R. Zheng and K. Garnder, Eds., ed Hershey, PA: IGI Global, 2017, pp. 255 - 272.
[19] A. Dirin and M. Nieminen, "mLUX: Usability and User Experience Development Frame-
work for M-Learning," International Journal of Interactive Mobile Technologies (iJIM), vol.
9, p. 37, 2015.
[20] K. Conklin, Eye-tracking: a guide for applied linguistics research. Cambridge, New York,
NY: Cambridge University Press, 2018.
[21] A. T. Duchowski, Eye tracking methodology: theory and practice, 2. ed. ed. London:
Springer, 2007.
7 Authors
Jan Delcker is a German research assistant at the Chair of Learning, Design and
Technology at the Area of Economic and Business Education of the University of
Mannheim, L4, 1, in Mannheim, Germany. He graduated from the University of Ap-
plied Science Mannheim with a master's degree in social work, focusing on media ed-
ucation and quantitative social research. His current research interests are mobile learn-
ing and digitalistion in school development.
Dirk Ifenthaler is Professor and Chair of Learning, Design and Technology at Uni-
versity of Mannheim, Germany and UNESCO Deputy Chair of Data Science in Higher
Education Learning and Teaching at Curtin University, Australia. His research focuses
on the intersection of cognitive psychology, educational technology, data analytics, and
organisational learning.
Article submitted 2019-09-13. Resubmitted 2019-10-09. Final acceptance 2019-10-10. Final version pub-
lished as submitted by the authors.
... Other applications of gamification for learning include interactive programming for teaching JavaScript programming [23] and a conceptional Engagement Framework for gamification learning platform [24], Game for design, development, and evaluation of language assessment tools [25], API-based backend system of the game client Discover Indonesia [26], a mobile educational game to learn Javanese vocabulary [27], a mobile guessing game as training for children with autism [28], a mobilebased Balinese Script Educational Game [29], an Androidbased sign language educational game [30], New Normal COVID-19 educational game [31], a rare animal recognition game with Construct 2 [32], research on Bingo games for elderly users [33], two-dimensional top-down puzzle adventure game [34], an educational game about Indonesia [35], a game for collecting experimental data obtained from youths who have joined criminal groups around Los Angeles city [36], and games as learning English vocabulary medium in college [37]. ...
... Because teaching and learning in schools underwent an accelerated digital transformation, and technology became a crucial component of teaching and learning processes, during and after the pandemic (Cheng et al., 2022), teachers found themselves playing a major role in promoting innovative and meaningful learning in online learning during Covid-19 crisis (Bano glu and G€ um€ uş, 2022;Delcker and Ifenthaler, 2020). ...
Purpose This research examined correlations between contextual factors: frequency of online teaching (OT) (number of hours per week), Transformational Leadership Style (TLS) and TPACK (TPACK) among Arab and Jewish teachers in Israel after more than a year of teaching online during the Covid-19 crisis. Design/methodology/approach Quantitative methodology elicited data from 437 questionnaires. An online questionnaire was first sent to Israeli Arab and Jewish teachers studying for M.A degrees in three randomly selected higher education institutes in Israel, and then the questionnaire was sent to other teachers, selected through the snowball method. After data collection common method variance was precluded. Findings The findings of this research indicated a positive correlation between frequency of OT, TLS and TPACK among Israeli teachers. Major differences were found between Arab and Jewish teachers: Arab teachers (AT) reported more improvement of TPACK, although they taught fewer hours than Jewish teachers (JT). In addition, TLS and sector (Arab/Jewish) moderated the positive correlation between the frequency of OT and TPACK among ATs. Originality/value The findings of this research validate the opportunity created by the Covid-19 crisis for cultivation of teachers’ TPACK through OT. This research contributes to extant relevant literature and practice concerning the influence of contextual factors on teachers' improvement of their TPACK while performing OT during the Covid-19 crisis and can inform the design of ecological and culturally appropriate education policies in the post-COVID-19 period. The research was built on the theories of TLS, which is a crucial component supporting the influence of technology integration. The findings strengthen existing knowledge on the unique capacity of TLS to buffer negative external influences imposed on teachers' TPACK and motivate them.
... In their work, Ciman et al. [94] designed a game to support children with cerebral visual impairment, developing a mobile version of the game to be used by children easily at home on any platform. Delcker & Ifenthaler [95] also developed a mobile app that makes an automated analysis of the data and provides information about children's language skills. Other papers focused on teachers, such as [96], where the authors used a GBA to develop a set of visualizations to support teachers in classrooms. ...
Full-text available
Technology has become an essential part of our everyday life, and its use in educational environments keeps growing. In addition, games are one of the most popular activities across cultures and ages, and there is ample evidence that supports the benefits of using games for assessment. This field is commonly known as game-based assessment (GBA), which refers to the use of games to assess learners' competencies, skills, or knowledge. This paper analyzes the current status of the GBA field by performing the first systematic literature review on empirical GBA studies. It is based on 65 research papers that used digital GBAs to determine: (1) the context where the study has been applied; (2) the primary purpose; (3) the domain of the game used; (4) game/tool availability; (5) the size of the data sample; (6) the computational methods and algorithms applied; (7) the targeted stakeholders of the study; and (8) what limitations and challenges are reported by authors. Based on the categories established and our analysis, the findings suggest that GBAs are mainly used in K-16 education and for assessment purposes, and that most GBAs focus on assessing STEM content, and cognitive and soft skills. Furthermore, the current limitations indicate that future GBA research would benefit from the use of bigger data samples and more specialized algorithms. Based on our results, we discuss current trends in the field and open challenges (including replication and validation problems), providing recommendations for the future research agenda of the GBA field.
... In their work, Ciman et al. [85] designed a game to support children with cerebral visual impairment, developing a mobile version of the game to be used by children easily at home on any platform. Delcker & Ifenthaler [86] also developed a mobile app that makes an automated analysis of the data and provides information about children's language skills. Other works are focused on teachers, such as [87], where authors used a GBA to develop a set of visualizations to support teachers in classrooms. ...
Full-text available
Technology has become an essential part of our everyday life, and its use in educational environments keeps growing. In addition, games are one of the most popular activities across cultures and ages, and there is ample evidence that supports the benefits of using games for assessment. This field is commonly known as game-based assessment (GBA), which refers to the use of games to assess learners' competencies, skills, or knowledge. This paper analyzes the current status of the GBA field by performing the first systematic literature review on empirical GBA studies, based on 66 research papers that used digital GBAs to determine: (1) the context where the study has been applied, (2) the primary purpose, (3) the knowledge domain of the game used, (4) game/tool availability, (5) the size of the data sample, (6) the data science techniques and algorithms applied, (7) the targeted stakeholders of the study, and (8) what limitations and challenges are reported by authors. Based on the categories established and our analysis, the findings suggest that GBAs are mainly used in formal education and for assessment purposes, and most GBAs focus on assessing STEM content and cognitive skills. Furthermore, the current limitations indicate that future GBA research would benefit from the use of bigger data samples and more specialized algorithms. Based on our results, we discuss the status of the field with the current trends and the open challenges (including replication and validation problems) providing recommendations for the future research agenda of the GBA field.
... The term gamification refers to using game mechanics and characteristics in nongame contexts to facilitate individuals to reach their goals [3]. Under the fundamental premise that games are enjoyable [18], much research has been implemented to induce the students to engage in the learning process [19] and increase the learning outcomes [20]. ...
Full-text available
This study explored students' perceptions about using gamified e-quizzes and conventional online quizzes for their class engagement. The participants were 130 female university students in Seoul with various majors. As a quasi-experimental study, this study compared the attitudes of a gamified e-quiz group (n=92) and a conventional online quiz group (n=38) after experiencing their respective quiz in-tervention of either nine gamified e-quizzes or nine online quizzes over a 15-week semester. Each group responded to a survey at the end of the semester. The quan-titative analyses of the surveys indicated that the perception of the two groups did not display statistically significant differences, each displaying positive views to-ward their quiz interventions for emotional, behavioral, and cognitive engage-ment. In addition, the two groups demonstrated neutral to positive attitudes for each quiz intervention for agentic engagement. Among many reported benefits, the students in both groups expressed that the quiz experiences facilitated them to understand the content knowledge and enjoy the assessment activities. These two advantages of the two quiz modes were seen to be related to close student-teacher interaction.
Educational game performance data has the potential to allow new types of complex, procedural skills to be assessed. However, prior work has shown that gameplay data do not readily align to existing assessment validation paradigms, and game performance scores are difficult to use for proficiency testing. A new assessment paradigm that can cope with the nature of gameplay data has not emerged. In this paper, we uncovered a range of structural issues in data collection caused by, and potentially solved by, the engineered environments in games. Choice and the iterative nature of games were found to allow curriculum specialisation. We found evidence that early attempts at new games are less reliable and perhaps best discarded, and we propose a solution to weight scores to reflect novelty in repeated tasks. We found capturing the effect of competitor or collaborator ability on performance challenging but propose the potential for bots to resolve this. Finally, we also investigated the use of response time as a proxy for ability. The physical measure of time proved difficult and potentially unfair to use, but we propose a possible stochastic treatment of speed that could allow scoring some skills in some games using response time.
Full-text available
Nowadays preschool education is considered crucial for a child’s development. Recent researches support the view that the role of kindergarten in children’s progress is very important, as this age is sensitive for their future academic and social life. The use of mobile learning is recognized as a tool that can foster the knowledge and the experiences for this age and the support of specific areas according to the educational perspective. In this paper we try to give a brief overview of the most representative studies of the last decade (2005-2015), which focus on the skills that are explored in kindergarten and are supported by the mobile applications. The effectiveness of mobile learning in special preschool education is also explored.
Full-text available
In this paper, we propose the mLUX framework, a model based on the user-centered design (UCD) framework, which is specifically for the development of m-learning applications. We present the results of four case studies conducted to develop m-learning applications in which the proposed mLUX framework was applied. The main goal of the proposed mLUX framework is to ensure that the stakeholders, especially students, recognize that m-learning applications are learning media that fulfill their essential educational requirements. We begin by reviewing the literature on the contributions to mobile learning usability made in various conferences and journals from 2002 to 2010. This review helped identify and recognize the methodology used to develop mobile learning applications during this period. Based on the literature review, as the empirical case studies, four m-learning applications are examined to demonstrate the performance of the proposed framework for the development of m-learning applications. We apply three distinct measurement criteria to assess the performances of the mLUX framework. This paper also argues that emotional factors, such as the user’s enjoyment, adjustability, and reliability, are significant design issues in m-learning.
Conference Paper
Full-text available
During infancy between 1-3 years the game is the predominant activity of the child. Within the playful behavior are embedded interests and forms of expression of the child's body, family, as well as the curiosity about the surroundings. Game behaviors allow a pertinent observation of the level of perception of reality and the relationship between child and educator, as the games scenarios lead to experience acquisition. Swimming through games contributes to the development of a balanced body and can be easily practiced by the children who do not have a very good physical condition and who usually don't like motion.
Eye-tracking is quickly becoming a valuable tool in applied linguistics research as it provides a 'real-time', direct measure of cognitive processing effort. This book provides a straightforward introduction to the technology and how it might be used in language research. With a strong focus on the practicalities of designing eye-tracking studies that achieve the standard of other well-established experimental techniques, it provides valuable information about building and designing studies, touching on common challenges and problems, as well as solutions. Importantly, the book looks at the use of eye-tracking in a wide variety of applied contexts including reading, listening and multi-modal input, writing, testing, corpus linguistics, translation, stylistics, and computer-mediated communication. Each chapter finishes with a simple checklist to help researchers use eye-tracking in a wide variety of language studies. Discussion is grounded in concrete examples, which will allow users coming to the technology for the first time to gain the knowledge and confidence to use it to produce high quality research.
This chapter focuses on mobile device usage of students in higher education. While more and more students embrace mobile devices in their daily life, institutions attempt to profit from those devices for educational purposes. It is, therefore, crucial for institutional development to identify students’ needs and how mobile devices may facilitate these needs. This longitudinal study with N = 172 participants compares the use of e-Readers and tablets for learning at a higher education institution. While e-Readers offer inexpensive solutions for reading texts, tablets provide a much wider range of applications, such as communicating with other students, accessing learning management systems, or conducting research online. Findings indicate that students evaluate tablets as a more useful device for learning. Interestingly, students using tablets also start to include more and more mobile learning technologies into their learning strategies.
Language delays of bilingual children can arise from language impairment (LI) but also from insufficient exposure to the target language. A reliable diagnosis of LI in bilingual children is therefore ideally based on the evaluation of both languages, as LI affects each language that is learned. However, due to the multitude of language combinations that are encountered in clinical practice, this is often not feasible. Bilingual norm-referencing may offer a solution, but the heterogeneity within the bilingual population makes it difficult to determine appropriate standards for every child. The present study examined an alternative approach to assessing both languages or using bilingual norm-referencing, aiming to assemble instruments that reduce bias against bilingual children. We used a four-group design, including monolingual and bilingual children with and without LI (N = 132), to first investigate the effects of LI and bilingualism on risks associated with a child’s early language development and the prevalence of language problems in the family, as reported by parents. Second, we evaluated the diagnostic validity of these two indices, and, in addition, combined these with two unbiased language measures which we previously examined in isolation: a quasi-universal nonword repetition task and a narrative task. Results showed that the index of Early Language Development was a strong predictor of LI. In combination with the two direct language measures, it excellently identified the presence or absence of LI in and across monolingual and bilingual learning contexts. Learning outcomes: As a result of this study, the reader will learn about an alternative approach to testing a bilingual child in both languages. The reader will recognize the importance of using unbiased measures for the identification of LI in a bilingual context, and, in addition, will appreciate the value of combining parental report with direct language measures.
The focus of this chapter is on designing engaging educational games for cognitive, motivational, and emotional benefits. The concept of engagement is defined and its relationship with motivation and cognition are discussed. Design issues with many educational games are examined in terms of factors influencing sustained motivation and engagement. A theoretical framework to design engaging digital games is presented, including three dimensions of engagement (i.e., behavioral, cognitive, and emotional). Later, the chapter considers how to harness the appealing power of engaging games for designing engaging educational games. Various motivational features of game design and learner experiences are considered. In conclusion, the chapter also discusses various methods to assess engagement in order to inform the design of educational games that motivate learners.
Routledge Introductions to Applied Linguistics is a series of introductory level textbooks covering the core topics in Applied Linguistics, primarily designed for those beginning postgraduate studies, or taking an introductory MA course as well as advanced undergraduates. Titles in the series are also ideal for language professionals returning to academic study.