Conference PaperPDF Available

Radical Tunes: Exploring the Impact of Music on Memorization of Stroke Order in Logographic Writing Systems

Authors:

Abstract and Figures

Music and rhythm are powerful tools that can be employed to enhance learning and memory. While games are commonly utilized to aid in second language acquisition, few have explored the implications of sound on learner's ability to draw and remember logographic characters (such as those in Chinese hanzi, Japanese kanji and Korean hanja). We created Radical Tunes, a kanji drawing music game, to explore the impacts of incorporating music on players' ability to retain meaning and stoke order of several kanji. In this paper, we describe the design rationale for Radical Tunes, and present results from a pilot study comparing a music focused version of the game with one that uses non-musical sound effects. Results show that while both conditions improved players' short-term ability to remember/draw kanji, there were no significant differences in improvement between the conditions. However, the use of music did improve immersion-an important factor related to learning. This work has implications for designers of second language acquisition games, and how they can incorporate rhythm and music into their games to increase player engagement.
Content may be subject to copyright.
Radical Tunes
Exploring the Impact of Music on Memorization of Stroke Order in Logographic Writing Systems
Oleksandra Keehl
University of California, Santa Cruz
Santa Cruz, CA
okeehl@ucsc.edu
Edward Melcer
University of California, Santa Cruz
Santa Cruz, CA
emelcer@ucsc.edu
ABSTRACT
Music and rhythm are powerful tools that can be employed to
enhance learning and memory. While games are commonly uti-
lized to aid in second language acquisition, few have explored the
implications of sound on learner’s ability to draw and remember
logographic characters (such as those in Chinese hanzi, Japanese
kanji and Korean hanja). We created Radical Tunes, a kanji draw-
ing music game, to explore the impacts of incorporating music on
players’ ability to retain meaning and stoke order of several kanji.
In this paper, we describe the design rationale for Radical Tunes,
and present results from a pilot study comparing a music focused
version of the game with one that uses non-musical sound eects.
Results show that while both conditions improved players’ short-
term ability to remember/draw kanji, there were no signicant
dierences in improvement between the conditions. However, the
use of music did improve immersion—an important factor related
to learning. This work has implications for designers of second
language acquisition games, and how they can incorporate rhythm
and music into their games to increase player engagement.
KEYWORDS
Educational Game; Music; Immersion; Second-Language Acquisi-
tion; Kanji; Radicals
ACM Reference Format:
Oleksandra Keehl and Edward Melcer. 2019. Radical Tunes: Exploring the
Impact of Music on Memorization of Stroke Order in Logographic Writing
Systems. In The Fourteenth International Conference on the Foundations of
Digital Games (FDG ’19), August 26–30, 2019, San Luis Obispo, CA, USA. ACM,
New York, NY, USA, 6 pages. https://doi.org/10.1145/3337722.3337764
1 INTRODUCTION
Logographic writing systems, such as Chinese, Japanese and Korean,
are notoriously dicult to learn. The students of these languages
have to copy the characters dozens of times in order to memorize
them. As a result, acquiring hundreds of characters through rote
learning is a daunting task and can discourage a casual learner from
pursuing a language.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
FDG ’19, August 26–30, 2019, San Luis Obispo, CA, USA
©2019 Association for Computing Machinery.
ACM ISBN 978-1-4503-7217-6/19/08. . . $15.00
https://doi.org/10.1145/3337722.3337764
Games have become a frequently utilized tool to aid in second-
language acquisition (SLA), addressing topics such as vocabulary
and meaning [
7
,
21
], tones [
11
], conversational phrases [
5
], and cul-
ture [
2
]. However, there has been relatively little focus on how SLA
games can be employed to teach writing skills for logographic char-
acters. For instance, while there are many educational games and
applications out there intended to help people memorize Japanese
kanji, most focus solely on the reading and meaning portions and
don’t teach stroke order (such as the popular service WaniKani
1
).
However, the stroke order is also important to learn, as it is instru-
mental for reading and writing handwritten scripts. Furthermore,
the majority of commercial writing apps primarily amount to digi-
tal ashcards. While convenient and useful to a motivated learner,
they oer little extrinsic motivation to a casual student, which is a
signicant hurdle for many kanji students [18].
To address these issues, we created a musical game for learning to
write kanji. The name of our game, Radical Tunes, is not incidental:
radicals are the building components of kanji (Figure 1), and music
can play an important role in memory and learning. For instance,
in a study by Wallace [
22
], subjects were best able to memorize
lyrics to a song if it was played to them several times with the same
rhythmically matching melody (as opposed to several matching
melodies or repeating but mismatched melody). Our hope is that the
musical aspects of the game will turn the tedious task of repeatedly
drawing the kanji into an enjoyable and educationally-eective
activity. To achieve this we are relying on the power of music as
a mnemonic device [
22
] and popularity of rhythm games, such as
Osu2which has over 13 million registered players.
Beyond its aesthetic appeal and mnemonic potential, music has
also been found to play a notable role in altering immersion [
14
,
20
].
Cairns et al. [
1
] describe immersion as "the degree of involvement
that players have with dierent aspects of the game leading to a
move of the attention, awareness and thoughts of the player from
the real world around them to the events happening within the
game". This suggests immersion is an important component of the
gaming experience, and other research has shown that it can also
enhance learning in a number of ways [
3
,
6
], making it important
to both SLA games and educational games in general.
In this paper, we provide the design rationale for Radical Tunes—
which ties unique melodies to radicals within a kanji. We also
present results from a pilot study exploring whether adding music
to an SLA game would improve learning outcomes and key factors
that can impact learning, such as immersion. To our knowledge,
no prior work has examined the relationship between music and
1https://www.wanikani.com
2https://osu.ppy.sh/home
FDG ’19, August 26–30, 2019, San Luis Obispo, CA, USA Oleksandra Keehl and Edward Melcer
Figure 1: Anatomy of kanji. Each kanji consists of one or
more radicals and each radical has one or more strokes.
ecacy of an SLA game for learning the writing stroke order of
kanji characters.
2 RELATED WORK
The following subsections provide a brief overview of games created
to teach various aspects of a language, and notable prior research
examining the relationship of music, immersion, and learning.
2.1 Second-Language Acquisition Games
Many studies have been conducted on the use of games in SLA,
including exploration of kinesthetic learning [
8
,
23
], augmented
reality [
16
], alternate reality [
4
], collaborative learning [
4
,
9
,
11
,
16
,
21
,
23
], and roleplaying [
2
] in games. These games addressed
various aspects of language acquisition, such as vocabulary and
meaning [
2
,
7
,
15
], pronunciation and tones [
9
,
11
,
17
], grammar
[4], writing [21, 23] and even culture [2, 16].
Notably, many of these studies showed that one of the main
benets from including games in SLA was increased engagement.
Multiple ndings highlighted that the majority of players felt en-
tertained and challenged, which could lead to dedicating more time
to the learning activity. Other benets observed in some of the
games were ease of access (on mobile devices) [
11
] and reduced
stress (from not having to perform in front of teachers/peers) [
4
].
However, to our knowledge, no one has conducted a study on the
eects of music on memorization of kanji or other logographic
characters.
In addition to positive outcomes of games on learning language,
some prior work has found negative eects as well. Specically,
DeHaan et al. has studied eects of cognitive load in interactive
Figure 2: Radicals and kanji used in study. Left to right: per-
son, mouth, tree, rest, episode, annoyed.
games on learning [
7
]. Using a commercial musical vocabulary
learning game Parappa the Rapper 2, they compared vocab retention
from students who played the game and those who watched the
gameplay without any interaction. In their study, the watching
students retained more vocabulary than the players, which lead the
authors to caution that it is possible for interactivity in a game to
aect learning outcomes negatively. We kept this concern in mind
when designing Radical Tunes. In Parappa the Rapper 2, the player
had to hit sequences of buttons in time with music. One could
master this activity without paying any attention to the words. In
Radical Tunes, the music is created as the player draws each stroke,
and thus the game’s interactivity is integral to the learning activity
instead of distracting from it.
2.2 Immersion, Music, and Learning
Music has been linked to various aspects of human performance.
One study suggested that calming music helped increase children’s
performance in math, word memorization and altruistic tendencies
[
10
]. Another study examined eects of music on lyrics memoriza-
tion and found that subjects exposed to a repeating melody with a
rhythm matching the lyrics performed better than those listening
to a mismatched, altering or no melody[22].
Music has also been linked to immersion in digital games [
20
],
and immersion, in turn, has been shown to have a positive eect on
learning [
2
,
3
]. Many of the papers mentioned in the SLA section,
while not explicitly mentioning immersion, listed engagement as a
factor leading to positive learning outcomes. Notably, engagement
is recognized as one of the core components of immersion [
1
,
3
].
Immersion is therefore another important factor to examine in our
study.
3 RADICAL TUNES
Radical Tunes is a musical game for learning to write kanji, the
Japanese characters borrowed from Chinese language. Kanji consist
of radicals and strokes (Figure 1). Radicals are building blocks of
kanji, and each kanji has at least one radical. Each radical has a
meaning, which may or may not contribute to the meaning of the
kanji, and these are often used in mnemonics. WaniKani
3
uses
made-up humorous meanings for radicals, which often have little
to do with the original meaning, but through the shock value have
a strong mnemonic eect. Each radical consists of one or more
strokes. Strokes don’t carry a meaning on their own.
Many mnemonic devices exist for remembering the shape and
meaning of a kanji [
19
], but to our knowledge, none of these are
focused on learning the proper writing stroke order—which is nec-
essary for handwriting prociency. As a result, we created Radical
3https://www.wanikani.com
Radical Tunes FDG ’19, August 26–30, 2019, San Luis Obispo, CA, USA
(a) (b)
(c) (d)
Figure 3: Radical Tunes screenshots. (a) Meaning pre- and post-test. (b) Stroke order pre- and post-test. (c) Stroke learning
screen. (d) Timed and scored game portion.
Tunes to ll that void. Past studies have shown the positive eects
of music on memorization of text, especially when it is rhythmically
aligned with the material [
22
]. We wanted to explore if this music
mnemonic eect could be extended to memorization of the stroke
order of a kanji.
Radical Tunes achieves the intended musical mnemonic mech-
anism by employing a unique instrument and melody for each
radical, thus giving each radical a melodic ngerprint of sorts.
These melodies are further broken down, with each element of
the melody corresponding to a stroke within the radical. As a re-
sult, as the student correctly draws a kanji, they will hear a unique
melody corresponding to that kanji unfolding with every stroke.
For our initial game we selected three of the six most commonly
occurring radicals in the J
¯
oy
¯
o kanji (the 2,136 characters required
for baseline literacy). We also selected three kanji which consisted
of combinations of those radicals, to add increased complexity, for
a total of six characters for our subjects to learn (Figure 2).
Radical Tunes teaches writing stroke order in three stages. First,
an animated demo of the stroke order is shown. Next, a player is
asked to trace the character with the help of arrows (Figure 3c),
and nally, they are encouraged to practice tracing the character
without the arrow order hints. If they draw a stroke incorrectly, a
soft ping alerts them of the mistake and they will have to attempt
the stroke again. After learning is complete, the player can play the
game, where they have to draw the kanji correctly and as quickly
as possible to earn the most points (Figure 3d).
4 METHODS
The goal of our pilot study was to investigate the impact that music
and rhythm would have on learning in Radical Tunes. Given that
immersion can play an important role in learning for educational
games [
3
,
6
] and music has been shown to impact immersion [
20
],
we also wanted to examine if the musical nature of Radical Tunes
increased player immersion.
Our study had two versions of the game: 1) the original Radical
Tunes with unique tunes for each radical and melodic elements ac-
companying each stroke; and 2) the control version where the music
was replaced with the sounds of chalk writing on a blackboard.
4.1 Procedure
The participants were told that the study was for an educational
game where they would learn the meaning and writing of six Japan-
ese characters. They were then randomly assigned to either the
music or control condition and given an Android tablet with the
appropriate version of Radical Tunes loaded. For the experiment,
meaning and stroke order tests were built into the beginning and
end of the game (Figures 3a and 3b).
For the meaning tests, the participants were shown the characters
one at a time and had to chose an answer from 6 possible meanings
FDG ’19, August 26–30, 2019, San Luis Obispo, CA, USA Oleksandra Keehl and Edward Melcer
Table 1: IEQ mean scores, standard deviations, signicant dierences between the two conditions, and the eect size, which is
in the medium to large range for signicant dierences.
Control Music Signicance Eect Size
IEQ Measures µ σ µ σ p r
IEQ Overall 100.33 18.1 115.25 15.29 .04* .41
Cognitive Involvement 30.08 7.01 36.08 5.13 .026* .44
Real World Dissociation 25.58 6.49 28.5 5.81 .26 .23
Emotional Involvement 12.33 3.58 15.58 3.48 .034* .42
Challenge 16.33 1.5 17.67 1.83 .063 .37
Control 16 3.54 17.42 3.15 .31 .21
(Figure 3a). The options were the same for pre- and post-tests. At the
end, the players were notied how many characters they identied
correctly, though not which ones. This was done to prevent the
participants from learning character meanings during the pretest
via conrmation of their guesses.
For the stroke order tests, the participants were asked to trace
each character on the screen with their nger. We placed orange
dots along the lines, and the dots turned green when touched, to
indicate which sections of a kanji were already traced (Figure 3b).
No feedback was given on this evaluation, as we were examining
their knowledge of stroke order without any hints.
After the pre-tests for meaning and stroke order, the participants
were taken through the educational section of Radical Tunes. First,
the subjects were shown the animation of the correct stroke order
with accompanying sounds. Then, they were asked to draw the
strokes by following the arrows on the screen (Figure 3c). Finally,
they were asked to trace the character unassisted. A soft ping
would let them know if they made a mistake, and they would
have to redraw the latest stroke. To ensure that each participant
received the same amount of instruction, we limited each step to
two unskippable repetitions.
Next, participants played the Radical Tunes game by tracing
the characters appearing on the screen as quickly and correctly as
possible (Figure 3d). The game included each character twice in
a randomized order. The same randomized order was used for all
participants.
After the game was completed, the nal score and time were
displayed and the participants were directed to take the meaning
and stroke order post-tests. Lastly, they completed a demographic
and experience questionnaire on a laptop.
4.2 Participants
Twenty four participants took part in the study. They were recruited
through class announcements and by word of mouth outside of
the university. There were 11 female, 12 male and one non-binary
participant. The ages ranged from 19 to 51 with the average age
of 25.6 (SD = 7.8). During the study, participants were randomly
assigned to one of the two conditions: music (7 female, 5 male) and
control (4 female, 1 non-binary, 7 male). All participants reported
no prior knowledge of Japanese or Chinese.
4.3 Measures
4.3.1 Immersive Experience estionnaire (IEQ). We used the Im-
mersive Experience Questionnaire [
13
] to measure the level of
immersion experienced by our subjects. IEQ takes into account
ve dimensions of immersion: cognitive involvement, emotional
involvement, real world dissociation, control and challenge. The
cognitive involvement dimension covers factors like attention and
eort the subjects invested into the game. Emotional involvement
covers the degree to which the subjects enjoyed the game and were
interested in seeing more of it. The real world dissociation mea-
sures the degree to which the participants lost track of time and
their surroundings. The control dimension assesses the perceived
responsiveness and ease of use of controls. The challenge dimension
measures the perceived diculty of the game.
In their paper, Jennett et al. suggested that immersion is an im-
portant component of what is perceived as a good game. For Radical
Tunes to serve its educational purpose, it is important for it to be
able to hold the players’ attention and provide an enjoyable expe-
rience that entices them to return. With this in mind, we decided
that IEQ would help us measure how well Radical Tunes performed
in this sense.
4.3.2 Radical Tunes scores. The participants were given pre- and
post-tests for meaning (Figure 3a) and stroke order (Figure 3b)
for each of the six characters. We used the relative improvement
between the pre/post tests to assess the learning outcome of the
game.
5 RESULTS
5.1 Prior Knowledge and Experience
According to a series of independent samples t-tests, participants in
the two conditions did not dier with respect to prior video game
experience (p = .61), prior rhythm game experience (p = .81), and
on the pre-test learning outcome measures: kanji meaning (p = .82)
and stroke order (p = .31).
We can therefore assume that participants in both groups had
similar prior game experience and knowledge of kanji for the fol-
lowing analyses.
5.2 Immersion Experience
We rst examine participants’ experience of immersion during
gameplay for both conditions (Table 1). To analyze dierences be-
tween IEQ scores for the two conditions we used an independent
Radical Tunes FDG ’19, August 26–30, 2019, San Luis Obispo, CA, USA
Table 2: Descriptive statistics of learning outcomes (mean-
ing tests, stroke order tests, and in-game scores).
Control Music
Learning Outcome Measures µ σ µ σ
Meaning Pre-test 1.08 .9 1.16 .84
Meaning Post-test 5.5 1 5.58 .67
Meaning Improvement +4.42 1.73 +4.42 1.17
Stroke Order Pre-test .33 .65 .59 .52
Stroke Order Post-test 5.41 .9 5.67 1.16
Stroke Order Improvement +5.08 .9 +5.08 1.38
In-Game Score 102.17 8.93 106.58 2.19
samples t-test. Table 1 shows descriptive statistics for IEQ scores, as
well as signicant dierences and eect sizes between conditions.
Results found a signicant dierence in favor of the music condition
of the game increasing overall IEQ scores (p = .04, r = .41), cognitive
involvement (p = .026, r = .44), and emotional involvement (p = .034,
r = .42). There were no signicant dierences for real world dis-
sociation, challenge, or control (all p’s > .063). This suggests that
adding melodies and music to our game helped to improve player
factors (cognitive involvement and emotional involvement) but not
game factors (challenge and control) of immersion [13].
5.3 Learning Outcomes
To better understand players’ learning outcomes from the game we
analyzed pre- and post-test scores for meaning and stroke order,
as well as nal scores from the game portion of Radical Tunes.
Descriptive statistics for the three measures are shown in Table 2. A
series of independent samples t-tests showed that both conditions
had a signicant increase in both meaning and stroke order scores
from pre- to post-test: control (meaning - p < .001, r = .95; stroke
order - p < .001, r = .96), and music (meaning - p < .001, r = .92;
stroke order - p < .001, r = .94). This suggests that the game itself was
indeed successful for short-term teaching of kanji meanings and
writing stroke orders to players. However, there were no signicant
dierences in improvement between conditions for meaning, stroke
order, or in-game score (all p’s > .11).
5.4 Perception of Usefulness
We asked participants to rate if they would recommend this game
to someone trying to learn Japanese or Chinese on a 7-point Likert
scale (1 - not at all to 7 - very likely). Scores were notably higher for
the music condition (µ= 6.58, σ= .67) than the control condition
(
µ
= 5.42,
σ
= 1.78). An independent samples t-test showed that
there was a signicantly higher likelihood of recommending the
game for the music condition (p > .045, r = .4).
6 DISCUSSION
6.1 Immersion and Learning in an SLA Game
The music condition participants showed signicantly more overall
immersion according to IEQ responses (Table 1), particularly in the
cognitive and emotional involvement dimensions. This falls in line
with existing studies showing increased immersion with addition
of pleasing music [
20
]. We expected challenge and control to be
similar across the two groups, since the game controls and tasks
were identical. Overall, this indicates that adding music to Radical
Tunes—and likely SLA games in general—can improve player fac-
tors (cognitive involvement and emotional involvement) but not
game factors (challenge and control) of immersion [13].
Both conditions showed a signicant improvement between pre-
and post-test scores for meaning and stroke order (all p’s < .001)—
indicating that the game is eective for helping players learn. How-
ever, there was no signicant dierence in the improvement be-
tween the two conditions (Table 2). This is possibly due to the
limited number of kanji tested and short-term interaction with
Radical Tunes, not providing enough time and diculty for the
music mnemonic approach to have a signicant impact on learning
outcomes.
6.2 Limitations
While the immersion and perceived usefulness results are promis-
ing, showing that players tend to prefer the music version of Radical
Tunes, we weren’t able to see any eects of music on memorization
of stroke order or kanji meaning. We theorize that these eects may
present themselves better over time with a larger number of kanji.
Furthermore, the number of participants in our study (n= 24) is
lower than n= 30 recommended to reach normal data distributions
for our observed eect size (this rule of thumb is derived from the
Central Limit Theorem [
12
]). This limits the scope of our results. To
remedy these limitations, we plan to conduct a future longitudinal
study with a larger number of participants and with expanded learn-
ing content to better observe the eects of kanji-specic melodies
on long-term stroke order memorization abilities of students.
6.3 Future Design Improvements
Our current implementation of Radical Tunes features melodic
elements of a xed duration in what we estimated would be the
appropriate length for each stroke to be drawn. Each element is
triggered when the player correctly begins to draw a stroke. If the
player goes out of bounds or lifts their nger half way through,
the melody is interrupted, which works as designed. However, if
the speed of the player’s drawing is signicantly dierent from our
estimate, the melody and drawing become out of sync. Several of
the participants in the music condition of Radical Tunes noted this
issue and expressed their wish for a better coupling of melody and
stroke speed. Dynamically adjusting stroke sounds is something
we intend to implement for our next round of experiments. Some
participants were also dissatised with the slower pacing of the
game, which resulted from us enforcing two repetitions of each of
the learning steps. In the next iteration, we plan to allow students
to navigate through sections of the game freely and learn things at
their own pace.
7 CONCLUSIONS
In this paper we described Radical Tunes—a musical game for learn-
ing to write Japanese kanji, which uses unique melodies to aid with
stroke order memorization. We also described the results of our
pilot study, which compared the musical version of Radical Tunes
FDG ’19, August 26–30, 2019, San Luis Obispo, CA, USA Oleksandra Keehl and Edward Melcer
with a version which instead had non-musical sound eects, i.e.,
the sound of chalk writing on a blackboard. Through use of the
Immersive Experience Questionnaire, we found that incorporation
of music in Radical Tunes signicantly increased player immersion.
The participants from both conditions showed signicant score im-
provement between pre- and post-tests, which shows that Radical
Tunes was eective in teaching the kanji to players, at least in the
short-term. The small scope of our pilot study—only six characters—
didn’t expose any mnemonic eects the music may have on players’
ability to retain the kanji long term, and it is something we plan to
investigate further in our future work.
REFERENCES
[1] Paul Cairns, Anna Cox, and A Imran Nordin. 2014. Immersion in digital games:
review of gaming experience research. Handbook of digital games 1 (2014), 767.
[2]
Alan Cheng, Lei Yang, and Erik Andersen. 2017. Teaching language and culture
with a virtual reality game. In Proceedings of the 2017 CHI Conference on Human
Factors in Computing Systems. ACM, 541–549.
[3]
M-T Cheng, H-C She, and Leonard A Annetta. 2015. Game immersion experience:
its hierarchical structure and impact on game-based science learning. Journal of
Computer Assisted Learning 31, 3 (2015), 232–253.
[4]
Thomas M. Connolly, Mark Stanseld, and Thomas Hainey. 2011. An Alternate
Reality Game for Language Learning: ARGuing for Multilingual Motivation.
Comput. Educ. 57, 1 (Aug. 2011), 1389–1415.
[5]
Gabriel Culbertson, Shiyu Wang, Malte Jung, and Erik Andersen. 2016. Social
situational language learning through an online 3D game. In Proceedings of the
2016 CHI Conference on Human Factors in Computing Systems. ACM, 957–968.
[6]
Chris Dede. 2009. Immersive interfaces for engagement and learning. science
323, 5910 (2009), 66–69.
[7]
Jonathan deHaan, W. Michael Reed, and Katsuko Kuwada. 2010. The eect of
interactivity with a music video game on second language vocabulary recall.
Language Learning & Technology 14, 2 (2010), 74–94.
[8]
Darren Edge, Kai-Yin Cheng, and Michael Whitney. 2013. SpatialEase: Learning
Language Through Body Motion. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems (CHI ’13). ACM, New York, NY, USA, 4.
[9]
Xiangmin Fan, Wencan Luo,and Jingtao Wang. 2017. Mastery Learning of Second
Language through Asynchronous Modeling of Native Speakers in a Collaborative
Mobile Game. In Proceedings of the 2017 CHI Conference on Human Factors in
Computing Systems. ACM, 4887–4898.
[10]
Susan Hallam, John Price, and Georgia Katsarou. 2002. The Eects of Background
Music on Primary School Pupils’ Task Performance. Educational Studies 28, 2
(2002), 111–122.
[11]
Andrew Head, Yi Xu, and Jingtao Wang. 2014. Tonewars: Connecting language
learners and native speakers through collaborative mobile games. In International
Conference on Intelligent Tutoring Systems. Springer, 368–377.
[12]
C.C. Heyde. 2006. Central Limit Theorem. In Encyclopedia of Actuarial Science.
American Cancer Society.
[13]
Charlene Jennett, Anna L. Cox, Paul Cairns, Samira Dhoparee, Andrew Epps,
Tim Tijs, and Alison Walton. 2008. Measuring and Dening the Experience of
Immersion in Games. Int. J. Hum.-Comput. Stud. 66, 9 (Sept. 2008), 641–661.
[14]
Kari Kallinen. 2004. The eects of background music on using a pocket computer
in a cafeteria: Immersion, emotional responses, and social richness of medium.
In Extended Abstracts on Human Factors in Computing Systems (CHI ’04). ACM.
[15]
Anuj Kumar, Pooja Reddy, Anuj Tewari, Rajat Agrawal, and Matthew Kam. 2012.
Improving literacy in developing countries using speech recognition-supported
games on mobile devices. In Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems. ACM, 1149–1158.
[16]
Zheng Liu, Holden. 2016. Analyzing students’ Language Learning Experience in
an Augmented Reality Mobile Game: An Exploration of an Emergent Learning
Environment. In Procedia - Social and Behavioral Sciences. Elsevier BV, 369–374.
[17]
Musa Nushi, Samaneh Shafeie, and Meisam Shaei. 2017. Game-based Teaching
of Stress Placement on Multi-syllabic English Words. Journal of Teaching English
Language Studies 5 (01 2017), 46–64.
[18]
Heath Rose. 2017. The Japanese writing system: Challenges, strategies and self-
regulation for learning kanji. Multilingual Matters.
[19]
Michael Rowley. 1992. Kanji pict-o-graphix: over 1,000 Japanese kanji and kana
mnemonics. Stone Bridge Press.
[20]
Timothy Sanders and Paul Cairns. 2010. Time perception, immersion and music in
videogames. In Proceedings of the 24th BCS interaction specialist group conference.
British Computer Society, 160–167.
[21]
Michael B Syson, Ma Regina E Estuar, and Karl T See. 2012. Abkd: Multimodal
mobile language game for collaborative learning of chinese hanzi and japanese
kanji characters. In Proceedings of the The 2012 IEEE/WIC/ACM International Joint
Conferences on Web Intelligence and Intelligent Agent Technology-Volume 03. IEEE
Computer Society, 311–315.
[22]
Wanda T. Wallace. 1994. Memory for music: Eect of melody on recall of text.
Journal of Experimental Psychology: Learning, Memory, and Cognition 20 (11 1994),
1471–1485.
[23]
Kelly Yap, Clement Zheng, Angela Tay, Ching-Chiuan Yen, and Ellen Yi-Luen
Do. 2015. Word out!: Learning the Alphabet Through Full Body Interactions. In
Proceedings of the 6th Augmented Human International Conference (AH ’15). ACM,
New York, NY, USA, 101–108.
... Specifically, we focus on the different variations of Visual and Auditory aesthetics that occur during in-game death. However, as noted below, these aesthetic decisions can greatly impact player emotion and the overall gaming experience (Kao and Harrell 2016;Keehl and Melcer 2019;Nacke and Grimshaw 2011;Sanders and Cairns 2010), and therefore aesthetics is an important category to consider for the design of in-game death. ...
... Audio, in the form of music and sound effects, is another critical aspect of the aesthetic experience. For instance, both music and sound effects have been shown to impact player immersion and emotional response (Kallinen 2004;Keehl and Melcer 2019;Nacke and Grimshaw 2011;Sanders and Cairns 2010). In our observed games, a number of different sound effects were employed during and immediately after death such as cries or grunts, squishing or wind noises, and short electronic sounds. ...
Chapter
Full-text available
Failure is a central aspect of almost every game experience, driving player perceptions of difficulty and impacting core game user experience concepts such as flow. At the heart of failure in many game genres is player death. While techniques such as dynamic difficulty adjustment have addressed tweaking game parameters to control the frequency of player death occurrence, there is a surprisingly limited amount of research examining how games handle what happens when a player actually dies. We posit that this is a rich, underexplored space with significant implications for player experience and related techniques. This chapter presents our exploration into the space of player death and rebirth through the creation of a generalized taxonomy of death in platformer games—one of the genres that features player death and respawning most heavily. In order to create this taxonomy, we collected and catalogued examples of death and respawning mechanics from 62 recent platformer games released on the digital distribution platform Steam after January 2018. Games selected varied equally across positive, mixed, and negative overall reviews in order to provide a broader range of mechanics, both good and bad. We observed gameplays of each individual game and noted the processes of death and rebirth, respectively. A grounded theory approach was then employed to develop the taxonomy of game death and respawning, resulting in five notable dimensions: (1) obstacles, (2) death conditions, (3) aesthetics, (4) changes to player progress, and (5) respawn locations. Finally, we discuss how the different dimensions and mechanics highlighted in our taxonomy have implications for key aspects of player experience, as well as how they could be used to improve the effectiveness of related techniques such as dynamic difficulty adjustment.
... To address these issues and the above question, we developed an educational kanjiwriting musical game. We named our game Radical Tunes [11], since its core idea is adding mnemonic melodies, or tunes, to radicals, the building blocks of kanji (see Figure 3). We hope that by adding the musical elements to the game we can turn the monotonous task of writing kanji over and over into an enjoyable and educationally effective activity. ...
... In this article, we provide the design rationale for Radical Tunes-which ties mnemonic melodies to radicals within kanji [11]. We describe two experimental studies exploring melody's effect on the engagement, immersion and learning of LWS. ...
Article
Full-text available
Millions of people worldwide are taking up foreign languages with logographic writing systems, such as Japanese or Chinese. Learning thousands of characters necessary for literacy in those languages is a unique challenge to those coming from alphabetic backgrounds, and sustaining motivation in the face of such a momentous task is a struggle for many students. Many games exist for this purpose, but few offer production memory practice such as writing, and the vast majority are thinly veiled flashcards. To address this gap, we created Radical Tunes—a musical kanji-writing game—which combines production practice with musical mnemonic by assigning a melody to each element of a character. We chose to utilize music as it is a powerful tool that can be employed to enhance learning and memory. In this article, we explore whether incorporating melodies into a kanji learning game can positively affect the memorization of the stroke order/direction and overall shape of several Japanese characters, similar to the mnemonic effect of adding music to text. Specifically, we conducted two experimental studies, finding that (1) music improved immersion—an important factor related to learning; and (2) there was a positive correlation between melody presence and character production, particularly for more complex characters.
... Although there are a number of definitions and interpretations of game aesthetics, aesthetics in relation to this taxonomy refers purely to the sensory phenomena that players encounter in the game [52]. Aesthetics can greatly impact player emotion and the overall gaming experience [29,30,51,58], and therefore is an important category to consider for the design of in-game death. Specifically, we focus on the different variations of Visual and Auditory aesthetics that occur during in-game death. ...
... Audio, in the form of music and sound effects, is another critical aspect of the aesthetic experience. For instance, both music and sound effects have been shown to impact player immersion and emotional response [28,30,51,58]. In the corpus, a number of different sound effects were employed during and immediately after death, such as cries or grunts, squishing noises, and electronic sounds. ...
Conference Paper
Full-text available
Failure, often represented through death, is a central aspect of almost every video game. In-game death can drive player perceptions of difficulty and greatly impact the core player experience; however, there is surprisingly limited amounts of research examining how games actually handle this occurrence. We posit that this is a rich, underexplored space with significant implications for player experience and the design of many games. This paper presents our initial exploration into the space of player death and rebirth through the creation of a generalized taxonomy developed from 62 recent platformer games. Our taxonomy consists of five key dimensions: 1) obstacles, 2) death conditions, 3) aesthetics, 4) changes to player progress, and 5) respawn locations.
... Prior research adds further support that the importance of avatar audio may be underappreciated. Audio in games is linked to increased physiological responses [90], emotional realism [20,66], performance [103], and immersion [67,115,131,168,199]. A meta-analysis of 83 studies in virtual environments found that adding audio has a small-to medium-sized effect on presence [55]. ...
Preprint
Full-text available
Avatar customization is known to positively affect crucial outcomes in numerous domains. However, it is unknown whether audial customization can confer the same benefits as visual customization. We conducted a preregistered 2 x 2 (visual choice vs. visual assignment x audial choice vs. audial assignment) study in a Java programming game. Participants with visual choice experienced higher avatar identification and autonomy. Participants with audial choice experienced higher avatar identification and autonomy, but only within the group of participants who had visual choice available. Visual choice led to an increase in time spent, and indirectly led to increases in intrinsic motivation, immersion, time spent, future play motivation, and likelihood of game recommendation. Audial choice moderated the majority of these effects. Our results suggest that audial customization plays an important enhancing role vis-\`a-vis visual customization. However, audial customization appears to have a weaker effect compared to visual customization. We discuss the implications for avatar customization more generally across digital applications.
... There have even been educational interactive narratives designed specifically to teach issues related to ethics (Hodhod et al. 2009), although they have yet to be evaluated for effectiveness. Interactive storytelling (and educational games in general Keehl and Melcer 2019;Melcer et al. 2017; have also been shown to increase engagement/motivation and learning for more rote topics with clearly defined answers and educational outcomes, such as in the areas of STEM (Rowe et al. 2011;Weng et al. 2011;Zhang et al. 2019). However, past work has not fully examined the capabilities of choice-based interactive storytelling games in teaching more ambiguous concepts such as moral reasoning and ethical decision-making. ...
Chapter
Full-text available
Responsible conduct of research (RCR) is an essential skill for all researchers to develop, but training scientists to behave ethically is complex because it requires addressing both cognitive (e.g., conceptual knowledge and moral reasoning skills) and socio-affective (e.g., attitudes) learning outcomes. Both classroom- and web-based forms of RCR training struggle to address these distinct types of learning outcomes simultaneously. This chapter presents a pair of experiments providing initial evidence that playing a single brief session of Academical, a choice-based interactive narrative game, has positive effects on all three key RCR learning outcomes. Our results highlight that utilizing a choice-based interactive storytelling game is a uniquely effective way to holistically address RCR learning outcomes that drive ethical research behaviors.
... There has been a number of works suggesting the potential for interactive narrative/storytelling in improving education [34,59,87,104,123], and prior work has found a substantial positive impact from adding narrative/storytelling to educational games in order to maintain or increase students' motivation [30,39,102,110]. However, there is still a surprisingly limited amount of work evaluating the efficacy of such games on learning outcomes, especially when compared to other genres of educational games [24,65,91], with the few notable exceptions being [80,89,92,110,122,124,131]. The majority of work on educational interactive narrative has instead focused primarily on adaptivity [52,66,77,97,109], interactivity [116,131], player and knowledge modeling [55,83,108], narrative planning and generation [54,105,121,133], emergent narrative [18], player experience [69], and the game creation process [30,117]. ...
Conference Paper
Full-text available
Interactive narratives are widely used to frame and contextualize education in games. However, the specifics of how their designs aid the learning process and outcomes remains relatively unexplored. To better understand this space, a study was conducted that focused on one sub-genre of interactive narrative, Visual Novels. Specifically, in this paper we conducted a survey of thirty-one existing educational Visual Novels, analyzing design elements that fostered learning and delivered educational content. The resulting taxonomy consists of five key dimensions for educational design and teaching strategies within Visual Novels: 1) Teaching Through Choice, 2) Teaching Through Scripted Sequences, 3) Teaching Through Mini-games, 4) Teaching Through Exploration and 5) Non-interactive Teaching. These dimensions demonstrate that there are a number of design considerations for supporting learning through Visual Novels. This work has implications for designers of educational games by classifying the different designs a Visual Novel can employ to teach-ultimately informing how to better present educational subject matter in interactive narrative games.
... There have even been educational interactive narratives designed specifically to teach issues related to ethics [17], although they have yet to be evaluated for effectiveness. Interactive storytelling (and educational games in general [23,34,35]) have also been shown to increase engagement/motivation and learning for more rote topics with clearly defined answers and educational outcomes, such as in the areas of STEM [53,70,73]. However, past work has not fully examined the capabilities of choice-based interactive storytelling games in teaching more ambiguous concepts such as moral reasoning and ethical decision-making. ...
Conference Paper
Full-text available
Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove difficult to teach due to the complexity of problems faced by researchers and the many underlying perspectives involved in such dilemmas. To address this issue, we created Academical, a choice-based interactive storytelling game for RCR education that enables players to experience a story from multiple perspectives. In this paper, we describe the design rationale of Academical, and present results from an initial study comparing it with traditional web-based educational materials from an existing university RCR course. The results highlight that utilizing a choice-based interactive story game is more effective for RCR education, with learners developing significantly higher engagement, stronger overall moral reasoning skills, and better knowledge scores for certain RCR topics.
Article
Full-text available
Accurate pronunciation is an important component of language ability and the main outward linguistic sign of whether someone is a native speaker of a language or not. An area of particular difficulty for Persian-speaking learners of English, which may cause 'foreign accent' or misunderstanding in speaking, is placement of stress on multi-syllable words. Game-based pronunciation teaching can be a pedagogical option to tackle that problematic feature of English pronunciation as it incorporates both implicit and explicit approaches to instruction. This study, therefore, investigates the effects of using a pronunciation game on learning stress placement on multi-syllable English words by young Iranian intermediate EFL learners. Fifty-two intermediate EFL learners were randomly selected through administration of Preliminary English Test (PET) and divided into two groups: the experimental group (group A) containing 27 students and the control group (Group B) containing 25 students. The experimental group received game-based instruction and materials for eight class sessions. Two weeks after the final class, a post-test was conducted to find out the changes. The results showed a significant difference between the performance of experimental and control groups on the post-test. The experimental group outperformed the control group and indicated quite positive attitudes towards the use of the game-based learning of pronunciation. The researchers recommend using such games to EFL teachers in order to motivate and help their students learn pronunciation more effectively.
Book
Full-text available
This is the first book devoted to exploring issues of learning written Japanese, focusing on the challenges the writing system poses for the second language learner. It weaves together previous research on Japanese second language acquisition and kanji learning with original studies on self-regulation and kanji learning strategies. For Japanese language learners, studying kanji requires laborious and often tedious efforts, but it can also build their self-confidence and creativity for devising learning strategies. This groundbreaking book draws on Rose's own experience as a learner, fills a research gap, and offers evidence-based insights to Japanese language learners, teachers, and researchers.
Article
Full-text available
In order to better promote the use of mobile technology in the field of language education, we created an augmented reality mobile game, Guardians of the Mo‘o, for ESL (English as Second Language) learners to enhance their cultural understanding, communicational skills and also language development. Through detailed analysis of the video recorded play-through data, we found that students were able to coordinate their actions and engage in both the virtual and physical world during problem-solving processes. Thus, we argue that the embodied gaming environment which Guardians of the Mo‘o provided afforded the dynamic learning experience, and it is a successful exploration of using mobile technology in creating a new learning environment.
Conference Paper
Acquiring Chinese tones is often considered as the most difficult task in learning Chinese as a Second Language (CSL). Recently, ToneWars, a collaborative mobile learning game, demonstrated the feasibility and efficacy of connecting CSL learners with native speakers for tone learning. However, the synchronous gameplay nature in ToneWars can be hard to scale due to the time constraint and limited availability of native speakers. We present principled research to make ToneWars scalable and sustainable. First, we address the scalability issue via asynchronous modeling of native speakers. Second, we quantify whether a CSL learner achieves native level mastery for a specific phrase, and explore the use of fine-grained feedback on language mastery as a sustainable motivator for language learning. The insights in this research are generalizable to designing second language learning technologies beyond Chinese. In a longitudinal study with 18 CSL learners, we found that asynchronous gameplay significantly improved learning with an average gain of 29.7 tones and 16.4 syllables, and helped participants achieve native level mastery on 58.2 out of 69 phrases.
Conference Paper
Many people want to learn a language but find it difficult to stay engaged. Ideally, we would have language learning tools that can make language learning more enjoyable by simulating immersion in a foreign language environment. Therefore, we adapted Crystallize, a 3D video game for learning Japanese, so that it can be played in virtual reality with the Oculus Rift. Specifically, we explored whether we could leverage virtual reality technology to teach embodied cultural interaction, such as bowing in Japanese greetings. To evaluate the impact of our virtual reality game designs, we conducted a formative user study with 68 participants. We present results showing that the virtual reality design trained players how and when to bow, and that it increased participants' sense of involvement in Japanese culture. Our results suggest that virtual reality technology provides an opportunity to leverage culturally-relevant physical interaction, which can enhance the design of language learning technology and virtual reality games.
Conference Paper
Learning a second language is challenging. Becoming fluent requires learning contextual information about how language should be used as well as word meanings and grammar. The majority of existing language learning applications provide only thin context around content. In this paper, we present work in Crystallize, a language learning game that combines traditional learning approaches with a situated learning paradigm by integrating a spaced-repetition system within a language learning roleplaying game. To facilitate long-term engagement with the game, we added a new quest paradigm, "jobs," that allow a small amount of design effort to generate a large set of highly-scaffolded tasks that grow iteratively. A large-scale evaluation of the language learning game "in the wild" with a diverse set of 186 people revealed that the game was not only engaging players for extended amounts of time but that players learned an average of 8.7 words in an average of 40.5 minutes.
Conference Paper
This paper presents Word Out, an interactive game for learning of the alphabet through full body interaction. Targeted for children 4-7 years old, Word Out employs the Microsoft Kinect to detect the silhouette of players. Players are tasked to twist and form their bodies to match the shapes of the letters displayed on the screen. By adopting full body interactions in games, we aim to promote learning through play, as well as encourage collaboration and kinesthetic learning for children. Over two months, more than 15,000 children have played Word Out installed in two different museums. This paper presents the design and implementation of the Word Out game, preliminary analyses of a survey carried out at the museums to share insights and discusses future work.
Chapter
This chapter focuses on the central limit theorem. A large variety of negligibility assumptions have been made about the differences Xni during the formulation of martingale central limit theorems. The classic condition of negligibility in the theory of sums of independent r.v. requires Xni to be uniformly asymptotically negligible. This is generally a little weaker than the summation condition. The conditional variance Vn2 is one of several estimates of the variance ESn2. It is an intrinsic measure of time for the martingale. For many purposes, the time taken for a martingale to cross a level is best represented through its conditional variance rather than the number of increments up to the crossing. The duality between the definitions of forward and reverse martingales suggests that forward martingale limit theorems should have reverse martingale duals. In the case of the central limit theorem, the analog is perhaps best presented by considering infinite martingale arrays.
Conference Paper
In this paper, we present ToneWars, a collaborative mobile game for learning Chinese as a Second Language (CSL). ToneWars provides a learning experience that combines mastery learning, microlearning, and group-based interaction between CSL learners and native speakers. The game explores how unique input modalities, like touch gestures and speech recognition, can improve language acquisition tasks on mobile devices. We report the design motivations and lessons learned through the iterative design process. We believe many insights from developing ToneWars are generalizable to designing productive language learning technology. Through a 24-participant (12 CSL and native speaker pairs) user study, we found ToneWars provides learning benefits for second-language learners and engages native speakers.