Conference PaperPDF Available

Towards the Prediction of Dyslexia by a Web-based Game with Musical Elements


Abstract and Figures

Current tools for screening dyslexia use linguistic elements, since most dyslexia manifestations are related to difficulties in reading and writing. These tools can only be used with children that have already acquired some reading skills and; sometimes, this detection comes too late to apply proper remediation. In this paper, we propose a method and present DysMusic, a prototype which aims to predict risk of having dyslexia before acquiring reading skills. The prototype was designed with the help of five children and five parents who tested the game using the think aloud protocol and being observed while playing. The advantages of DysMusic are that the approach is language independent and could be used with younger children, i.e., pre-readers.
Content may be subject to copyright.
Towards the Prediction of Dyslexia by a Web-based Game
with Musical Elements
Maria Rauschenberger
Web Research Group Luz Rello
HCI Institute Ricardo Baeza-Yates
Web Research Group
Universitat Pompeu Fabra Carnegie Mellon University Universitat Pompeu Fabra
Emilia Gomez
Music Technology Group
Universitat Pompeu Fabra
Current tools for screening dyslexia use linguistic elements,
since most dyslexia manifestations are related to difficulties
in reading and writing. These tools can only be used with
children that have already acquired some reading skills and;
sometimes, this detection comes too late to apply proper re-
mediation. In this paper, we propose a method and present
DysMusic, a prototype which aims to predict risk of hav-
ing dyslexia before acquiring reading skills. The prototype
was designed with the help of five children and five parents
who tested the game using the think aloud protocol and be-
ing observed while playing. The advantages of DysMusic
are that the approach is language independent and could be
used with younger children, i.e., pre-readers.
CCS Concepts
Human-centered computing Empirical studies in
accessibility; Accessibility design and evaluation meth-
ods; Software and its engineering Interactive games;
Dyslexia; Detection; Pre-Readers; Serious Games; Gamifi-
cation; Web-based Assessment
The American Psychiatric Organization defines dyslexia
as a specific learning disorder which is caused by the ‘phono-
logical skills deficiencies associated with phonological cod-
ing deficits’ [1, 16]. This means that people with dyslexia
have problems spelling words and decoding what they have
heard. Dyslexia does not affect general intelligence, and it
is frequent: around 5% to 15% of the population has this
learning disorder [1].
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from
W4A 2017, April 02 - 04, 2017, Perth, Western Australia, Australia
© 2017 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ISBN 978-1-4503-4900-0/17/04. . . $15.00
Jeffrey P. Bigham
HCI and LT Institutes
Carnegie Mellon University
The impact of dyslexia is related to difficulties with read-
ing and writing and for that reason current tools for screen-
ing dyslexia use linguistic elements. Examples of digital
screening tools for detecting dyslexia in English are Lex-
ercise Screener [7] and Nessy [8]. Additionally, the com-
puter game Dytective [13] is available in English and Span-
ish with an accuracy of 83% for detecting if a person may
have dyslexia or not, according to a game that includes
recognition of linguistic elements, phonological awareness
and reading comprehension.
Reading and spelling tests need a minimum knowledge of
phonological awareness, grammar, and vocabulary of a child
to be able to predict dyslexia. This means that children can
be detected only after they begin to learn to read (generally,
during the first year of school or later). This puts students
with dyslexia behind; therefore, new ways of detecting the
risk of having dyslexia are needed for pre-readers. Also, all
reading and spelling tests are language dependent.
The difficulty in detecting dyslexia before children learn
how to read and write is that the standard linguistic mani-
festations to detect dyslexia (difficulties in reading and writ-
ing) are missing. Hence, to detect dyslexia in a child before
they gain phonological awareness, new indicators of dyslexia
need to be discovered. Huss et al. [5] show that sound struc-
ture is related to the auditory perception of children with
dyslexia. Therefore, we aim to find out if we can distinguish
two groups of pre-readers (with and without dyslexia) using
musical elements in a first prototype of a game called Dys-
Music (see Figure 1). In this paper, we present the design of
the first version of the game DysMusic and an usability test
with five children and five adults to discover problems that
could influence the prediction of the risk of having dyslexia
(first prototype was presented in [12]). The rest of the paper
is organized as follow: next section presents related work;
Section 3 motivates the use of music for predicting dyslexia
while Section 4 presents the design of DysMusic. In Section 5
we present the usability test and the last sections give our
conclusions and future work.
We focus on examples for digital detection of dyslexia for
pre-readers. The tool DYSL-X aims to predict if a child
has dyslexia at the age of five [4, 15]. It contains three
mini-games for different activities to measure indicators (e.g.
Figure 1: Example of the game DysMusic for the
first two clicks on two sound cards (left) and then a
pair of equal sounds is found (right).
‘letter knowledge, FM detection, end-phoneme recognition’
[4]) for dyslexia and takes about one hour to complete. The
mini-games are designed with a high focus on the game play
to motivate the player.
The AGTB 5–12 is a computer-based memory test for
children of five to twelve years-old [14]. The application
provides four tasks for the phonological working memory
processing, six tasks for the central working memory and
two tasks for the visual-spatial working memory. Every task
takes seven minutes for testing.
The Bielefelder Screening (BISC) is for children at
the last year of kindergarten [14]. It contains nine tasks for
phonological perception, phonological working memory pro-
cessing, long-term memory, and visual attention. It clusters
the participants in risk groups for dyslexia and takes around
20 to 25 minutes.
In summary, the games described are designed with a high
focus on the game design and the suitable interaction for
pre-readers. These tools are mainly focused on having letter
knowledge and phonological awareness being AGTB 5–12
and BISC the only ones screening pre-readers. All these
tools require a minimum linguistic knowledge.
People with dyslexia have auditory and visual perception
difficulties that seem to be caused by difficulties with the
short-term memory [10]. Johnson’s work found that 26 out
of 60 children with dyslexia had speech and language dif-
ficulties and that 75 out of 120 children with dyslexia had
problems in the auditory learning [6]. This is applied es-
pecially to the phonemic discrimination and therefore the
perception of new words are problematic [6], as used in the
game Dytective [13].
The visual perception has been explored with various
games as described in the previous section. To explore
the auditory perception without directly referring to letter
knowledge or phonological awareness, music can be used as
the communication channel.
By musical elements we mean different acoustic param-
eters of sound (i.e., frequency,duration, orintensity) which
relate with perceptual parameters (such aspitch,loudness,
ortimbre). Some musical elements for readers at the age
of 10 have been explored, and some differences have been
found for rise time, duration, intensity (related to loudness)
and frequency (related to pitch) between children with
and without dyslexia [5]. The musical elements would be
especially helpful for children that have no knowledge of
written language.
The case of pre-readers is especially challenging, because
dyslexia is known for causing reading and writing problems.
As we mentioned, the auditory perception might be corre-
lated to the causes of dyslexia if the cause is phonological
[16]. To make our detection game language independent, we
will use music (phonological grammar [11]), which is similar
in prosodic structure in language.
In the literature review we already found indicators in the
hearing perception to distinguish between readers with and
without dyslexia [5, 10]. However, the auditory perception
without using linguistic features for pre-readers has been not
studied and used for the prediction of dyslexia in a game.
The game aims to detect differences in the perception of
auditory elements for children with and without dyslexia
caused by problems with the short-term memory. As is well-
known, a game is more fun than doing a test. We can think
that DysMusic is an adaptation of the already existing visual
game Memory.
Instead of finding the same picture under various analog
cards, the child will try to find the same musical element
behind various digital cards. The game has four tasks and
they have to be played without interruptions. Each task
includes two subtasks. One subtask has four cards with two
sound files (see Figure 1) and the other one has six cards
with three sound files. The number of cards is reduced to
four and six cards for two reasons: (i) if there is a difference
in the perception, only two different musical elements will
already show the difference; and (ii) increasing the number of
cards is a challenge in itself, especially for younger children.
To avoid external factors that could cause differences in
the perception (e.g., due to color blindness), no visual cues
(color, shape) where used to distinguish the cards. To avoid
random matches, no match is possible within the first click of
every subtask. Sound cards for every subtask are in random
order and the tasks are counter-balanced with Latin Squares
All musical samples are generated by means of a simple si-
nusoidal waveform (tone) using the Audacity free software.2
Four musical parameters are modified, obtaining a set of
waveforms as the one shown in Figure 2 for Rhythm. The
exact details of each one are given in Table 1.3
First, internal feedback from HCI researchers improved
the application and only minor changes on the game play
needed to be done. After that the usability test was con-
ducted by children and parents who are not the authors of
this paper and are not familiarized with the research.
1An example of a visual memory game can be found on
2Audacity is available at
3The generated musical elements are available at http://bit.
Figure 2: The waveform of the musical element Rhythm.
Musical element Sound Properties
Always the same waveform: sinus, mono files
amplitude: 0.8
musical elements: 2 to 3 sound files
frequency : 440 Hz unless specified
Frequency 2 elements* : 440 Hz, 452.8929 Hz
(change of 3 elements** : 2 previous sounds
tone frequency) and 446.3998 Hz
fade in/out: 0.025s
duration: 0.350s
Length 2 elements: 0.350s, 0.437s
(change of 3 elements: 2 previous sounds
tone length) and 0.525s
fade in/out: 0.025s
Rise Time 2 elements: 0.025s fade in, 0.250s
(change of fade in, both with fade out of
rise time) 0.025s
3 elements: 2 previous sounds and
0.025s fade in and 0.250s fade out
duration: 0.500s
(change in rise time
for different
musical events)
2 elements: (i) musical events with
rise time equal to 100ms, 100ms
and 0.025s; and (ii) rise time equal
to 100ms, 0.025s and 100ms
3 elements: 2 previous sounds plus
one with rise time equal to 0.025s,
100ms, 100ms)
duration: 0.300s
Table 1: Musical elements generated for the four
tasks in DysMusic. * 2 elements: (1/2 semitone -
50 cents interval); ** 3 elements: 3 sounds spaced
by 25 cents (quarter of a semitone) - 2 previous ones.
5.1 Methodology
Design: Since DysMusic is a new application a five user
test [9] is conducted to discover (usability) problems which
could unintended influence the planned study for predicting
risk of having dyslexia. It should be mentioned that a five
user test is a preliminary test for finding major usability
problems and does not aim to find all usability problems.
In a within-subject design, all participants played all four
tasks of the game DysMusic. Only parents entered addi-
tional details for the study (e.g., background information)
while using the think aloud protocol [2].
Participants: We recruited ten participants which were
five children (users) and five parents. Two female parents
(both with age 35) and three male parents (ages 35, 40, and
40) participated. Each parent had two children and five of
their children took part in the user testing of DysMusic.
Two female children (ages 3 and 8) and three male children
(ages 5, 9 and 9) participated. All participants were German
native speakers. Since there is no indication of significant
differences in usability studies for people with or without
dyslexia, we did not differentiate the two groups for the us-
ability study.
Procedure: All participants played DysMusic over the
website4 with the same tablet (Android Galaxy Tab A).
They choose if they wanted to use headphones or not while
playing (only one female parents used headphones). First
the parent read the study instructions and played with the
sound cards (see Figure 1) while using the think aloud pro-
tocol. Afterwards, the parent or the first author filled in the
background details for the child and the child played with
the sound cards. After each subtask the first author asked
the participant ‘How difficult or easy was it to distinguish
between the sounds?’. At the end of game each participant
was asked if they had further comments on the interaction
design of the game or the musical elements.
5.2 Results
We present now the results of the usability test and the
changes we made to DysMusic.
Wording and Text: Generally, the parents found the
text easy to read and to understand. They reported some
spelling mistakes which were mainly caused by the transla-
tion process from English to German e.g. study (English)
vs. Studie (German).
Also they mentioned the large amount of text for Online-
Consent and one parent suggested to only present the im-
portant information of the Online-Consent and give the pos-
sibility for further reading.
Interaction: All participants were able to play the mem-
ory game with the musical elements instantly and all became
faster after the first tasks, independently of the musical ele-
ment. Only the youngest child (3 years old) had major prob-
lems with the amount of six sound cards for all musical ele-
ments and did not find any sound pairs. We consider to only
use four cards when younger children play DysMusic. Some
participants suggested to include the button ‘let’s play’ into
the game summary to make the interaction more visible.
The first author observed that the participants started to
play faster, especially after the first task, and the delay of
releasing the sound cards for the next click helped to control
the speed of the game interaction without being annoyingly
User-Interface: In general, all participants liked the
structure, layout, and the game elements, e.g., story, and the
spoken motivation feedback ‘Yeah’. One participant com-
mented that the footer of the game was very visible (large)
4The demo of the prototype DysMusic is available at http:
and suggested to make it smaller and more conspicuous. We
did this change accordingly.
Musical Elements: All participants commented that
they had to listen and concentrate carefully to be able to
distinguish the musical elements. Participants had a dif-
ferent perception on how difficult it was to distinguish the
sounds and finding the card pairs, depending on the musical
elements. But all participants mentioned the first musical el-
ement of the first subtask always as difficult, independently
from the musical element (because of the counter-balanced
design, the musical element order changed). This seems to
be because it is the first time they play. For the second
subtask, they were already familiarized with the parameter
and were able to name it. Two children and three par-
ents mentioned difficulties in recognizing the musical ele-
ment Length. One parent and one child of this group and
another parent described more difficulties with the musi-
cal element Frequency. Only three parents reported difficul-
ties in distinguishing the musical element Rhythm and two
children mentioned difficulties for the musical element Rise
Functionality: The motivation sound between the exer-
cises were not always played on the tablet and needed to be
debugged for different devices. Besides, the video sometimes
could not be played instantly which might have been caused
by bad Wi-Fi connection. A change of the video player, from
HTML 5: video-tag to YouTube: Iframe-tag, prevented the
loading problems.
Other Comments: In general, all participants found the
task easy to understand. The children expressed more fun
while playing than the parents, e.g., smiling or laughing.
Three users commented that the game was fun and all users
reacted positive on the spoken feedback ‘Well done’ when it
was played. We included more game sound elements, e.g.,
after each found pair we added a spoken feedback like ‘Great’
or ‘Super’.
The main advantages of DysMusic are that it is language
independent and that could be used by pre-readers. Re-
garding the user test, all participants understood the game
easily and played with no interruptions. The next step is to
include the visual elements into the game. After that it will
be possible to perform a set of experiments with 30 partic-
ipants to find how musical and visual elements can distin-
guish between a person with or without dyslexia. To study
the dependency on different languages, experiments will be
conducted in English, Spanish, and German. The best sce-
nario would be to prove that the approach is truly language
independent and only the interface needs to be translated.
The contents of this paper were partly funded by grants
from the National Science Foundation (NSF) and the Na-
tional Institute on Disability, Independent Living, and Reha-
bilitation Research (NIDILRR). We deeply thank all parents
and children for playing DysMusic (German version). Spe-
cial thanks to Hendrik Witzel for his advice during the de-
velopment of DysMusic and suggestions on fonts and color.
We thank Samsung Spain for donating the tablet used in the
[1] American Psychiatric Association. Diagnostic and
Statistical Manual of Mental Disorders. American
Psychiatric Association, may 2013.
[2] H. Brau and F. Sarodnick. Methoden der Usability
Evaluation. Verlag Hans Huber, Bern, 2 edition, 2006.
[3] A. P. Field and G. Hole. How to design and report
experiments. SAGE Publications, 2003.
[4] L. Geurts, V. Vanden Abeele, V. Celis, J. Husson,
L. Van den Audenaeren, L. Loyez, A. Goeleven,
J. Wouters, and P. Ghesqui`ere. DIESEL-X: A
Game-Based Tool for Early Risk Detection of Dyslexia
in Preschoolers. In Describing and Studying
Domain-Specific Serious Games, pages 93–114.
Springer International Publishing, 2015.
[5] M. Huss, J. P. Verney, T. Fosker, N. Mead, and
U. Goswami. Music, rhythm, rise time perception and
developmental dyslexia: Perception of musical meter
predicts reading and phonology. Cortex,
47(6):674–689, jun 2011.
[6] D. J. Johnson. Persistent auditory disorders in young
dyslexic adults. Bulletin of the Orton Society,
30(1):268–276, jan 1980.
[7] Lexercise. Dyslexia Test - Online from Lexercise., 2016.
[Online; accessed 07-September-2016].
[8] Nessy. Dyslexia Screening - Nessy UK. https:
2011. [Online; accessed 07-September-2016].
[9] J. Nielsen. Why You Only Need to Test with 5 Users.
Jakob Nielsens Alertbox, 19(September 23):1–4, 2000.
[10] K. Overy. Dyslexia, Temporal Processing and Music:
The Potential of Music as an Early Learning Aid for
Dyslexic Children. Psychology of Music,
28(2):218–229, oct 2000.
[11] R. F. Port. Meter and speech. Journal of Phonetics,
31:599–611, 2003.
[12] M. Rauschenberger. DysMusic: Detecting Dyslexia by
Web-based Games with Music Elements. In The Web
for All conference Addressing information barriers
W4A’16, Montreal, Canada, 2016. ACM Press.
Doctoral Consortium.
[13] L. Rello, M. Ballesteros, A. Ali, M. Serra, D. Alarc´on,
and J. P. Bigham. Dytective: Diagnosing risk of
dyslexia with a game., 2016.
[Online; accessed 06-January-2017].
[14] C. Steinbrink and T. Lachmann.
orung. Springer Berlin Heidelberg,
Berlin, Heidelberg, 2014.
[15] L. Van den Audenaeren, V. Celis, V. Vanden Abeele,
L. Geurts, J. Husson, P. Ghesqui`ere, J. Wouters,
L. Loyez, and A. Goeleven. DYSL-X: Design of a
tablet game for early risk detection of dyslexia in
preschoolers. In Games for Health, pages 257–266.
Springer Fachmedien Wiesbaden, Wiesbaden, 2013.
[16] F. R. Vellutino, J. M. Fletcher, M. J. Snowling, and
D. M. Scanlon. Specific reading disability (dyslexia):
what have we learned in the past four decades?
Journal of Child Psychology and Psychiatry,
45(1):2–40, jan 2004.
... They also always spell with unpredictable and inconsistent letters. Dyslexic children are prone to writing letters and numbers backward, such as '6' instead of '9' or 'b' instead of 'd.' Due to these difficulties, dyslexic children could not understand what has been taught by their teacher [5]. These difficulties are worsening if dyslexic children are not identified earlier. ...
... Another approach used in this application is Gamification Based Learning [5]. This approach will ensure children have fun, thus enhancing the quality of experience during screening tests. ...
... In general, the use of multimedia elements to convey or display information to the targeted users can enhance the quality of experience [3] [6]. The use of multimedia elements in applications such as Kiddo Disleksia is broadly implemented in numerous areas, especially education because it may increase children's motivation, consequently, improve learning performance [4] [5]. From recent research, the use of multimedia may also increase the ability of dyslexic children in learning and assessment [1]. ...
Full-text available
Identifying dyslexia among Malaysian citizens, especially children nowadays is a prominent issue. The usual practice of dyslexia screening tests in the Malaysian school system is by teacher's observation and intervention. However, this is usually time-consuming, less accurate and lacking additional supporting tools. Moreover, dyslexic children enrolling in a normal education system will encounter many problems for the teacher and the children themselves. A Malay language mobile-based application for a dyslexia screening test named Kiddo Disleksia has been developed to solve this issue. However, it has not been tested in terms of Quality of Experience (QoE) as well as usability. Therefore, this research aims to test the QoE level of Kiddo Disleksia and also to compare the traditional dyslexia screening test with Kiddo Disleksia in terms of usability. To test the QoE, several special education teachers are required to rate Kiddo Disleksia using Mean Opinion Score (MOS). Ten children were tested using Kiddo Disleksia and 80% of them recognized as dyslexic. This result also similar with the traditional paper-based screening test. Therefore, the Kiddo Disleksia application is considered reliable for dyslexia screening tests for children. For QoE, the results show that the mean values of MOS are 3.9 and above. Therefore, the quality of experience during dyslexia screening tests can be enhanced using Kiddo Disleksia.
... Early detection of dyslexia is critical in reducing the adverse effects and consequences of learning disabilities in children's education and serves as a foundation for screening interventions. The majority of recent screening applications use gamification but are language-independent to support pre-readers [26]. For example, gaming apps have been developed for preschoolers (without reading skills) to detect dyslexia through motor skills indicators [25] or through the identification of words with similar phonetic endings [18]. ...
Conference Paper
Full-text available
We designed and evaluated Augmenta11y, a cross-platform application that aims to provide ubiquitous reading and learning companionship to children with dyslexia. We developed the application following the Reflective Agile Iterative Design (RAID) framework, a participatory approach involving dyslexic individuals and special educators to provide feedback throughout the prototyping processes. We then conducted a user study with ten children with dyslexia between 10-14 years of age. We employed the think-aloud, post-usability session interview, and questionnaire methods to evaluate the tool's usability and usefulness. Our study revealed the Augmenta11y application's effectiveness in assisting children with dyslexia by providing affordable and accessible reading practices. We offer comprehensive suggestions and guidelines for designing future assisted reading applications. We foresee the opportunity for Augmenta11y to be an accessible, low-cost assistive reading solution for dyslexic children with little to no access to educational specialists or after-school practices/programs. The Augmenta11y application is available on iOS and Android.
... The content design, user interface, game play, interaction and implementation for the auditory and visual parts of the game are described in the following sections. First, we describe the selection of content and follow with the description of the game MusVis, which already integrates the changes suggested after an usability test (Rauschenberger et al., 2017b). ...
Full-text available
Children with dyslexia have difficulties learning how to read and write. They are often diagnosed after they fail school even if dyslexia is not related to general intelligence. Early screening of dyslexia can prevent the negative side effects of late detection and enables early intervention. In this context, we present an approach for universal screening of dyslexia using machine learning models with data gathered from a web-based language-independent game. We designed the game content taking into consideration the analysis of mistakes of people with dyslexia in different languages and other parameters related to dyslexia like auditory perception as well as visual perception. We did a user study with 313 children (116 with dyslexia) and train predictive machine learning models with the collected data. Our method yields an accuracy of 0.74 for German and 0.69 for Spanish as well as a F1-score of 0.75 for German and 0.75 for Spanish, using Random Forests and Extra Trees, respectively. We also present the collected user data, game content design, potential new auditory input, and knowledge about the design approach for future research to explore universal screening of dyslexia. Universal screening with language-independent content can be used for the screening of pre-readers who do not have any language skills, facilitating a potential early intervention.
... An early game design and content was previously tested with a five-user study [23] to eliminate major usability mistakes which could have an influence on the prediction and validated our game measurements [34]. In our pilot study (n = 178), we found that there were four significant game measurements for Spanish, German, and English as well as eight significant game measurements for Spanish [32]. ...
Conference Paper
Full-text available
Children with dyslexia are often diagnosed after they fail school even if dyslexia is not related to general intelligence. In this work, we present an approach for universal screening of dyslexia using machine learning models with data gathered from a web-based language-independent game. We designed the game content taking into consideration the analysis of mistakes of people with dyslexia in different languages and other parameters related to dyslexia like auditory perception as well as visual perception. We did a user study with 313 children (116 with dyslexia) and train predictive machine learning models with the collected data. Our method yields an accuracy of 0.74 for German and 0.69 for Spanish as well as an F1-score of 0.75 for German and 0.75 for Spanish, using Random Forests and Extra Trees, respectively. To the best of our knowledge, this is the first time that risk of dyslexia is screened using a language-independent content web-based game and machine-learning. Universal screening with language-independent content can be used for the screening of pre-readers who do not have any language skills, facilitating a potential early intervention.
... Howev-er, a large-scale study in a great number of languages (e.g. English, Spanish etc.) was needed to validate the effectiveness of the improved model [29]. ...
Full-text available
In contemporary times, various studies have been conducted around the world, researching the use of Information and Communication Technologies (ICTs) on dyslexia screening in all ages. Screening procedure constitutes the first vital step to investigate the kind of these experiencing difficulties , which sets the foundations for a well-designed intervention. Thus, ICTs is becoming an emerging unique supplementary strategy, using cost-effective and easy to access advanced tests for gaining in-depth knowledge about stu-dents' learning profiles. The purpose of this literature review is to present an indicative number of digital assessment tools (e.g. computerized systems, software , applications etc.) for detecting dyslexia, that have been designed or have been implemented on dyslexic population during the last thirteen years (2005-2018). The classification of proposed papers has been carried out according to the life stage, each of whom analyzed.
This paper presents an ongoing project focused on the framework Listen by Looking, that includes several tools to support the implementation of digital games, allowing the interactive display of music sheets, piano rolls and dialog windows with contextual information. Using this framework, a serious game for mobile devices was designed to entertain and teach music to users by listening to songs and answering questions about the melodies and musical structures learned during the game. The usefulness of game was assessed in a controlled setup, by comparing the answers of the app users with those of people who listened the same music without the app. Furthermore, an external synchronization tool through a data-over-sound channel has been tested in order to simultaneously manage a wide network of devices. The results are encouraging for future developments of the framework, with the possibility of using it during a public performance and with different games.
Full-text available
Children with dyslexia have difficulties learning how to read and write. They are often diagnosed after they fail in school, even though dyslexia is not related to general intelligence. In this thesis, we present an approach for earlier screening of dyslexia using a language-independent game in combination with machine learning models trained with the interaction data. By earlier, we mean before children learn how to read and write. To reach this goal, we designed the game content with knowl- edge of the analysis of word errors from people with dyslexia in different languages and the parameters reported to be related to dyslexia, such as auditory and visual perception. With our two de- signed games (MusVis and DGames), we collected data sets (313 and 137 participants) in different languages (mainly Spanish and German) and evaluated them with machine learning classifiers. For MusVis we mainly use content that refers to one single acoustic or visual indicator, while DGames content refers to generic content related to various indicators. Our method provides an accuracy of 0.74 for German and 0.69 for Spanish and F1-scores of 0.75 for German and 0.75 for Spanish in MusVis when Random Forest and Extra Trees are used. DGames was mainly evaluated with German and reached a peak accuracy of 0.67 and a peak F1-score of 0.74. Our results open the possibility of low-cost and early screening of dyslexia through the Web.
Conference Paper
Dyslexia is a widespread specific learning disorder, which can have a particularly negative influence on the learning success of children. Early detection of dyslexia is the foundation for early intervention, which is the key to reduce the adverse effects of dyslexia, e.g., bad school grades. In this paper, we present the prototype of a puzzle app, which we explicitly designed with the human-centred design (HCD) process for dyslexia screening in pre-reader using new indicators related to motor skills to ensure users needs for the data collection to apply machine learning prediction. The app records the telemetry of the gaming sequence in order to derive future statements about the prevalence of dyslexia based on the telemetry data. The high-fidelity prototype was evaluated with a five-user test usability study with five German-speaking child-parent pairs. The results show how young children and parents are interacting with new games, and how new applications (web and mobile technologies) which are used for online experiments, could be developed. The usability of the prototype is suitable for the target group with only minor limitations. CCS
Nowadays, being excluded from the web is a huge disadvantage. People with dyslexia have, despite their general intelligence, difficulties for reading and writing through their whole life. Therefore, web technologies can help people with dyslexia to improve their reading and writing experience on the web. This chapter introduces the main technologies and many examples of tools that support a person with dyslexia in processing information on the web, either in assistive applications for reading and writing as well as using web applications/games for dyslexia screening and intervention.
Conference Paper
Full-text available
The aim of this research is to show that a playful approach combined with music can detect children with dyslexia. Early detection will prevent children from suffering in school until they are detected due to bad grades. Our envisioned web application will contribute to 10% of the population by giving them a chance to succeed in life and find their skills to impress the world.
Warum haben ca. 5% der Grundschulkinder trotz normaler Begabung und ausreichendem Schulunterricht erhebliche Schwierigkeiten beim Erlernen des Lesens und Schreibens? Wodurch ist eine solche Lese- Rechtschreibstörung gekennzeichnet, wie wirkt sie sich im Schulalltag und im späteren Leben aus? Wie kann man die Lese-Rechtschreibstörung möglichst früh diagnostizieren und wie kann man die betroffenen Kinder optimal fördern? Dies sind zentrale Fragen, die in diesem Buch auf der Basis des aktuellen Standes der internationalen Forschung behandelt werden. Zunächst wird die unbeeinträchtigte Entwicklung der Schriftsprache dargestellt, eine wichtige Grundlage für das Verständnis der Lese-Rechtschreibstörung. Danach wird das Erscheinungsbild der Lese-Rechtschreibstörung ausführlich beschrieben sowie die klinische Klassifikation erläutert und kritisch diskutiert. Im Anschluss werden verschiedene wissenschaftlich begründete Ansätze zu Ursachen der Lese- Rechtschreibstörung besprochen. Das bis dahin vermittelte Wissen bildet das Fundament für die Auseinandersetzung mit eher anwendungsbezogenen Fragen zur Diagnostik und Förderung: Kann man ein Risiko für Lese-Rechtschreibschwierigkeiten bereits im Vorschulalter feststellen? Welche Verfahren werden für die Diagnostik der Lese-Rechtschreibstörung eingesetzt? Kann man der Entstehung von Lese-Rechtschreibschwierigkeiten vorbeugen? Welche Ansätze zur Intervention gibt es und welche sind besonders vielversprechend? Die Zielgruppen Studierende der Fachrichtungen (Schul-)Psychologie, Logopädie und Patholinguistik sowie Lehramtsstudenten und alle, die sich beruflich mit der Thematik auseinandersetzen, insbesondere Lehrer, Schulpsychologen, Psychologen, Lerntherapeuten, Erzieher, Psychotherapeuten, Kinder- und Jugendpsychiater sowie der interessierte Laie. Die Autoren Priv.-Doz. Dr. habil. Claudia Steinbrink ist Wissenschaftliche Mitarbeiterin am Lehrstuhl für Kognitive und Entwicklungspsychologie der Technischen Universität Kaiserslautern. Schwerpunkte ihrer Lehre sind schulbezogene Themen der Entwicklungs- und Pädagogischen Psychologie. Den Schwerpunkt ihrer Forschung bilden grundlagenwissenschaftliche wie anwendungsbezogene Projekte zu kognitiven Grundlagen der Schriftsprachentwicklung und der Lese-Rechtschreibstörung. Prof. Dr. habil. Thomas Lachmann ist Inhaber des Lehrstuhls für Kognitive und Entwicklungspsychologie und Leiter des Fernstudiengangs Psychologie kindlicher Lern- und Entwicklungsauffälligkeiten an der Technischen Universität Kaiserslautern. Zusammen mit seiner Arbeitsgruppe arbeitet er an der experimentellen Erforschung der Entwicklung kognitiver Funktionen, insbesondere der neuro-kognitiven Grundlagen des Schriftspracherwerbs und der Lese- Rechtschreibstörung.
The goal of the DYSL-X project is to develop a tool to predict whether a preschooler (5 yrs) shows high risks for developing dyslexia. This tool is a tablet game that incorporates tests to take specific performance measures that allow for this prediction. The game will thus serve as an assessment tool to be used in school psychology services and clinical diagnostic and rehabilitation centers. In order to design the optimal tablet game for preschoolers, during the first phase of the projects several existing games for preschoolers were evaluated in order to derive design guidelines for games targeted at preschoolers. These design guidelines are presented in this paper and next, we show how these guidelines were used to develop the final game of the DYSL-X project.
DIESEL-X is a computer game that was developed to detect a high risk for developing dyslexia in preschoolers. The game includes three mini-games that test the player on three skills that are considered to yield outcome measures that predict the onset of dyslexia: the detection threshold of frequency modulated tones, a test on phonological awareness in which the player has to identify words that have the same phonetic ending, and a test on letter knowledge. In order to keep the motivation of the player high during testing, these tests are embedded in a computer game. We discuss the participatory design process that was adopted to design and develop the game, the rationale behind the design decisions, and we describe the resulting games. © Springer International Publishing Switzerland 2015. All rights reserved.
In diesem Kapitel werden zunächst die genetischen und neurobiologischen Grundlagen der Lese-Rechtschreibstörung vermittelt (▶ Abschn. 4.1 und ▶ Abschn. 4.2). Im Hauptteil des Kapitels werden dann in ▶ Abschn. 4.3 einige etablierte Theorien zu kognitiven Ursachen der Lese-Rechtschreibstörung genauer besprochen, die sich darin unterscheiden, welche Informationsverarbeitungsdefizite konkret für die gravierenden und überdauernden Schwierigkeiten im Lesen und Schreiben verantwortlich gemacht werden, und wie die Verursachung dieser Defizite auf biologischer Ebene erklärt wird. Folgende Theorien zur kognitiven Ursache der Lese-Rechtschreibstörung werden dargestellt: die phonologische Theorie der Lese-Rechtschreibstörung, drei Theorien zur Störung der zeitlichen auditiven Informationsverarbeitung bei Lese-Rechtschreibstörung, die Cerebellum-Theorie der Lese-Rechtschreibstörung, die magnozelluläre Theorie der Lese-Rechtschreibstörung sowie die Theorie des funktionalen Koordinationsdefizits.
There is extensive evidence suggesting that the language and literacy problems experienced by dyslexics are caused by deficits in various sensory, cognitive and motor processes. Several theories on the underlying cause of these deficits are converging on the idea that the fundamental problems derive from abnormal neurological timing, or "temporal processing". It has been proposed that temporal processing ability can be improved through training, and that this will lead to improved language and literacy skills (Tallal et al., 1996). Music training, requiring very accurate timing skills, can offer a medium for the development and improvement of temporal processing ability, and thus may provide a valuable form of extra remediation for dyslexic children. This article reports some preliminary work in this area, which has produced encouraging results. Further research is also outlined. © 2000 by the Society for Research in Psychology of Music and Music Education.
Please do not request a full text copy of this book, I find such requests very disrespectful. Thanks for your understanding. I have a lot of free material on my website and youTube channel.