ThesisPDF Available

Early Screening of Dyslexia Using a Language-Independent Content Game and Machine Learning

Authors:
Thesis

Early Screening of Dyslexia Using a Language-Independent Content Game and Machine Learning

Abstract

Children with dyslexia have difficulties learning how to read and write. They are often diagnosed after they fail in school, even though dyslexia is not related to general intelligence. In this thesis, we present an approach for earlier screening of dyslexia using a language-independent game in combination with machine learning models trained with the interaction data. By earlier, we mean before children learn how to read and write. To reach this goal, we designed the game content with knowl- edge of the analysis of word errors from people with dyslexia in different languages and the parameters reported to be related to dyslexia, such as auditory and visual perception. With our two de- signed games (MusVis and DGames), we collected data sets (313 and 137 participants) in different languages (mainly Spanish and German) and evaluated them with machine learning classifiers. For MusVis we mainly use content that refers to one single acoustic or visual indicator, while DGames content refers to generic content related to various indicators. Our method provides an accuracy of 0.74 for German and 0.69 for Spanish and F1-scores of 0.75 for German and 0.75 for Spanish in MusVis when Random Forest and Extra Trees are used. DGames was mainly evaluated with German and reached a peak accuracy of 0.67 and a peak F1-score of 0.74. Our results open the possibility of low-cost and early screening of dyslexia through the Web.
Early Screening of Dyslexia Using a
Language-Independent Content Game and
Machine Learning
Maria Rauschenberger
DOCTORAL THESIS UPF / 2019
Directors of the thesis:
Prof. Dr. Ricardo Baeza-Yates
Department of Information and Communication Technologies,
Universitat Pompeu Fabra
Dr. Luz Rello
Department of Information Systems and Technology,
IE Business School, IE University
ii
Dedicated to people with dyslexia
For your own sanity, you have to remember that not all problems
can be solved. Not all problems can be solved, but all problems
can be illuminated.” – Ursula Franklin
iii
Abstract
Children with dyslexia have difficulties learning how to read and
write. They are often diagnosed after they fail in school, even
though dyslexia is not related to general intelligence. In this the-
sis, we present an approach for earlier screening of dyslexia using
a language-independent game in combination with machine learn-
ing models trained with the interaction data. By earlier, we mean
before children learn how to read and write.
To reach this goal, we designed the game content with knowl-
edge of the analysis of word errors from people with dyslexia in
different languages and the parameters reported to be related to
dyslexia, such as auditory and visual perception. With our two de-
signed games (MusVis and DGames), we collected data sets (313
and 137 participants) in different languages (mainly Spanish and
German) and evaluated them with machine learning classifiers. For
MusVis we mainly use content that refers to one single acoustic or
visual indicator, while DGames content refers to generic content
related to various indicators. Our method provides an accuracy of
0.74 for German and 0.69 for Spanish and F1-scores of 0.75 for
German and 0.75 for Spanish in MusVis when Random Forest and
Extra Trees are used. DGames was mainly evaluated with German
and reached a peak accuracy of 0.67 and a peak F1-score of 0.74.
Our results open the possibility of low-cost and early screening of
dyslexia through the Web.
vii
Resum
Els nens amb dislèxia tenen dificultats per aprendre a llegir i es-
criure. Sovint se’ls diagnostica després de fallar a l’escola, encara
que la dislèxia no estigui relacionada amb la intel·ligència general.
En aquesta tesi, presentem un enfocament per a la selecció prè-
via de la dislèxia mitjançant un joc independent del llenguatge en
combinació amb models d’aprenentatge automàtic formats amb les
dades d’interacció. Abans volem dir abans que els nens aprenguin
a llegir i escriure.
Per assolir aquest objectiu, vam dissenyar el contingut del joc
amb el coneixement de l’anàlisi de paraules d’error de persones
amb dislèxia en diferents idiomes i els paràmetres relacionats amb
la dislèxia com la percepció auditiva i la percepció visual. Amb els
nostres dos jocs dissenyats (MusVis i DGames) vam recollir con-
junts de dades (313 i 137 participants) en diferents idiomes (prin-
cipalment espanyols i alemanys) i els vam avaluar amb classifica-
dors d’aprenentatge automàtic. Per a MusVis utilitzem principal-
ment contingut que fa referència a un únic indicador acústic o vi-
sual, mentre que el contingut de DGames fa referència a diversos
indicadors (també contingut genèric). El nostre mètode proporcio-
na una precisió de 0,74 per a l’alemany i 0,69 per a espanyol i una
puntuació de F1 de 0,75 per a alemany i de 0,75 per a espanyol
a MusVis quan s’utilitzen arbres extraestats. DGames es va ava-
luar principalment amb alemany i obté la màxima precisió de 0,67
i la màxima puntuació de F1 de 0,74. Els nostres resultats obren
la possibilitat de la dislèxia de detecció precoç a baixos costos ia
través del web.
viii
Resumen
Los niños con dislexia tienen dificultades para aprender a leer y
escribir. A menudo se les diagnostica después de fracasar en la
escuela, incluso aunque la dislexia no está relacionada con la in-
teligencia general. En esta tesis, presentamos un enfoque para la
detección temprana de la dislexia utilizando un juego independi-
ente del idioma en combinación con modelos de aprendizaje au-
tomático entrenados con los datos de la interacción. Temprana
aquí significa antes que los niños aprenden a leer y escribir.
Para alcanzar este objetivo, diseñamos el contenido del juego
con el conocimiento del análisis de las palabras de error de las per-
sonas con dislexia en diferentes idiomas y los parámetros reporta-
dos relacionados con la dislexia, tales como la percepción auditiva
y la percepción visual. Con nuestros dos juegos diseñados (MusVis
y DGames) recogimos conjuntos de datos (313 y 137 participantes)
en diferentes idiomas (principalmente español y alemán) y los eva-
luamos con clasificadores de aprendizaje automático. Para MusVis
utilizamos principalmente contenido que se refiere a un único indi-
cador acústico o visual, mientras que el contenido de DGames se
refiere a varios indicadores (también contenido genérico). Nues-
tro método proporciona una exactitud de 0,74 para alemán y 0,69
para español más una puntuación F1 de 0,75 para alemán y 0,75
para español en MusVis cuando se utilizan Random Forest y Ex-
tra Trees, respectivamente. DGames fue evaluado principalmente
con alemán, obtienendo una exactitud de 0,67 y una puntuación F1
de 0,74. Nuestros resultados abren la posibilidad de una detección
precoz y de bajo coste de dislexia a través de la Web.
ix
Abstrakt
Kinder mit einer Lese-/Rechtschreibstörung (LRS) haben Schwie-
rigkeiten, Lesen und Schreiben zu lernen. Sie werden oft nach dem
Schulversagen diagnostiziert, auch wenn die LRS unabhängig von
der allgemeinen Intelligenz ist. In dieser Arbeit stellen wir einen An-
satz für ein früheres Screening von der LRS mit einem sprachunab-
hängigen Spiel in Kombination mit machinellen Lernmodellen vor,
die mit den Interaktionsdaten trainiert wurden. Mit früher meinen
wir, bevor Kinder das Lesen und Schreiben lernen.
Um dieses Ziel zu erreichen, haben wir Spielinhalte mit dem
folgenden Wissen erstellt: die Analyse von Fehlerwörtern von Men-
schen mit einer (LRS) in verschiedenen Sprachen und die Parame-
ter, die mit der LRS zusammenhängen, wie z.B. auditive Wahrneh-
mung und visuelle Wahrnehmung. Mit unseren beiden entwickelten
Spielen (MusVis und DGames) haben wir Datensätze (313 und 137
Teilnehmer/innen) in verschiedenen Sprachen (hauptsächlich Spa-
nisch und Deutsch) gesammelt und mit Machine Learning Classifier
ausgewertet. Wir verwenden für MusVis hauptsächlich Inhalte, die
sich auf einen einzigen akustischen oder visuellen Indikator bezie-
hen, während DGames-Inhalte auf verschiedene Indikatoren (auch
generische) bezogen sind. Unsere Methode liefert für MusVis eine
Genauigkeit von 0,74 für Deutsch und 0,69 für Spanisch und einen
F1-Score von 0,75 für Deutsch und 0,75 für Spanisch in MusVis,
wenn Random Forest und Extra Trees verwendet werden. DGa-
mes wurde hauptsächlich mit deutscher Sprache ausgewertet und
erreicht die höchste Genauigkeit von 0,67 und den höchsten F1-
Score von 0,74. Unsere Ergebnisse eröffnen die Möglichkeit, die
Lese-/Rechtschreibstörung kostengünstig und frühzeitig über das
Internet zu erkennen.
x
Bibliography
[1] M. I. Ahmad and S. Shahid. Design and Evaluation of Mo-
bile Learning Applications for Autistic Children in Pakistan. In
J. Abascal, S. Barbosa, M. Fetter, T. Gross, P. Palanque, and
M. Winckler, editors, INTERACT, volume 9296 of Lecture
Notes in Computer Science, pages 436–444, Cham, 2015.
Springer International Publishing.
[2] American Psychiatric Association. Diagnostic and Statistical
Manual of Mental Disorders. American Psychiatric Associa-
tion, London, England, May 2013.
[3] J. Arnowitz, M. Arent, and N. Berger. Effective Prototyping
for Software Makers. Elsevier, 2007.
[4] T. Asvestopoulou, V. Manousaki, A. Psistakis, I. Smyr-
nakis, V. Andreadakis, I. M. Aslanides, and M. Papadopouli.
Dyslexml: Screening tool for dyslexia using machine learn-
ing, 2019.
[5] R. Baeza-Yates. Big, small or right data: Which is the proper
focus? https://www.kdnuggets.com/2018/10/big-small-
right-data.html, 2018. [Online, accessed 22-July-2019].
153
[6] A. Bandhyopadhyay, D. Dey, and R. K. Pal. Prediction of
Dyslexia using Machine Learning — A Research Travelogue,
volume 24. Springer Singapore, 2018.
[7] P. Banerjee. About Face 2.0: The Essentials of Interac-
tion Design: Alan Cooper and Robert Reimann Published by
John Wiley & Sons, 2003, 576 pp, ISBN 0764526413, vol-
ume 3. Wiley Publishing, Inc., 2004.
[8] A. A. Benasich and P. Tallal. Infant discrimination of rapid au-
ditory cues predicts later language impairment. Behavioural
Brain Research, 136(1):31–49, 2002.
[9] A. A. Benasich and P. Tallal. Dyslexia risk gene relates to
representation of sound in the auditory brainstem. Develop-
mental Cognitive Neuroscience, 24(February):63–71, 2017.
[10] G. Berget and A. MacFarlane. Experimental Methods in IIR.
In Proceedings of the 2019 Conference on Human Informa-
tion Interaction and Retrieval - CHIIR ’19, pages 93–101,
New York, New York, USA, 2019. ACM Press.
[11] S. Bertoni, A. Facoetti, S. Franceschini, C. E. Palazzi, and
D. Ronzani. A Web Application for Reading and Attentional
Assessments. In Proceedings of the 4th EAI International
Conference on Smart Objects and Technologies for Social
Good - Goodtechs ’18, pages 142–147, New York, New York,
USA, 2018.
[12] Bildungsministerium. Rundschreiben des Bildungsminis-
teriums: Schule und Datenschutz-Grundverordnung der
EU (DSGVO) (Circular letter of the Ministry of Educa-
tion: School and General Data Protection Regulation of the
EU (GDPR)). https://schulrecht-sh.de/texte/d/dgsvo_
und_schule.pdf. [Online, accessed 13-June-2019].
154
[13] C. M. Bishop. Pattern Recognition and Machine Learning,
volume 1. Springer Science+Business Media, LLC, Singa-
pore, 2006.
[14] D. W. Black, J. E. Grant, and American Psychiatric Associ-
ation. DSM-5 guidebook: The essential companion to the
Diagnostic and statistical manual of mental disorders, fifth
edition. American Psychiatric Association, 5th edition edi-
tion, 2016.
[15] E. Borleffs, B. A. M. Maassen, H. Lyytinen, and F. Zwarts.
Cracking the Code: The Impact of Orthographic Trans-
parency and Morphological-Syllabic Complexity on Read-
ing and Developmental Dyslexia. Frontiers in Psychology,
9(JAN):1–19, jan 2019.
[16] H. Brau and F. Sarodnick. Methoden der Usability Evaluation
(Methods of Usability Evaluation). Verlag Hans Huber, Bern,
2 edition, 2006.
[17] J. Brownlee. A Gentle Introduction to Effect Size Measures
in Python. https://machinelearningmastery.com/effect-
size-measures-in-python/. [Online, accessed 10-June-
2019].
[18] Bruno and R. M. Comprehensive Test of Phonological Pro-
cessing (CTOPP). Diagnostique, 24(1), 2000.
[19] K. Caine. Local standards for sample size at chi. In CHI’16,
pages 981–992, 2016.
[20] B. Carbol. Research Brief: Use of the Dyslexia Quest App as
a Screening Tool. Technical report, Schmidt & Carbol Con-
sulting Group, 2014.
[21] H. W. Catts, A. McIlraith, M. S. Bridges, and D. C. Nielsen.
Viewing a phonological deficit within a multifactorial model of
dyslexia. Reading and Writing, 30(3):613–629, mar 2017.
155
[22] E. Chatzidaki, M. Xenos, and C. Machaira. Let’s Play a
Game! Kin-LDD: A Tool for Assisting in the Diagnosis of Chil-
dren with Learning Difficulties. Multimodal Technologies and
Interaction, 3(1):16, mar 2019.
[23] C. Coleman, N. Gregg, L. McLain, and L. W. Bellair. A Com-
parison of Spelling Performance Across Young Adults With
and Without Dyslexia. Assessment for Effective Intervention,
34(2):94–105, 2008.
[24] F. Cuetos, J. L. Ramos, and E. Ruano. PROESC. Evalu-
ación de los procesos de escritura (Writing processes as-
sessment). Madrid: TEA, 2002.
[25] F. Cuetos, B. Rodríguez, E. Ruano, and D. Arribas.
PROLEC-R: Batería de Evaluación de los Procesos Lec-
tores, Revisada (Battery of reading processes assessment—
Revised), 2007.
[26] F. J. Damerau. A technique for computer detection and
correction of spelling errors. Communications of the ACM,
7(3):171–176, mar 1964.
[27] G. De Zubicaray and N. O. Schiller. The Oxford handbook
of neurolinguistics. Oxford University Press, New York, NY,
2018.
[28] S. Deterding, D. Dixon, R. Khaled, and L. Nacke. From Game
Design Elements to Gamefulness: Defining ?Gamification?
Proceedings of the 2011 annual conference extended ab-
stracts on Human factors in computing systems - CHI EA ’11,
page 2425, 2011.
[29] Deutsche Gesellschaft für Kinder und Jugendpsychiatrie
Psychosomatik und Psychotherapie e.V. (DGKJP). Diagnos-
tik und Behandlung von Kindern und Jugendlichen mit Lese-
und / oder Rechtschreibstörung Registernummer 028-044
156
(Diagnostics and treatment of children and adolescents with
reading and / or spelling disorder register number 028-044).
https://www.awmf.org/uploads/tx_szleitlinien/028-
044l_S3_Lese-Rechtschreibst%C3%B6rungen_Kinder_
Jugendliche_2015-06.pdf. [Online, accessed 22-Mai-
2018].
[30] T. Dietterich. Overfitting and undercomputing in machine
learning. ACM Computing Surveys, 27(3):326–327, sep
1995.
[31] G. Esser, A. Wyschkon, and M. Schmidt. Was wird
aus Achtjährigen mit einer Lese- und Rechtschreibstörung?
Ergebnisse im Alter von 25 Jahren. (What about eight-year-
olds with a reading and spelling disorder? Results at the age
of 25 years.). Zeitschrift für Klinische Psychologie und Psy-
chotherapie, 4:235–242, 2002.
[32] European Commission/EACEA/Eurydice. The Struc-
ture of the European Education Systems 2018/19:
Schematic Diagrams. Eurydice Facts and Figures.
https://eacea.ec.europa.eu/national-policies/
eurydice/sites/eurydice/files/the_structure_of_
the_european_education_systems_2018_19.pdf, 2018.
[Online, accessed 22-Mai-2018].
[33] European Union. General Data Protection Regulation. Reg-
ulation (EU) 2016/679 of the European Parliament and of the
Council of 27 April 2016, repealing Directive 95/46/EC, Offi-
cial Journal L 119, 4.5.: 1–88. https://eur-lex.europa.
eu/eli/reg/2016/679, 2016.
[34] J. J. Faraway and N. H. Augustin. When small data beats
big data. Statistics & Probability Letters, 136:142–145, may
2018.
157
[35] H. Fastl and E. Zwicker. Psychoacoustics. Springer Berlin
Heidelberg, Berlin, Heidelberg, third edition, 2007.
[36] A. Fawcett and R. Nicolson. The Dyslexia Screening Test:
Junior (DST-J). Harcourt Assessment, 2004.
[37] A. P. Field and G. Hole. How to design and report experi-
ments. SAGE Publications, London, 2003.
[38] S. Franceschini, S. Gori, M. Ruffino, K. Pedrolli, and A. Fa-
coetti. A Causal Link between Visual Spatial Attention and
Reading Acquisition. Current Biology, 22(9):814–819, may
2012.
[39] A. Frid and Z. Breznitz. An SVM based algorithm for analysis
and discrimination of dyslexic readers from regular readers
using ERPs. In 2012 IEEE 27th Convention of Electrical and
Electronics Engineers in Israel, pages 1–4. IEEE, nov 2012.
[40] N. Gaab. The Gaab Lab for Developmental Neuroscience.
http://thegaablab.com/app.html, 2017. [Online, ac-
cessed 06-June-2019].
[41] N. Gaab. Early Literacy Screener. https://www.
bostonearlyliteracyscreener.com/, 2018. [Online, ac-
cessed 06-June-2019].
[42] N. Gaab. The Screening Battery Risk Indicators Assess-
ment. http://thegaablab.com/imgs/PowerpointApp.pdf,
unknown. [Online, accessed 06-June-2019].
[43] O. Gaggi, G. Galiazzo, C. Palazzi, A. Facoetti, and
S. Franceschini. A serious game for predicting the risk of de-
velopmental dyslexia in pre-readers children. In 2012 21st
International Conference on Computer Communications and
Networks, ICCCN 2012 - Proceedings, pages 1–5, Munich,
Germany, 2012. IEEE.
158
[44] O. Gaggi, C. E. Palazzi, M. Ciman, G. Galiazzo, S. Frances-
chini, M. Ruffino, S. Gori, A. Facoetti, O. Gaggi, C. E.
Palazzi, G. Galiazzo, S. Franceschini, S. Gori, and A. Fa-
coetti. Serious Games for Early Identification of Develop-
mental Dyslexia. Comput. Entertain. Computers in Entertain-
ment, 15(4), 2017.
[45] L. Geurts, V. Vanden Abeele, V. Celis, J. Husson, L. Van
den Audenaeren, L. Loyez, A. Goeleven, J. Wouters, and
P. Ghesquière. DIESEL-X: A Game-Based Tool for Early
Risk Detection of Dyslexia in Preschoolers. In Describing
and Studying Domain-Specific Serious Games, pages 93–
114. Springer International Publishing, Switzerland, 2015.
[46] U. Goswami. A temporal sampling framework for develop-
mental dyslexia. Trends in Cognitive Sciences, 15(1):3–10,
jan 2011.
[47] U. Goswami, L. Barnes, N. Mead, A. J. Power, and V. Leong.
Prosodic Similarity Effects in Short-Term Memory in Develop-
mental Dyslexia. Dyslexia, 22(4):287–304, 2016.
[48] U. Goswami, R. Cumming, and A. Wilson. Rhythmic Percep-
tion, Music and Language: A New Theoretical Framework for
Understanding and Remediating Specific Language Impair-
ment Background to the Project. Technical report, University
of Cambridge, 2016.
[49] U. Goswami, N. Mead, T. Fosker, M. Huss, L. Barnes, and
V. Leong. Impaired perception of syllable stress in children
with dyslexia: A longitudinal study. Journal of Memory and
Language, 69(1):1–17, 2013.
[50] GRAPHO GROUP. GraphoGame — Home. https://www.
graphogame.com/. [Online, accessed 19-June-2019].
159
[51] M. Grund, C. L. Naumann, and G. Haug. Diagnostischer
Rechtschreibtest für 5. Klassen: DRT 5 (Diagnostic spelling
test for fifth grade: DRT 5). Deutsche Schultests. Beltz Test,
Göttingen, 2., aktual edition, 2004.
[52] T. K. Guttorm, P. H. T. Leppänen, A. Tolvanen, and H. Lyyti-
nen. Event-related potentials in newborns with and with-
out familial risk for dyslexia: principal component analysis
reveals differences between the groups. Journal of Neural
Transmission, 110(9):1059–1074, sep 2003.
[53] J. A. Hämäläinen, H. K. Salminen, and P. H. T. Leppänen.
Basic Auditory Processing Deficits in Dyslexia: Systematic
Review of the Behavioral and Event-Related Potential/ Field
Evidence. Journal of Learning Disabilities, 46(5):413–427,
2013.
[54] J. Hamari, J. Koivisto, and H. Sarsa. Does Gamification
Work? – A Literature Review of Empirical Studies on Gam-
ification. In 2014 47th Hawaii International Conference on
System Sciences, pages 3025–3034. IEEE, jan 2014.
[55] M. Hasselhorn, R. Schumann-Hengsteler, J. Gronauer,
D. Grube, C. Mähler, I. Schmid, K. Seitz-Stein, and
C. Zoelch. AGTB 5-12 — Arbeitsgedächtnistestbat-
terie für Kinder von 5 bis 12 Jahren (AGTB 5-12 —
Working memory tasks to test children at the age
of 5 till 12). https://www.testzentrale.de/shop/
arbeitsgedaechtnistestbatterie-fuer-kinder-von-
5-bis-12-jahren.html, 2012. [Online, accessed 04-June-
2019].
[56] M. Hasselhorn and C. Zoelch. Funktionsdiagnostik des Ar-
beitsgedächtnisses (Functional diagnostics of the working
memory). Hogrefe Verlag, Göttingen, 2012.
160
[57] T. Helland and R. Kaasa. Dyslexia in English as a second
language. Dyslexia, 11(1):41–60, feb 2005.
[58] A. Hevner, S. T. March, J. Park, and S. Ram. Design Science
in Information Systems Research. MIS Quarterly, 28(1):75,
2004.
[59] A. Hinderks, M. Schrepp, M. Rauschenberger, S. Olschner,
and J. Thomaschewski. Konstruktion eines Fragebogens
für jugendliche Personen zur Messung der User Experience.
(Construction of a questionnaire for young people to mea-
sure user experience.). In Usability Professionals Konferenz
2012, pages 78–83. German UPA e.V., Stuttgart, 2012.
[60] B. Hornsby and P. Cox. Overcoming dyslexia : a straight-
forward guide for families and teachers. Vermilion, London,
1996.
[61] R. R. Huffman, A. Roesler, and B. M. Moon. What is design in
the context of human-centered computing? IEEE Intelligent
Systems, 19(4):89–95, 2004.
[62] K. Huotari and J. Hamari. Defining gamification. In Proceed-
ing of the 16th International Academic MindTrek Conference
on - MindTrek ’12, MindTrek ’12, page 17, New York, NY,
USA, 2012. ACM.
[63] M. Huss, J. P. Verney, T. Fosker, N. Mead, and U. Goswami.
Music, rhythm, rise time perception and developmental
dyslexia: Perception of musical meter predicts reading and
phonology. Cortex, 47(6):674–689, jun 2011.
[64] IDC Worldwide. Shipment forecast of tablets, laptops and
desktop PCs worldwide from 2010 to 2019 (in million units).
Statista, 2020:2020, 2016.
161
[65] International Dyslexia Association. Frequently Asked Ques-
tions, 2019.
[66] D. Irblich, S. K. Diakonie, and G. Renner. AGTB 5-12 —
Arbeitsgedächtnistestbatterie für Kinder von 5 bis 12 Jahren
(AGTB 5-12 — Working memory tasks to test children at the
age of 5 till 12). Praxis der Kinderpsychologie und Kinderpsy-
chiatrie, 62(5):1–389, 2013.
[67] ISO/TC 159/SC 4 Ergonomics of human-system interac-
tion. Part 210: Human- centred design for interactive sys-
tems. In Ergonomics of human-system interaction, vol-
ume 1, page 32. International Organization for Standardiza-
tion (ISO), Brussels, 2010.
[68] ISO/TC 159/SC 4 Ergonomics of human-system interaction.
ISO 9241-11, Ergonomics of human-system interaction —
Part 11: Usability: Definitions and concepts, 2018.
[69] A. Jain and D. Zongker. Feature selection: evaluation, appli-
cation, and small sample performance. IEEE Transactions on
Pattern Analysis and Machine Intelligence, 19(2):153–158,
1997.
[70] H. Jansen, G. Mannhaupt, H. Marx, and H. Skowronek.
BISC - Bielefelder Screening zur Früherkennung von Lese-
Rechtschreibschwierigkeiten (Early detection with the Biele-
felder screening of reading and spelling difficulties). Hogrefe,
Verlag für Psychologie, Göttingen, 2002.
[71] D. J. Johnson. Persistent auditory disorders in young
dyslexic adults. Bulletin of the Orton Society, 30(1):268–276,
jan 1980.
[72] L. M. Justice, W.-Y. Ahn, and J. A. R. Logan. Identifying
Children With Clinical Language Disorder: An Application of
162
Machine-Learning Classification. Journal of Learning Disabil-
ities, pages 1–15, 2019.
[73] I. Kecskes and T. Papp. Foreign Language and Mother
Tongue. Psychology Press, New York, 1 edition, jun 2000.
[74] B. Kitchenham and P. Brereton. A systematic review of sys-
tematic review process research in software engineering. In-
formation and Software Technology, 55(12):2049–2075, dec
2013.
[75] F. Kyle, J. Kujala, U. Richardson, H. Lyytinen, and
U. Goswami. Assessing the Effectiveness of Two Theoreti-
cally Motivated Computer-Assisted Reading Interventions in
the United Kingdom: GG Rime and GG Phoneme. Reading
Research Quarterly, 48(1):61–76, 2013.
[76] F. Laamarti, M. Eid, and A. El Saddik. Review Article
An Overview of Serious Games. International Journal of
Computer Games Technology, 2014(15 October 2014):1–15,
2014.
[77] A. Le Floch and G. Ropars. Left–right asymmetry of the
Maxwell spot centroids in adults without and with dyslexia.
Proceedings of the Royal Society B: Biological Sciences,
284(1865):20171380, oct 2017.
[78] V. Levenshtein. Binary codes capable of correctingspurious
insertions and deletions of ones. Problems of Information
Transmission, 1:8–17, 1965.
[79] Lexercise. Dyslexia Test - Online from Lexercise. http://
www.lexercise.com/tests/dyslexia-test, 2016. [Online;
accessed 18-September-2017].
[80] H. Lyytinen, J. Erskine, J. Hämäläinen, M. Torppa, and
M. Ronimus. Dyslexia - Early Identification and Prevention:
163
Highlights from the Jyväskylä Longitudinal Study of Dyslexia.
Current Developmental Disorders Reports, 2(4):330–338,
dec 2015.
[81] C. Maehler and K. Schuchardt. Working memory in chil-
dren with specific learning disorders and/or attention deficits.
Learning and Individual Differences, 49:341–347, 2016.
[82] C. Männel, G. Schaadt, F. K. Illner, E. van der Meer, and
A. D. Friederici. Phonological abilities in literacy-impaired
children: Brain potentials reveal deficient phoneme discrimi-
nation, but intact prosodic processing. Developmental Cog-
nitive Neuroscience, 23:14–25, 2016.
[83] G. Mioni, A. Capodieci, V. Biffi, F. Porcelli, and C. Cornoldi.
Difficulties of children with symptoms of attention-
deficit/hyperactivity disorder in processing temporal in-
formation concerning everyday life events. Journal of
Experimental Child Psychology, 182:86–101, jun 2019.
[84] A. Mora, D. Riera, C. Gonzalez, and J. Arnedo-Moreno. A
Literature Review of Gamification Design Frameworks. In
VS-Games 2015 - 7th International Conference on Games
and Virtual Worlds for Serious Applications, 2015.
[85] C. Moritz, S. Yampolsky, G. Papadelis, J. Thomson, and
M. Wolf. Links between early rhythm skills, musical train-
ing, and phonological awareness. Reading and Writing,
26(5):739–769, 2013.
[86] Nessy. Dyslexia screening - Nessy UK. https://www.nessy.
com/uk/product/dyslexia-screening/, 2011. [Online; ac-
cessed 18-September-2017].
[87] R. I. Nicolson and A. J. Fawcett. Comparison of deficits in
cognitive and motor skills among children with dyslexia. An-
nals of Dyslexia, 44(1):147–164, 1994.
164
[88] Niedersächsisches Vorschrifteninformationssystem.
VORIS Kultusministerium | 25b - 81402 | Verwal-
tungsvorschrift (Niedersachsen) | Umfragen und Erhe-
bungen in Schulen | i. d. F. v. 01.12.2015 | gültig ab
01.12.2015 | gültig bis 31.12.2019 (VORIS Ministry of
Education and Cultural Affairs | 25b - 81402 | Adminis-
trative regulations (Lower Saxony) | Polls and surveys
in schools | in the version of 01.12.2015 | valid from
01.12.2015 | valid until 31.12.2019). http://www.nds-
voris.de/jportal/?quelle=jlink&query=VVND-224100-
MK-20140101-SF&psml=bsvorisprod.psml&max=true. [On-
line, accessed 13-June-2019].
[89] J. Nielsen. Why You Only Need to Test with 5 Users. Jakob
Nielsens Alertbox, 19(September 23):1–4, 2000. [Online, ac-
cessed 11-July-2019].
[90] J. Nijakowska. Dyslexia in the foreign language classroom.
Multilingual Matters, 2010.
[91] D. Norman. The Design of Everyday Things: Revised and
Expanded Edition. Basic Books, New York, nov 2013.
[92] K. Overy. Dyslexia, Temporal Processing and Music: The
Potential of Music as an Early Learning Aid for Dyslexic Chil-
dren. Psychology of Music, 28(2):218–229, oct 2000.
[93] O. Ozernov-Palchik, M. Gonzalez, L. Hillyer, J. Dief-
fenbach, J. Gabrieli, and N. Gaab. Copy of Early
Literacy Assessments. https://docs.google.com/
spreadsheets/d/16m40o49LZ_9wZI9VPAxhHFlATvhSM1mm-
0oGr48jFfo/edit#gid=734901246, 2019. [Online, accessed
06-June-2019].
[94] E. Paulesu, L. Danelli, and M. Berlingeri. Reading the
dyslexic brain: multiple dysfunctional routes revealed by a
165
new meta-analysis of PET and fMRI activation studies. Fron-
tiers in human neuroscience, 8:830, 2014.
[95] P. M. Paz-Alonso, M. Oliver, G. Lerma-Usabiaga,
C. Caballero-Gaudes, I. Quiñones, P. Suárez-Coalla,
J. A. Duñabeitia, F. Cuetos, and M. Carreiras. Neural cor-
relates of phonological, orthographic and semantic reading
processing in dyslexia. NeuroImage. Clinical, 20:433–447,
2018.
[96] K. Peffers, T. Tuunanen, M. A. Rothenberger, and S. Chat-
terjee. A Design Science Research Methodology for Informa-
tion Systems Research. Journal of Management Information
Systems, 24(8):45–78, 2007.
[97] Y. Petscher, H. Fien, C. Stanley, B. Gearin, J. M. Fletcher,
Y. Petscher, C. Stanley, B. Gearin, N. Gaab, J. M. Fletcher,
and &. Johnson. Screening for Dyslexia. Technical report,
National Center on Improving Literacy, 2019. [Online, ac-
cessed 06-June-2019].
[98] A. Poole, F. Zulkernine, and C. Aylward. Lexa: A tool for
detecting dyslexia through auditory processing. 2017 IEEE
Symposium Series on Computational Intelligence, SSCI
2017 - Proceedings, 2018-January:1–5, 2018.
[99] R. F. Port. Meter and speech. Journal of Phonetics, 31:599–
611, 2003.
[100] Precision Learning Center. App Development. http://www.
precisionlearningcenter.org/research-app.html. [On-
line, accessed 06-June-2019].
[101] Precision Learning Center. School Partnership.
http://www.precisionlearningcenter.org/school-
partnership.html. [Online, accessed 06-June-2019].
166
[102] A. Protopapas, A. Fakou, S. Drakopoulou, C. Skaloumbakas,
and A. Mouzaki. What do spelling errors tell us? Classifica-
tion and analysis of errors made by Greek schoolchildren with
and without dyslexia. Reading and Writing, 26(5):615–646,
may 2013.
[103] M. Ptok, K. Berendes, S. Gottal, B. Grabherr, J. Schneeberg,
and M. Wittler. Lese-Rechtschreib-Störung: Die Bedeutung
der phonologischen Informationsverarbeitung für den Schrift-
spracherwerb (Dyslexia: The meaning of phonological infor-
mation processing for the acquisition of written language).
Hno, 55(9):737–748, 2007.
[104] K. Pugh and L. Verhoeven. Introduction to This Special Issue:
Dyslexia Across Languages and Writing Systems. Scientific
Studies of Reading, 22(1):1–6, 2018.
[105] M. Rauschenberger. Entwicklung von Designentwürfen zur
Unterstützung von Hafenmanövern für Lotsen mittels der Hi-
erarchischen Aufgabenanalyse. (Deriving designs in harbour
manoeuvre for harbour pilots with the hierarchical task anal-
ysis.), 2015.
[106] M. Rauschenberger. DysMusic: Detecting Dyslexia by Web-
based Games with Music Elements. In The Web for All
conference Addressing information barriers – W4A’16, Mon-
treal,Canada, 2016. ACM Press.
[107] M. Rauschenberger, S. Füchsel, L. Rello, C. Bayarri, and
A. Gòrriz. A game to target the spelling of German children
with dyslexia. In Proceedings of the 17th international ACM
SIGACCESS conference on Computers & accessibility - AS-
SETS ’15, Lisbon, Portugal, 2015.
[108] M. Rauschenberger, S. Füchsel, L. Rello, C. Bayarri, and
J. Thomaschewski. Exercises for German-speaking children
167
with dyslexia. In Human-Computer Interaction–INTERACT
2015, pages 445–452, Bamberg, Germany, 2015.
[109] M. Rauschenberger, S. Füchsel, L. Rello, and
J. Thomaschewski. DysList German resource: A language
resource of German errors written by children with dyslexia.
https://zenodo.org/record/809801#.XOVWRFMzYWo, 2017.
[Online, accessed 06-June-2019].
[110] M. Rauschenberger, A. Hinderks, and J. Thomaschewski.
Benutzererlebnis bei Unternehmenssoftware: Ein Praxis-
bericht über die Umsetzung attraktiver Unternehmenssoft-
ware. (Enterprise Software User Experience: A real-world
report on how enterprise software can be made attractive.).
In Usability Professionals Konferenz 2011, volume 1, pages
154–158. German UPA e.V., Stuttgart, 2011.
[111] M. Rauschenberger, C. Lins, N. Rousselle, S. Fudickar, and
A. Hain. A Tablet Puzzle to Target Dyslexia Screening in Pre-
Readers. In Proceedings of the 5th EAI International Confer-
ence on Smart Objects and Technologies for Social Good -
GOODTECHS, pages n–n, Valencia, 2019.
[112] M. Rauschenberger, S. Olschner, M. P. Cota, M. Schrepp,
and J. Thomaschewski. Measurement of user experience:
A Spanish Language Version of the User Experience Ques-
tionnaire (UEQ). In A. Rocha, J. A. CalvoManzano, L. P.
Reis, and M. P. Cota, editors, Sistemas Y Tecnologias De
Informacion, pages 471–476, Madrid,Spain, 2012.
[113] M. Rauschenberger, L. Rello, and R. Baeza-Yates. A Tablet
Game to Target Dyslexia Screening in Pre-readers. In Mo-
bileHCI’18, pages 306–312, Barcelona, 2018. ACM Press.
[114] M. Rauschenberger, L. Rello, and R. Baeza-Yates. Tech-
nologies for Dyslexia. In Y. Yesilada and S. Harper, editors,
168
Web Accessibility Book, volume 1, pages 603–627. Springer-
Verlag London, London, 2 edition, 2019.
[115] M. Rauschenberger, L. Rello, R. Baeza-Yates, and J. P.
Bigham. Towards language independent detection of
dyslexia with a web-based game. In W4A ’18: The Internet
of Accessible Things, pages 4–6, Lyon, France, 2018. ACM.
[116] M. Rauschenberger, L. Rello, R. Baeza-Yates, E. Gomez,
and J. P. Bigham. Supplement:DysMusicMusicalElements:
Towards the Prediction of Dyslexia by a Web-based Game
with Musical Elements. https://doi.org/10.5281/zenodo.
809783, June 2017. [Online, accessed 09-June-2019].
[117] M. Rauschenberger, L. Rello, R. Baeza-Yates, E. Gomez,
and J. P. Bigham. Towards the Prediction of Dyslexia by a
Web-based Game with Musical Elements. In The Web for All
conference Addressing information barriers – W4A’17, pages
4–7, Perth, Western Australia, 2017. ACM Press.
[118] M. Rauschenberger, L. Rello, S. Füchsel, and
J. Thomaschewski. A language resource of German
errors written by children with dyslexia. In Proceedings of
the Tenth International Conference on Language Resources
and Evaluation (LREC 2016), Paris, France, may 2016.
European Language Resources Association (ELRA).
[119] M. Rauschenberger, M. Schrepp, M. P. Cota, S. Olschner,
and J. Thomaschewski. Efficient Measurement of the User
Experience of Interactive Products. How to use the User
Experience Questionnaire (UEQ). Example: Spanish Lan-
guage. International Journal of Artificial Intelligence and In-
teractive Multimedia (IJIMAI), 2(1):39–45, 2013.
[120] M. Rauschenberger, M. Schrepp, and J. Thomaschewski.
User Experience mit Fragebögen messen–Durchführung
169
und Auswertung am Beispiel des UEQ (Measuring User Ex-
perience with Questionnaires–Execution and Evaluation us-
ing the Example of the UEQ). In Usability Professionals Kon-
ferenz 2013, pages 72–76, 2013.
[121] M. Rauschenberger, A. Willems, M. Ternieden, and
J. Thomaschewski. Gamification and learning environ-
ments — protocol for the systematic literature review (proto-
col v1.3). https://doi.org/10.5281/zenodo.3257277, May
2018. [Online, accessed 04.-Juli-2019].
[122] M. Rauschenberger, A. Willems, M. Ternieden, and
J. Thomaschewski. Towards the use of gamification frame-
works in learning environments. Journal of Interactive Learn-
ing Research, 30(2), 2019.
[123] L. Rello. DysWebxia: A Text Accessibility Model for Peo-
ple with Dyslexia Luz Rello. PhD thesis, Universtat Pompeu
Fabra, 2014.
[124] L. Rello and R. Baeza-Yates. Good fonts for dyslexia. Pro-
ceedings of the 15th International ACM SIGACCESS Con-
ference on Computers and Accessibility, page 14, 2013.
[125] L. Rello, R. Baeza-Yates, A. Ali, J. P. Bigham, and M. Serra.
Predicting risk of dyslexia with an online gamified test. arXiv
preprint arXiv:1906.03168, V.1:1–13, jun 2019.
[126] L. Rello, R. Baeza-Yates, and J. Llisterri. A resource of errors
written in Spanish by people with dyslexia and its linguistic,
phonetic and visual analysis. Language Resources and Eval-
uation, 51(2):1–30, feb 2016.
[127] L. Rello, M. Ballesteros, A. Ali, M. Serra, D. Alarcón, and
J. P. Bigham. Dytective: Diagnosing Risk of Dyslexia with
a Game. In Pervasive Health 2016, pages 89–96, Cancun,
Mexico, may 2016. ACM Press.
170
[128] L. Rello, C. Bayarri, Y. Otal, and M. Pielot. A computer-based
method to improve the spelling of children with dyslexia. In
Proceedings of the 16th international ACM SIGACCESS con-
ference on Computers & accessibility - ASSETS ’14, AS-
SETS ’14, pages 153–160, New York, USA, 2014. ACM
Press.
[129] L. Rello, E. Romero, M. Rauschenberger, A. Ali, K. Williams,
J. P. Bigham, and N. C. White. Screening Dyslexia for English
Using HCI Measures and Machine Learning. In Proceedings
of the 2018 International Conference on Digital Health - DH
’18, pages 80–84, New York, New York, USA, 2018. ACM
Press.
[130] H. Remschmidt and S. von Aster. Kinder- und Jugendpsychi-
atrie: Eine praktische Einführung (Child and Adolescent Psy-
chiatry: A Practical Introduction). Thieme, Stuttgart [u.a.], 4.,
edition edition, 2005.
[131] J. W. Rice. The Gamification of Learning and Instruction. In-
ternational Journal of Gaming and Computer-Mediated Sim-
ulations, 4(4):81–83, 2013.
[132] A. D. Ritzhaupt, N. D. Poling, C. A. Frey, and M. C. Johnson.
A Synthesis on Digital Games in Education: What the Re-
search Literature Says from 2000 to 2010. Jl. of Interactive
Learning Research, 25(2):263–282, 2014.
[133] E. J. Rolka and M. J. Silverman. A systematic review of music
and dyslexia. Arts in Psychotherapy, 46:24–32, 2015.
[134] R. Rouse. Game Design: Theory and Practice, Second Edi-
tion: Theory and Practice, Second Edition. Wordware Pub-
lishing, Inc., 2004.
[135] C. Rowland and M. Charlier. User Experience Design for the
Internet of Things. O’Reilly Media, Inc., 2015.
171
[136] B. Sawyer. The ”Serious Games” Landscape. Presented
at the Instructional & Research Technology Symposium for
Arts, Humanities and Social Sciences, Camden, USA., 2007.
[137] J. Schell. Design Outside the box: DICE 2010. Dice Summit,
2010.
[138] G. Schulte-Körne. Diagnostik und Therapie der Lese-
Rechtscheib-Störung (The prevention, diagnosis, and treat-
ment of dyslexia). Deutsches Ärzteblatt international,
107(41):718–727, 2010.
[139] G. Schulte-Körne, W. Deimel, M. Jungermann, and H. Rem-
schmidt. Nachuntersuchung einer Stichprobe von lese-
rechtschreibgestörten Kindern im Erwachsenenalter (Follow-
up examination of a sample of children with reading and
spelling difficulties in adulthood). Zeitschrift für Kinder- und
Jugendpsychiatrie und Psychotherapie, 31(4):267–276, nov
2003.
[140] G. Schulte-Körne, W. Deimel, K. Müller, C. Gutenbrunner,
and H. Remschmidt. Familial Aggregation of Spelling Disabil-
ity. Journal of Child Psychology and Psychiatry, 37(7):817–
822, oct 1996.
[141] Scikit-learn. 3.1. Cross-validation: evaluating estimator per-
formance. https://scikit-learn.org/stable/modules/
cross_validation.html, 2019. [Online, accessed 17-June-
2019].
[142] Scikit-learn. 3.3. Model evaluation: quantifying the quality of
predictions. https://scikit-learn.org/stable/modules/
model_evaluation.html#scoring-parameter, 2019. [On-
line, accessed 23-July-2019].
172
[143] Scikit-learn Developers. Scikit-learn Documentation. https:
//scikit-learn.org/stable/documentation.html. [On-
line, accessed 20-June-2019].
[144] K. Seaborn and D. I. Fels. Gamification in theory and action:
A survey. International Journal of Human Computer Studies,
74:14–31, 2015.
[145] P. H. K. Seymour, M. Aro, and J. M. Erskine. Foundation
literacy acquisition in European orthographies. British journal
of psychology (London, England : 1953), 94(Pt 2):143–74,
may 2003.
[146] D. Sheskin. Handbook of Parametric and Nonparametric Sta-
tistical Procedures. Boca Raton: CRC, page 972, 2000.
[147] H. A. Simon. The sciences of the artificial, (third edition),
volume 3. MIT Press, 1997.
[148] C. Steinbrink and T. Lachmann. Lese-Rechtschreibstörung
(Dyslexia). Springer Berlin Heidelberg, 2014.
[149] P. Tallal. Improving language and literacy is a matter of time.
Nature reviews. Neuroscience, 5(9):721–728, 2004.
[150] P. Tamboer, H. C. M. Vorst, S. Ghebreab, and H. S. Scholte.
Machine learning and dyslexia: Classification of individual
structural neuro-imaging scans of students with and without
dyslexia. NeuroImage. Clinical, 11:508–514, 2016.
[151] UCSF Dyslexia Center. AppRISE for Preschoolers to
Kindergartners. https://dyslexia.ucsf.edu/content/
iscreener-preschoolers-kindergartners. [Online, ac-
cessed 06-June-2019].
[152] L. Van den Audenaeren, V. Celis, V. Vanden Abeele,
L. Geurts, J. Husson, P. Ghesquière, J. Wouters, L. Loyez,
173
and A. Goeleven. DYSL-X: Design of a tablet game for
early risk detection of dyslexia in preschoolers. In Games for
Health, pages 257–266. Springer Fachmedien Wiesbaden,
Wiesbaden, 2013.
[153] V. Vanden Abeele, J. Wouters, P. Ghesquière, A. Goeleven,
and L. Geurts. Game-based Assessment of Psycho-acoustic
Thresholds. In Proceedings of the 2015 Annual Sympo-
sium on Computer-Human Interaction in Play - CHI PLAY
’15, pages 331–341, New York, New York, USA, 2015. ACM
Press.
[154] S. Varma and R. Simon. Bias in error estimation when us-
ing cross-validation for model selection. BMC Bioinformatics,
7:91, feb 2006.
[155] F. R. Vellutino, J. M. Fletcher, M. J. Snowling, and D. M.
Scanlon. Specific reading disability (dyslexia): what have
we learned in the past four decades? Journal of Child Psy-
chology and Psychiatry, 45(1):2–40, jan 2004.
[156] T. R. Vidyasagar and K. Pammer. Dyslexia: a deficit in visuo-
spatial attention, not in phonological processing. Trends in
Cognitive Sciences, 14(2):57–63, 2010.
[157] T. Wallbaum, M. Rauschenberger, J. Timmermann,
W. Heuten, and S. C. Boll. Exploring Social Aware-
ness. In Extended Abstracts of the 2018 CHI Conference
on Human Factors in Computing Systems - CHI ’18, pages
1–10, New York, New York, USA, 2018. ACM Press.
[158] J. G. Walls, G. R. Widmeyer, and O. A. El Sawy. Building an
information system design theory for vigilant EIS. Information
Systems Research, 3(1):36–59, 1992.
[159] J. M. Ward-Lonergan and J. K. Duthie. The State of Dyslexia:
Recent Legislation and Guidelines for Serving School-Age
174
Children and Adolescents With Dyslexia. Language Speech
and Hearing Services in Schools, 49(4):810, 2018.
[160] K. Werbach and D. Hunter. For the win : how game thinking
can revolutionize your business. Wharton, 2012.
[161] K. Werbach and D. Hunter. The gamification toolkit : dynam-
ics, mechanics, and components for the win. Wharton Digital
Press, Philadelphia :, 2015.
[162] Wikipedia. Memory (Spiel) (Memory Game). https://de.
wikipedia.org/wiki/Memory_(Spiel), 2019. [Online, ac-
cessed 22-Mai-2018].
[163] World Health Organization. International Statistical Classifi-
cation of Diseases and Related Health Problems 10th Revi-
sion (ICD). World Health Organization, 2010.
[164] World Health Organization. International Classification of
Diseases 11th Revision. World Health Organization, 2019.
[165] C. J. Yuskaitis, M. Parviz, P. Loui, C. Y. Wan, and P. L. Pearl.
Neural Mechanisms Underlying Musical Pitch Perception
and Clinical Applications Including Developmental Dyslexia.
Current neurology and neuroscience reports, 15(8):51, 2015.
[166] G. Zichermann and C. Cunningham. Gamification by Design.
O’Reilly books, Canada, 2011.
[167] J. C. Ziegler, C. Perry, A. Ma-Wyatt, D. Ladner, and
G. Schulte-Körne. Developmental dyslexia in different lan-
guages: language-specific or universal? Journal of experi-
mental child psychology, 86(3):169–93, nov 2003.
175
... Therefore, researchers have to make the best of a limited data set and avoid over-fitting, being aware of issues such as small data, imbalanced data, variance, biases, heterogeneity of participants, or evaluation metrics. We address the main criteria to avoid overfitting and taking care of imbalanced data sets related to health from a previous research project with different small data sets related to early and universal screening of dyslexia [28,29,35]. Our main contribution is a list of nine criteria when exploring small imbalanced data for machine learning predictions. ...
... The four green boxes match the four HCD phases (see Figure 2, yellow boxes). Next, we describe each of the six DSRM steps following Figure 2 with an example from our previous research on early screening of dyslexia with a web game using machine learning [28,29,35]. ...
... Because of the web implementation technique we used, a double click on a web application generally zooms in, which was not intended in a tablet. Therefore, we controlled the layout setting for mobile devices to avoid the zoom-effect on tablets, which caused interruptions during the game [28]. The evaluation requires the collection of remote data with the experimental design to use the dependent measures for statistical analysis and prediction with machine learning classifiers. ...
Conference Paper
Full-text available
When discussing interpretable machine learning results, researchers need to compare results and reflect on reliable results, especially for health-related data. The reason is the negative impact of wrong results on a person, such as in missing early screening of dyslexia or wrong prediction of cancer. We present nine criteria that help avoiding over-fitting and biased interpretation of results when having small imbalanced data related to health. We present a use case of early screening of dyslexia with an imbalanced data set using machine learning classification to explain design decisions and discuss issues for further research.
... In this work we address the main criteria to avoid overfitting and taking care of imbalanced data sets related to health from a previous research experience with different small data sets related to early and universal screening of dyslexia [32,34,39]. We mean by universal screening of dyslexia a language-independent screening. ...
... The four green boxes match the four HCD phases (see Figure 2, yellow boxes). Next, we describe each of the six DSRM steps following Figure 2 with an example from our previous research on early screening of dyslexia with a web game using machine learning [32,34,39]. ...
... Because of the web implementation technique used, a double click on a web application to zoom in, which was not good in a tablet. Therefore, we controlled the layout setting for mobile devices to avoid the zoom-effect on tablets, which caused interruptions during the game [32]. The evaluation requires the collection of remote data with the experimental design to use the dependent measures for statistical analysis and prediction with machine learning classifiers. ...
Article
Full-text available
When discussing interpretable machine learning results, researchers need to compare them and check for reliability, especially for health-related data. The reason is the negative impact of wrong results on a person, such as in wrong prediction of cancer, incorrect assessment of the COVID-19 pandemic situation, or missing early screening of dyslexia. Often only small data exists for these complex interdisciplinary research projects. Hence, it is essential that this type of research understands different methodologies and mindsets such as the Design Science Methodology, Human-Centered Design or Data Science approaches to ensure interpretable and reliable results. Therefore, we present various recommendations and design considerations for experiments that help to avoid over-fitting and biased interpretation of results when having small imbalanced data related to health. We also present two very different use cases: early screening of dyslexia and event prediction in multiple sclerosis.
... In general, people with dyslexia appears very intelligent, but they are unable to read, write and spell in a correct way. Children with dyslexia have difficulties in learning how to read and write [2]. Rauschenberger et al. ...
Chapter
Augmented Reality Game for based learning has been enhanced the learning experienced and developed the knowledge and skills of the user. The project methodology used in this study is the game development life cycle (GDLC). It includes initiation, pre-production, production, testing, beta testing and release. The purpose of this project is to produce video games for children with dyslexia who have visual and auditory learning difficulties related to memory, time management, speed processing, organization, organization and planning. The objective of this product was to develop Augmented Reality games for dyslexia students, second was to develop Reality-based games on reading, spelling and numbers for dyslexia students using the Unity Game Engine, the third was to test the appropriateness of learning based on reading, spelling games and numbers for dyslexic students. The number of user targets used to test this product are dyslexia students, teachers who teach dyslexia students, expert programmers and designer games and evaluators from the eLearning Carnival & Conference (eLCC 2019). The result of this DyslexiAR learning game is that dyslexic students have a better understanding of learning to read, spell and learn numbers.
Article
Full-text available
Dyslexia is a specific learning disorder related to school failure. Detection is both crucial and challenging, especially in languages with transparent orthographies, such as Spanish. To make detecting dyslexia easier, we designed an online gamified test and a predictive machine learning model. In a study with more than 3,600 participants, our model correctly detected over 80% of the participants with dyslexia. To check the robustness of the method we tested our method using a new data set with over 1,300 participants with age customized tests in a different environment -a tablet instead of a desktop computer- reaching a recall of over 78% for the class with dyslexia for children 12 years old or older. Our work shows that dyslexia can be screened using a machine learning approach. An online screening tool in Spanish based on our methods has already been used by more than 200,000 people.
Data
Full-text available
Gamification is an established concept to apply game elements in different contexts to engage and motivate users. Gamification has been successfully used in various use cases and application as well as general frameworks have been established. To support the design of learning environments in order to improve students’ engagement and motivation, applying the concept of gamification is beneficial to emphasize engagement and motivation. This article presents the results of a literature review performed to examine the systematic use of gamification in learning environments. Therefore, ten frameworks matching the search criteria of the systematic literature review are analyzed. The results show that game elements are used heterogeneously and only game elements related to the dynamics, emotions and progression are preferred in learning environments. Because of the diversity of game elements used in the applications analyzed, no reliable standard can be given to design gamified learning environments with game elements. Instead, we provide a short overview of how game elements have been applied in the different application context. As gamification is a relatively young field of research, future work is necessary to give a comprehensive assessment of the topic.
Article
Gamification is an established concept to apply game elements in different contexts to engage and motivate users. Gamification has been successfully used in various use cases and applications as well as general frameworks have been established. To support the design of learning environments in order to improve students' engagement and motivation, applying the concept of gamification is beneficial to emphasize engagement and motivation. This article presents the results of a literature review performed to examine the systematic use of gamification in learning environments. Therefore, ten frameworks matching the search criteria of the systematic literature review are analyzed. The results show that game elements are used heterogeneously and game elements related to the emotions and progression are preferred in learning environments. Because of the diversity of game elements used in the applications analyzed, no reliable standard can be given to design gamified learning environments with game elements. Instead, we provide a short overview of how game elements have been applied in the different application context. As gamification is a relatively young field of research, future work is necessary to give a comprehensive assessment of the topic. 148 Rauschenberger, Willems, Ternieden, and Thomaschewski https://www.learntechlib.org/primary/p/181283/
Conference Paper
Dyslexia is a widespread specific learning disorder, which can have a particularly negative influence on the learning success of children. Early detection of dyslexia is the foundation for early intervention, which is the key to reduce the adverse effects of dyslexia, e.g., bad school grades. In this paper, we present the prototype of a puzzle app, which we explicitly designed with the human-centred design (HCD) process for dyslexia screening in pre-reader using new indicators related to motor skills to ensure users needs for the data collection to apply machine learning prediction. The app records the telemetry of the gaming sequence in order to derive future statements about the prevalence of dyslexia based on the telemetry data. The high-fidelity prototype was evaluated with a five-user test usability study with five German-speaking child-parent pairs. The results show how young children and parents are interacting with new games, and how new applications (web and mobile technologies) which are used for online experiments, could be developed. The usability of the prototype is suitable for the target group with only minor limitations. CCS
Chapter
Nowadays, being excluded from the web is a huge disadvantage. People with dyslexia have, despite their general intelligence, difficulties for reading and writing through their whole life. Therefore, web technologies can help people with dyslexia to improve their reading and writing experience on the web. This chapter introduces the main technologies and many examples of tools that support a person with dyslexia in processing information on the web, either in assistive applications for reading and writing as well as using web applications/games for dyslexia screening and intervention.
Article
In this study, we identified child- and family-level characteristics most strongly associated with clinical identification of language disorder for preschool-aged children. We used machine learning to identify variables that best classified children receiving therapy for language disorder among a sample of 483 3- to 5-year-old children (54% affected). Using a dichotomous outcome based on receipt of language therapy, we applied the least absolute shrinkage and selection operator (LASSO) classification approach to a range of background data available on the children, including teacher and caregiver ratings of communication and social skills. The sample was randomly split into a training (67% of children) and test sample (33% of children) to examine out-of-sample classification accuracy. The full model had excellent classification accuracy based on area under the curve (AUC) of .87 and .85 on the training and test sets, respectively, when utilizing all available background data. Variables most strongly contributing to accurate classification of language-therapy receipt were cognitive impairment, age, gender, and teacher- and parent-reported communication, social, and literacy skills. Use of machine-learning approaches to classify children receiving language services in school settings may provide a valuable approach for identifying those factors that best differentiate children with and without language disorders from a clinical perspective.
Conference Paper
Designing user studies in the interactive information retrieval (IIR) paradigm on people with impairments may sometimes require different methodological considerations than for other users. Consequently, there may be a tension between what the community regards as being a rigorous methodology against what researchers can do ethically with their users. This paper discusses issues to consider when designing IIR studies involving people with dyslexia, such as sampling, informed consent and data collection. The conclusion is that conducting user studies on participants with dyslexia requires special considerations at all stages of the experimental design. The purpose of this paper is to raise awareness and understanding in the research community about experimental methods involving users with dyslexia, and addresses researchers, as well as editors and reviewers. Several of the issues raised do not only apply to people with dyslexia, but have implications when researching other groups, for instance elderly people and users with learning, cognitive, sensory or motor impairments.
Article
This paper aims to provide an overview and some insight into what is known about screening for dyslexia. Section I provides a brief overview of “what is dyslexia” and the importance of screening for dyslexia risk. In Section II of this paper, we discuss the neurological and behavioral aspects relevant to dyslexia as well as the emerging research in both areas. Section III provides a robust presentation of viewpoints and considerations for best practices in behavioral screening. Section IV provides a brief overview of key statistical considerations one should consider when evaluating a screener with a companion technical report provided in Appendix A. Section V concludes with a checklist to support teachers, school psychologists, and school-based assessment teams in evaluating and choosing universal screeners.
Conference Paper
This work aims to design and develop a web application for reading and attentional evaluation in schooling children using a standardized tests battery. The application allows the automatic and immediate calculation of child's scores by inserting times and errors committed by the subject. The obtained scores are needed to establish whether the participant's performance is in deficit or not (i.e., Z-score).