School Start Screening Tool

Abstract

This paper concentrates on the implementation of a new screening method in Austrian schools to assess children and find out if they have any special educational needs in educational areas like phonology, counting, knowledge of letters and numbers and others. The underlying structure of the assignments was developed by developmental psychologists from the University of Graz and the University of Vienna and then implemented as a mobile application for Android tablets. For testing and evaluation purposes, the app's prototype was first used voluntary by multiple Austrian schools. Furthermore, the mobile application had to be fully functional when offline as well as a central web application had to be developed to give the developmental psychologists access to the screening results. However, the main goal was to develop the application in a way that especially children without sufficient skills in reading and writing are able to understand the assignment's tasks and to perform input.

Full text

School Start Screening Tool
Paul Krassnig
Graz University of Technology
Austria
krassnig@student.tugraz.at
Markus Ebner
Educational Technology
Graz University of Technology
Austria
markus.ebner@tugraz.at
Martin Ebner
Educational Technology
Graz University of Technology
Austria
martin.ebner@tugraz.at
Abstract: This paper concentrates on the implementation of a new screening method in Austrian
schools to assess children and find out if they have any special educational needs in educational
areas like phonology, counting, knowledge of letters and numbers and others. The underlying
structure of the assignments was developed by developmental psychologists from the University of
Graz and the University of Vienna and then implemented as a mobile application for Android
tablets. For testing and evaluation purposes, the app’s prototype was first used voluntary by
multiple Austrian schools. Furthermore, the mobile application had to be fully functional when
offline as well as a central web application had to be developed to give the developmental
psychologists access to the screening results. However, the main goal was to develop the
application in a way that especially children without sufficient skills in reading and writing are able
to understand the assignment’s tasks and to perform input.
Introduction
Already before the introduction of general compulsory schooling, tests were developed to recognize if
children in preschool age are ready for school. One of these tests was the so-called “Gulden-Apple-Test“, which was
performed by placing a penny and an apple in front of a child (Dietrich Rüdiger, 1976). In case the child picked the
penny over the apple, the child was seen as mature enough for school because the child showed a form of cognitive
maturity. However, today there are many different screening methods to assess children in different educational
subjects. These educational subjects include reading, writing, mathematics, motoric and social development, all of
them are important competences for children to be ready for school (Wolfgang Schneider, 2018). To assess the
child’s skills in its cultural competences, including reading, writing and mathematics, there are multiple screenings
to choose from already. Research found out that skills like phonological awareness, vocabulary, syntax and
disclosure are related to the development of reading skills. In German-speaking areas for example, a group reading
test for primary schools, called “Würzburger Leise Leseprobe”, developed by the German Institute of Psychology at
the University of Würzburg, can be used to determine the child’s level of competence in reading. The WLLP tasks
consist out of reading of individual words and selecting the corresponding item out of four images while recording
the child’s reading speed (Marcus Hasselhorn, 2000). A diagnostic screening to assess the children’s mathematical
level of skill is named “Early Numeracy Research Project” (ENRP) or “ElementarMathematische BasisInterview”
(EMBI) for German-speaking countries. The EMBI is based on the ENRP developed by the Department of
Education of the Australian state of Victoria. Both screenings can be used to assess the mathematical skills of
children in preschool age or in first or second grade of school. The screening is done by interviewing every child
individually by using “growth points”, which can only successfully achieved by the children if they solve a task in a
-213-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021
Originally published in: Krassnig, P., Ebner, M. & Ebner, M. (2021). School Start Screening Tool. In T. Bastiaens (Ed.), Proceedings of EdMedia + Innovate
Learning (pp. 213-226). United States: Association for the Advancement of Computing in Education (AACE). Retrieved July 26, 2021
from https://www.learntechlib.org/primary/p/219660/.

specific way (D. Clarke et al., 2002). An example screening in German-speaking countries for assessing the writing
competences of children is called “Deutsche Rechtschreibtests” (DERET), which is available in both, group and
individual test. The test consists out of a continuous text, for which the children have to write a minimum number of
words dependent on their school grade, and a fill-in-the-blank text to identify in which area the children have
problems (Schneider, Marx, and Marcus Hasselhorn, 2008).
The Austrian Federal Minister for Education, Science and Research passed a new reform in the year 2018
to introduce nationwide precisely defined school criteria as the basis for a successful school start. The new reform
should be implemented until the year 2024 and involves various changes in the Austrian educational system. One of
them is the future nationwide usage of a screening to inform parents if their children are ready for school, with the
help of a uniform supportive procedure, called “School Start Screening”. The School Start Screening should assess
the children’s skills that are required for a successful school start. The screening should be available as a paper-
pencil version as well as a digital screening in form of a mobile Android (android.com) application for tablet
computers. This paper describes the development of the mobile application’s prototype and how it was possible to
implement multiple different assignments for children without necessary skills in reading and writing by making use
of a visual language. (Federal Ministry of Education, Science and Research, 2020)
Research Design
Based on the Austrian Federal Minister for Education, Science and Research’s reform, it is planned to
assess the children before they enter the first class, in the first class and in the second class to identify children with
development weaknesses in the first place as well as to track the children’s development. Each screening done with
the help of the mobile application’s prototype took place in multiple Austrian schools which volunteered to be part
of the first screening processes. Before a screening can be performed, it is necessary to obtain the signature of one of
the children’s legal guardians to agree the application’s data and privacy policy because of the saved data’s
sensibility. Each screening has to be performed by the child next to a screening supervisor, who is usually part of the
school’s teaching staff. The screening supervisor is responsible for the whole screening process that includes the
introduction of the screening procedure to the child, the start of the screening as well as to perform some input for
some assignments. Most of the assignments usually can be performed by the child alone, however some assignments
require the help of the screening supervisor, which are referred to as “supervisor assisted assignments” in the
following sections. To indicate these assignments, the app shows a simple screen only with the frame story’s
character in front of the illustrated landscape and plays an audio file pointing out that the next assignment is asked
by the screening supervisor before the start of a supervisor assisted assignment. Furthermore, after all assignments
are done the app automatically calculates the child’s performance and gives feedback if the child has any
educational needs in some of the assessed areas.
Local Database Management of the App
A requirement for the mobile tablet application was to be fully functional when offline. According to the
developmental psychologists, some Austrian schools suffer from insufficient internet connection or do not have any
internet connection at all. This means it was not possible to assume that the app can be connected to the server all
along and the assignment’s data has to be downloaded before a screening could be performed. Therefore, it was
necessary to make use of the mobile application’s local database to store the assignment’s data and data based on the
already finished screenings with the children’s answers. Android offers different possibilities to store data locally
with different features like “app-specific storage” for storing files only accessible for the app itself, “shared storage”
to shared files between apps, “preferences” for simple key value pairs and a local database which enables the
developer to store more structured data. Due to the more complex structure of the assignment’s data, the database
storage was the storage option of choice. By using Android’s Room library ( Android’s Room Library) it was
possible to save the necessary data in an underlying SQLite database by communicating with the provided
abstraction layer on top of the database. The Room library enables the developer to create database migrations to
guarantee the app uses the right database schemas after a database update. In case an application update also
contained changes in the database structure, it was necessary to write the specific database migrations with the
according SQL queries to change the database schema accordingly without losing any data. Room also makes use of
convenience annotations in form of “Data Access Objects” (DAO) and “Entities” that define the underlying data
structure of the application’s objects. Out of the box the library already offers annotation like “@PrimaryKey”,
-214-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

“@ColumnInfo”, etc. to describe characteristics of the object’s database columns and provides functions to insert,
update or delete entries without the need to write SQL queries.
The Central Web Application
Before a screening can be performed, the screening assignments have to be created by the developmental
psychologists. Since it was necessary to make it possible to perform different screenings in different schools, a
central platform with the ability to create different screenings and to assign them to different schools had to be
created. This was done by implementing a web application with access only available to the developmental
psychology teams of the University of Graz and the University of Vienna. The functions of the web application
include:
the creation of schools which perform screenings
the creation of screenings and its assignments
the assignment of screenings to schools
to give access to the already collected screening result data
The web application and its application programming interface (API) were implemented using the “Zend
Framework 3” (framework.zend.com), a PHP framework which is now continued under the name “Laminas Project”
and which was founded by the creators of PHP 3 and PHP 4, Andi Guttmans and Zeev Suraski, in 1997. The data
created through the web application and through the API is stored in a MySQL database by using the Object
Relation Mapper (ORM) library of the “Doctrine Project” (doctrine-project.org). Similar to Android’s Room library,
Doctrine’s ORM makes also use of a Database Abstraction Layer (DBAL). The ORM and DBAL libraries make it
possible to concentrate more on implementing the application’s actual business logic, while the libraries provide
transparent persistence and straightforward database management of PHP objects. The DBAL offers a lightweight
API providing additional features like database manipulations by translating the application’s PHP data objects
(PDO) into SQL queries and by running SQL queries that return PDOs to manage database communication on its
own.
To make it possible for the developmental psychologists to perform scientific calculations on the collected
data to enhance the screening assignments and to develop statistics about the general educational development of
children in Austrian schools, it was necessary to provide the collected data in a form the developmental
psychologists could use it easily for further processing. Since the developmental psychologists usually used Excel
for calculations on similar data it was decided to enable the web application’s users to download the collected data
in form of an Excel file. The creation of the Excel file containing all data of the already done screenings was
implemented using a GitHub project named “PhpSpreadsheet” (github.com/PHPOffice/PhpSpreadsheet). According
to the project’s documentation, the library holds an in-memory representation of the spreadsheet, which means the
processing of the Excel file is limited to the memory limits of the server. Due to the high number of data collected
while the first screening test runs, the server’s memory limits were reached quickly. Therefore, it was necessary to
change the server’s implementation of the Excel file creation. At first all results were saved into one Excel file
which became too big in its size because of multiple hundred lines in each of the file’s multiple Excel sheets. Due to
the urgency to solve the problem and to give the developmental psychologists the ability to download the saved data
again, the Excel file was simply split into multiple files, with each file containing up to 100 screening runs bundled
into a ZIP-file to download.
Concept of Assignments
The underlying concept of each screening assignment was developed by the developmental psychology
team of the University of Graz and the University of Vienna, which included the underlying structure of each
assignment and what data had to be saved for further processing. Each screening consists out of ten different
assignment types and can contain multiple assignments of the same type. The different assignment types part of the
screening are:
Phonological awareness assignment
-215-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

Knowledge of letters assignment
Number recognition assignment
Comparison of numbers assignment
Repeat numbers assignment
Quantity comparison assignment
Quantity knowledge assignment
Arithmetic problems assignment
RAN (Rapid Automatized Naming) assignment
Sorting assignment
As already mentioned, some assignments need the help of the screening supervisor to perform some input
or to communicate the assignment’s task to the child. What assignments belong to these supervisor assisted
assignments and their particularities are described later in the implementation section.
Due to the fact that some assignments make use of a timer or stopwatch, it is necessary for some
assignments to make the children familiar with their task before the assignment starts. These assignments usually
make use of some exercise assignments to be sure the children understood their task before the real assignments
start. The exercise assignments, unlike the real assignments, do not save any data based on the children’s given
answer but give them feedback on the correctness of their answer. Furthermore, the exercise assignments are hard-
coded and are therefore the same for every screening run. In case the assignment makes use of a timer and the child
did not select any answer before the timer of the exercise assignment ran out, the app informs the child to try to
answer the next assignment faster.
Designing an App for Children
Since the app was used by children without necessary skills in reading, writing and mathematics it was
necessary to think about a way how to give instructions to the children, they are able to understand, as well as how
to make it possible for the children to perform inputs to select their answers. There are already multiple scientific
studies when it comes to the development of game-based apps for children. Most studies recommend using audio
files and images or choosing text appropriate to the children’s reading level (Ferreira, Gouin-Vallerand, and Hotte,
2016). Since several screenings had to be carried out with the prototype app in the voluntary schools and several
hundred screenings will usually have to be carried out in every school in the future, it is important to note that the
process of a screening should not take up too much time. Therefore, it was not possible to use simple texts in the app
and give children time to understand them. This led to the decision to primarily use audio and image files to
communicate instructions in a visual language. Furthermore, some assignments require the children to select one of
multiple possible answers containing small strings with one to three characters or answers which contain small
numbers. To entertain the children as good as possible as well as to keep them motivated over the whole screening
process, the app’s graphics were designed by designers of the Technical University of Graz’s Institute of Interactive
Systems and Data Science. The app’s audio files also had to be recorded before starting the actual implementation of
the assignments. These audios were recorded in German by an employee of the University of Graz’s Institute of
Psychology, Developmental Psychology, which also had an education in speech therapy. The recording happened in
a recording studio of the Graz University of Technology’s Educational Technology Unit.
“Poldi” and the Applications Frame Story
It was also important to wrap the assignments into a, for the child interesting, frame story to keep the
children motivated over the whole screening process. By investigating popular educational mobile applications for
iOS and Android, it can be noticed that some of them use a character that leads the user through the application. For
example, the web application as well as the mobile apps of Duolingo (duolingo.com) use a green illustrated bird that
-216-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

motivates the users and gives them instructions. Together with the help of the developmental psychologists a similar
illustrated character was created and named “Poldi”. The developed frame story introduces Poldi, an elf who lives in
a magic world and who has found a treasure map that leads to a secret treasure. Since Poldi is not able to find the
treasure by its own, the elf asks the children for help by solving the different upcoming assignments. For each
completed assignment, the children gain one or multiple keys that are needed to open the treasure at the end of the
screening process. Since it is important to keep the children motivated over the whole screening process, the
children also gain keys if they did select incorrect answers. In case the children are not able to solve an assignment,
they can select a magic wand in the upper right corner of the screen that skips the current assignment. At the start of
the screening, the frame story of Poldi is told to the children and the function of the magic wand is explained. To
limit the number of times the children select the magic wand, Poldi asks them only to click on it if they really do not
know the correct answer. Right after the last assignment is done, the app presents the found treasure to the children
and congratulates them on their successful adventure. Independent on how many assignments were solved correctly
by the child or how often the magic wand was used, the child can open the treasure by clicking on it. After the click
on the treasure happened, fireworks are shown in the background and matching sound effects are played as well as
the treasure will open and the gold coins in the treasure will become visible.
Implementation
When starting the mobile tablet application, a login form is shown. In case the school is already logged in,
the app automatically opens the menu screen. To log into the application, it is necessary to have access to a reliable
internet connection so that the app is able to perform a HTTP request to the server’s API to authenticate the logged
in school. The authentication process is done with the help of JSON Web Tokens, also known as JWT ( jwt.io),
which offers a secure way to transfer data in form of a JSON object by signing the payload with a Message
Authentication Code (MAC) or a public and private key. The passwords are hashed using the PHP function
“password_hash” configured to use the “bcrypt”-Algorithm and are compared to their hashes that are stored in the
database. After the school is successfully authenticated, the app shows the menu screen to the user.
Figure 1. Menu of the tablet application
The application’s menu, shown in figure 1, offers the user to switch to the screening-start-screen, the
screening-update-screen, the screening-result-upload-screen or to log out of the application. To download the latest
assigned screenings as well as for uploading the collected data of the finished screenings, it is also necessary to be
connected to the internet to communicate with the server’s API. Before it is possible to start a screening, at least one
screening has to be stored in the app’s database. In case the app does not have any screenings saved, the app informs
the users that they have to perform a screening update first, by showing a warning in the app’s screening-selection-
screen. To perform a screening update, the users have to click the second menu button with the label
“Screeningupdateanfrage” (German for “screening update request”), which shows some information about the
screening update as well as a button to start the screening update and a progress bar to indicate the progress after a
screening update was started. When a screening update is performed, the app downloads the screenings assigned to
the logged in school and their associated audio and image files and saves them locally.
By selecting the first menu button, with the label “Screening durchführen” (German for “perform a
screening”), the app shows a form in which the screening supervisor has to input the identifier linked to the child.
-217-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

The child’s identifier is preprinted on additional sheets of paper, which have to be collected together with the data
and privacy policy. In case the parents agree the policy, the screening supervisor fills out a form on an additional
sheet of paper to record some background information about the child, for example if the main language spoken at
home is other than German, which is important information for the developmental psychologists. The signed data
and privacy policy and all other important documents were later collected by the psychologists after the screenings
done with the app’s prototype were finished. After the child’s identifier is entered a second time to avoid typing
errors, the app goes to the screening-selection-screen listing all locally saved screenings. By selecting a screening
and clicking the go-ahead button, the app shows an alert view warning the screening supervisor that the screening
starts after clicking the alert view’s go-ahead button.
All collected screening results can be uploaded by the users by clicking the third button in the app’s menu
with the label “Screeningergebnisupload” (German for “Screening result upload”). This will open the screening-
result-upload-screen, which shows a label with information about the screening upload as well as a button which
will trigger the app’s upload function. After all screening results are uploaded successfully, the app deletes the
uploaded results.
Screening Start
Figure 2. Screening start screen
After the alert view’s go-ahead button in the screening-selection-screen is clicked, the screening supervisor
has to place the tablet in front of the child and the screening intro with Poldi starts, shown in figure 2. After the
screening’s start screen becomes visible, the app plays an audio file explaining Poldi’s situation to the children and
asks them to help Poldi finding the treasure by solving the upcoming assignments. Furthermore, the speech bubble
above Poldi’s head changes its content images based on the objects and topics Poldi is talking about, for example an
image of the treasure is presented when Poldi is talking about it. After the audio file is finished, the children have to
click on the way sign to continue, which is indicated by the floating hand above the sign.
For each screening assignment, first the app plays an audio file explaining the child’s task or requests the
screening supervisor to explain the assignment to the child. After the assignment is explained and in case exercise
assignments are part of the assignment, the exercise assignments have to be solved before the real assignment starts.
Independent on how many assignments were successfully solved, one or multiple keys are shown to the child after
all assignments of the same assignment type are done and before the next assignments start.
-218-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

Phonological Awareness Assignment
Figure 3. Phonological Awareness Assignment
The phonological awareness assignment, shown in figure 3, is the first assignment of a screening. It is a
single choice assignment with the child’s goal to select the according image based on the played audio file. Out of
the three different possible answers, the child has to select the item that starts with the same letter as the item named
by the audio file. After the assignment’s audio file is played, the images in the wooden sings are presented one after
another while an according audio file is speaking out loud the name of the object in the image. For example, the
assignment would be “Mouse starts with ‘M’, which word also starts with ‘M’?” which would make the third
possible answer, the mouth, the right answer. In case the children want to hear the audio files again, it is possible to
click on the replay button shown in the speech bubble above Poldi to repeat the audio file.
For the evaluation of the assignment the app saves which answer was selected by the child.
Knowledge of Letters and Number Recognition Assignment
Figure 4. Number recognition assignment
The knowledge of letters and the number recognition assignment are similar in their structure, with their
only difference that the child has to look for the right letter in the knowledge of letters assignment and for the right
number in the number recognition assignment. In figure 4 a screenshot of the implementation of the number
recognition assignment is shown. At the beginning of the assignment, the mole next to Poldi is introduced as Willy
who is looking for his home-molehill. The child has to help Willy by selecting the molehill with the letter or number
named by the assignment’s audio file. After the intro audio file has been played, a hovering hand is displayed over
Willy, instructing the child to click on Willy to start the assignment and play the audio file of the assignment.
For the evaluation of the assignment the app saves which molehill was selected.
-219-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

Comparison of Numbers Assignment
Figure 5. Comparison of numbers assignment
The comparison of numbers assignment is the first assignment type of the screening run, which makes use
of exercise assignments. After the intro audio file is finished and the assignment is explained, the three exercise
assignments are asked one after another. Different to the real assignments, the exercise assignments of this
assignment do not make use of a timer. The ladybugs shown in figure 5 are part of the assignment’s story to
motivate the child to be as fast as possible. Since the ladybugs are hungry, they will eat the leaf if the child is not
fast enough. Due to the fact that the exercise assignments do not use a timer, the ladybugs are not shown for the
exercise assignments. In case the child selected the incorrect answer, the app plays back an audio file requesting the
child to look more carefully until the child selects the right answer. After all three exercise assignments are done, the
real assignments start. The timer for the real assignments is configured to skip the assignment if the child takes
longer than five seconds to select an answer. This also changes the pictures of the leaves to pictures of leaves that
have already been nibbled on by the ladybugs.
For further evaluation, the app saves the selected answers and how long the child took to select an answer
or if the timer did run out before an answer was given.
Repeat Numbers Assignment
Figure 6. Repeat numbers assignment
The following assignment belongs to the supervisor assisted assignments, which means the supervisor has
to be in control of the tablet. After the child is informed by the tablet that the next assignment is asked by the
screening supervisor, the screening supervisor takes the tablet. The children’s goal in the repeat numbers
assignment, shown in figure 6, is to repeat all numbers the audio file is naming in the same order. The assignment’s
audio files only can be played back once, to be sure that the children hear the numbers only a single time.
Furthermore, the assignment uses number sequences consisting out of a minimum of two and a maximum of five
numbers. In the case the child was not able to repeat the number sequence correctly, there is a second chance to
-220-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

repeat another number sequence with the same number of digits. After all number sequences are done, which is the
case if the first or second chance of the last number sequence is repeated correctly or if the first and the second
chance of any number sequence is answered incorrectly, the tablet shows an alert view informing the screening
supervisor to place the tablet again in front of the child. The assignment’s audio file is played back after the
screening supervisor clicks on the button with the label “Audiodateien abspielen” (German for: “play audio files”).
Next to the button, the numbers named by the audio files are shown to help the screening supervisor to compare the
numbers named in the audio file with the numbers named by the child. Depending on the correctness of the child’s
answer, the screening supervisor has to click the according button with the label “correct” or “incorrect”. In case the
child did not name any number, the checkbox “no answer given” has to be selected.
For the evaluation the app has to save if a correct or incorrect answer was given. Based on the saved data’s
structure it is also possible to find out what number sequences were done, which number sequence was the last one
and how many second chances the child processed.
Quantity Comparison Assignment
Figure 7. Quantity comparison assignment
Similar to the comparison of numbers assignment, the quantity comparison assignment, shown in figure 7,
makes use of three exercise assignments. The only difference to the comparison of numbers assignment is that the
exercise assignments actually make use of a timer. The goal of the children is to select the suitcase with the most
gold coins in it without counting them. To prevent the children from counting the gold coins, the developmental
psychologists set the use of a timer as a requirement for the assignment. In case the child selects the incorrect
suitcase of an exercise assignment, Poldi asks the child to look closer and to try again. Although, in case the timer
runs out before the child selects a suitcase Poldi only requests the child to be quicker and the app moves to the next
exercise assignment. After all exercise assignments are done and before the real assignments start, Poldi again
prompts the child to select the suitcase with the most gold coins as quick as possible.
The app saves which suitcases was selected, the time the child took to select a suitcase or if the time ran
out.
-221-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

Quantity Knowledge Assignment
Figure 8. Quantity knowledge assignment
The quantity knowledge assignment, shown in figure 8, is one of the simplest assignments in its structure.
The assignment also belongs to the supervisor assisted assignments with the only difference that the screening
supervisor has to show the tablet screen to the child. The children’s goal is to count all visible keys in Poldi’s speech
bubble and to tell their answer to the screening supervisor. Based on the correctness of the children’s answer, the
supervisor has to select the according buttons visible in the lower right corner. The buttons are intentionally
designed to be more difficult to see, so that the children do not click the “correct” button on their own.
For the evaluation the application saves the correctness of the child’s answer.
Arithmetic Problems Assignment
Figure 9. Arithmetic problems assignment
Another supervisor assisted assignment is the arithmetic problems assignment, as shown in figure 9. The
children’s goal is to solve the math problems spoken out loud by the screening supervisor. The screening supervisor
than has to input the child’s answer into the input field next to “Bitte Antwort eingeben” (German for: “please enter
answer”) or to select the checkbox “Keine Antwort gegeben” (German for: “no answer given). By clicking the go-
ahead button in the lower right corner, the app saves the supervisor’s input and goes ahead to the next assignment.
For the evaluation the app saves the answer given by the child or if no answer was given.
-222-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

Rapid Automized Naming (RAN) Assignment
Figure 10. RAN assignment Figure 11. Sorting assignment
The rapid automized naming assignment, shown in figure 10, is another supervisor assisted assignment.
Each school that volunteered in the prototype screening process also got a screening manual that describes the
screening process very detailed. With the stated screening manual, the schools also got some sheets of paper with
multiple images and numbers printed on it in the same order as the gray rectangles visible in figure 10. These sheets
of paper are used for the RAN assignment, which consists out of two different assignments, the image naming
assignment and the number naming assignment. The image naming assignment requires the child to speak out loud
the names of the objects seen in the image. The number naming assignment is basically the same but requires the
child to speak out loud the number printed on another sheet of paper. To start the assignment the screening
supervisor first has to work with the child on some test items before the real assignment can be started. These test
items also are printed on a sheet of paper and do not require any interaction with the app. After the test items are
done, the screening supervisor is able start the assignment by clicking the start button, which is placed on the lower
right corner, instead of the go-ahead button visible in figure 10. By clicking the start button, an audio file plays back
“ready, set, go” to prompt the child to start naming the images. Every incorrectly named item has to be recorded by
the supervisor by clicking the according rectangle in the tablet app’s view that becomes highlighted by a green
background. In case the timer ran out before the child named all items, the supervisor has to click the last-named
item, shown by changing the background color of the item to red. Although, if the child was able to name all items,
the supervisor has to stop the timer by clicking the stop button in the lower right corner of the app. The number of
the incorrectly named items can be changed in a popup window shown by the app after the assignment is done.
After the image naming assignment is done, the number naming assignment can be started. Both
assignments are completely similar in its structure with the only difference that if the child was not able to name all
numbers which are used as test items for the number naming assignment, the number naming assignment is skipped.
The app saves the number of incorrectly named items, the last item named as well as the remaining time if
the child was quick enough and if the number naming assignment was skipped.
Sorting Assignment
The screening’s last assignment, as well as the longest one, is the sorting assignment, shown in figure 11.
Similar to the RAN assignment, the assignment makes use of additional material provided to all volunteered
schools. The children’s goal is to sort multiple cards given to them by the screening supervisor to three different
stacks. Each card has 12 items printed on them. These stacks are split into cards which have a flower, a banana or
both printed on them. Before the assignment starts, the supervisor asks the child to assign the first few cards as
-223-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

training cards to the stacks without using the application. After all training cards are done, the assignment can be
started by the supervisor clicking on the start button shown in an overlay view. For each card assigned by the child
after the assignment is started, the supervisor has to click the according treasure visible in figure 11. In case the child
changes the assignment of a card, the supervisor is able to go to the previous cards by clicking the button left from
the card in the middle of the screen. Since the assignment assesses the child’s attentiveness, it is highly possible that
the children need to be motivated sometimes. For every motivation spoken out by the supervisor, the motivation
button in the left lower corner has to be clicked to increase the motivation counter. In case a child really does not
want to finish the assignment, the assignment can be cancelled. This can be done by clicking the red button in the
lower left corner and by confirming the alert view asking if the assignment really should be cancelled. Furthermore,
the assignment has integrated a timer with seven minutes that stops the assignment if the timer finishes before the
child assigned every card. In case the child manages to assign every card within seven minutes, the timer
automatically stops counting at the moment the last card is assigned. By clicking the button with the label “Aufgabe
abschließen” (German for: “finish assignment”), the assignment is finished, and the tablet can be placed in front of
the child again.
For the evaluation the app saves the number of motivations needed, the last assigned card in case the time
ran out, the remaining time if the child was quicker than the timer and if the assignment was aborted including the
last assigned card before the abort button was clicked. In addition, the app saves for every card if the card was
assigned correctly.
Automatic Evaluation
After all screening assignments are finished, the app shows the treasure and congratulates the child by
displaying fireworks and playing the according audio file as already described in one of the previous chapters. The
application’s assignments are allocated to the seven following areas of competences:
1. “Phonology” consisting out of the phonological awareness and RAN assignments
2. “Letters and sounds” consisting out of the phonological awareness, knowledge of letters and the
number recognition assignments
3. “Quantity and number knowledge” consisting out of the knowledge of letters, number recognition,
comparison of numbers and quantity comparison assignments
4. “Counting” consisting out of the knowledge of letters, number recognition, comparison of
numbers and quantity comparison assignments
5. “Working memory” consisting out of the quantity knowledge and arithmetic problems
assignments
6. “Attentiveness – stamina and accuracy” is calculated by a specific part of the sorting assignment
results’ data
7. “Attentiveness – motivation” is based on different data of the sorting assignment results
Based on the correctness of the child’s answers, the app calculates if the child has any special educational
needs and which level of special educational needs the child has in the different assessed areas. These levels include
“Leichter Förderbedarf” (German for: “slight educational needs”), “Deutlicher Förderbedarf” (German for: “obvious
educational needs”) and no educational needs which is presented by a check mark as visible in figure 12.
-224-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

Figure 12. The screening's evaluation result screen
Structure of the Excel File
The Excel File’s structure was built based on the requirements of the developmental psychologists. All
results of one assignment type were put into a different Excel sheet. As described previously, the implementation of
creating a single file to download had to be changed to creating multiple files, each containing up to a hundred
screening runs. For example the quantity comparison Excel sheet consists out of rows including the child’s
identifier, the date of the screening run, the identifier of the screening progressed, a serial number which increases
for every assignment result of the same assignment type and screening run, a row which contains the item type of
the image used (only relevant for the psychologists), the name of the images used in the assignment, the correct
answer, the given answer, the correctness of the child’s answer, the time the child took to give an answer and the
name and ID of the school. Based on different highlighting of the cells, including bold text or background colors, it
can easily be recognized if the assignment was solved correctly.
Discussion
The first part of the screenings processed with the prototype app were done at the beginning of 2019. These
screenings only contained three of the planned assignment types but were processed to get first feedbacks from
schools, teachers, children, and from the developmental psychologists. After all remaining screening assignments
were implemented at the beginning of September 2019, the next screenings took part. The children already assessed
with the first prototype as well as multiple children which did not have taken part in the first screening runs were
part of the second screenings. Altogether, the second screening runs examined more than 800 children in the 25
Austrian schools that volunteered.
Due to the high number of children that took part in the prototype screenings, the evaluation of the app was
done by gathering reviews from the school’s screening supervisors, which were usually the school’s teachers. These
reviews were done by the developmental psychologists and then discussed with the developer through interview
questions to evaluate the tablet app and the web application. Based on their answers, the assignments were
implemented very good to fit their requirements, which was possible by developing the applications in a more agile
way by discussing every major milestone, like a finished implementation of a new assignment type, with the
developmental psychologists and make adjustments in the app’s code when needed. Due to the short time available
from beginning of the application’s implementation and the first screenings, it was not possible to implement
automated unit tests to ensure the prototype’s stability. Therefore, some technical errors occurred during the
prototype screenings, which then could be fix quickly. These problems included the already mentioned problem with
the Excel file that became too big in its size, a problem that hindered the mobile app from correctly downloading all
audio and image files in some rare cases and an out of memory (OOM) exception, which occurred in the very first
screenings if multiple screenings with multiple assignments were done one after another. The teachers’ experience
using the tablet app as well as the paper pencil screening, including the same assignments as the tablet app, were
also very positive. Most teachers would choose the mobile app over the paper pencil version due to its usability and
advantages, like less paperwork. Some teachers only saw a problem in the screening’s length, which is only
influenced by the number of assignments created by the developmental psychologists. Furthermore, the children did
-225-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

become familiar with the app very quickly and only very few children had problems first. These children usually did
not have any experience with tablet computers but could be helped immediately by the screening supervisors.
Conclusion
With the help of the developmental psychologists, there was definitely done a step in the right direction to
digitalizing a new screening process that helps the educational development of children in Austria. However, it yet
has to be defined if the screening should be used voluntary or be part of the children’s school start in the future by
the Austrian government. According to the data gathered by the interview, 70% of the teachers involved in the
screening runs see the screening as a new helpful basis for decision making, nevertheless only half of the
volunteered schools’ principles would base their decision primarily on the screenings’ results and only 8% of the
teachers belief the screening would be used in schools if its use would by voluntary.
Acknowledgements
We thank the developmental psychology teams of the University of Graz and the University of Vienna for
the successful collaboration, the designers of the Institute of Interactive Systems and Data Science at the Graz
University of Technology for the nicely designed graphical assets, as well as the recording team of the Graz
University of Technology’s Educational Technology Unit for recording the audio files used in the mobile
application.
Literature
Clarke, Doug et al. (2002). “Early Numeracy Research Project Final Report, February, 2002.”
Dietrich Rüdiger, Adam Kormann, Helmut Peez (1976). Schuleintritt und Schulfähigkeit: Zur Theorie und Praxis
der Einschulung. München: Reinhardt
Federal Ministry of Education, Science and Research (Jan. 2020). Das Pädagogik-Paket. Second, updated edition
Ferreira, Susan M, Charles Gouin-Vallerand, and Richard Hotte (2016). “Game based learning: A case study on
designing an educational game for children in developing countries.” In: 2016 8th International Conference on
Games and Virtual Worlds for Serious Applications (VS-GAMES). IEEE, pp. 1–8
Marcus Hasselhorn, Wolfgang Schneider, Harald Marx (2000). Diagnostik von Lese-Rechtschreibschwierigkeiten:
Tests und Trends. Hogrefe Verlag. ISBN: 3-8017-1375-X
Wolfgang Schneider, Harald Marx, and Marcus Hasselhorn (2008). Diagnostik von Rechtschreibleistungen und
Rechtschreibkompetenz. Hogrefe Verlag
Wolfgang Schneider, Marcus Hasselhorn (2018). Schuleingangsdiagnostik. Hogrefe Verlag GmbH & Co. KG.
ISBN: 978-3-8017-2926-4
-226-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021