Content uploaded by Martin Ebner
Author content
All content in this area was uploaded by Martin Ebner on Jun 26, 2017
Content may be subject to copyright.
Originally published in: Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative Learning Analytics
Application. In J. Johnston (Ed.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2017 (pp.
810-820). Association for the Advancement of Computing in Education (AACE).
The Multiplication Table as an innovative Learning Analytics Application
Eltion Kraja
Educational Technology, Graz University of Technology, Austria
krajael@sbox.tugraz.at
Behnam Taraghi
Educational Technology, Graz University of Technology, Austria
b.taraghi@tugraz.at
Martin Ebner
Educational Technology, Graz University of Technology, Austria
martin.ebner@tugraz.at
Abstract: The main topic of this paper is the development of a web-based application that helps
children to learn the one-digit multiplication table. The developed application supports individual
learning process of the pupils and also provides the teachers with the possibility to intervene
according to the analysis of users’ answers. The application uses modern technologies in order to
offer high performance and availability to the users. The system also provides an interface for
mobile clients, which present the questions and the processed data in different forms. The answers
of the pupils, as well as other gathered data from the application show interesting results related to
the participation and learning improvement.
Introduction
Nowadays the possibility for the users to use their mobile devices, websites or different technologies for
learning is growing. Also the learning process and the methodology of learning have changed, depending on the way
in which the teachers present the learning material and how the pupils consume it.
Independently from the form of learning, the characteristics of learning are (Berg 2011): 1) individuality, as
there are different methods and strategies how to acquire knowledge, 2) activity, considering that the process of
learning is a physical and mental activity, 3) constructiveness that includes the symbols and conventions we use to
learn, 4) accumulation of knowledge - new knowledge is added to the preexisting, 5) self regulation, the learning
process has its individual pace (fast, slow), and 6) dependence on the situation because learning is also a product of
experience, emotions etc..
Using digital and electronic material, the users produce many data. The data can be used to evaluate the
behavior of the users and also to make considerations concerning a group of users i.e. a school class of pupils.
Having big amount of user data, intervention strategies and predictions can be created in order to influence the
process. The produced data can be used for different purposes, such as for data evaluation, for the identification of
patterns, for a better use of resources, to create individual learning plans, for the identification of the strength and
weakness of the group and to improve the learning materials and communication.
Learning analytics as a field of study that processes and analyzes such data helps to improve learning
efficiency. The 2016 Horizon Report1 describes learning analytics as: "an educational application of web analytics
aimed at learner profiling, a process of gathering and analyzing details of individual student interactions in online
learning activities".
[1] https://library.educause.edu/topics/teaching-and-learning/learning-analytics (last visit 15.10.2016)
Originally published in: Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative Learning Analytics
Application. In J. Johnston (Ed.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2017 (pp.
810-820). Association for the Advancement of Computing in Education (AACE).
Usability is another field of study in this work. It plays a significant role in web applications. Usability
(Scholtz at al. 2016) is defined by the ISO as follows: “the extent to which a product can be used by specified users
to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”.
In this work, we introduce a learning application, which is designed for school pupils to learn one-digit
multiplication problems. To make the application more interesting, the posed questions and the tasks are presented
in form of a game. The idea of gamification has been applied more often in many game based learning applications
in recent years. While playing a game, users can be motivated by trying to achieve more points, badges, themes etc.
Gamification is defined as: “using game-based mechanics, aesthetics and game thinking to engage people, motivate
action, promote learning, and solve problems” (Kapp 2012). Children tend to learn and discover new things, but
most often there is a problem of knowledge presentation. Furthermore, if one asks a child “What is work?”, she
would probably answer “homework”. Asking, “What is fun”, the answer could be “video games”. The motivation of
the children to play games and doing homework while it is amusing, can be included in the learn processes1.
This paper describes the implementation and the features of the new “1x1 trainer” application. The created
platform is aimed to apply the knowledge from the mentioned research fields above to replace the old version that
was in operation till March 2016. The application’s logic is based on the previous work by (Schön et al. 2012). The
implemented algorithm provides users with the multiplication tasks. The selection of tasks is based on user’s profile
and the degree of her knowledge. The new 1x1 web application refractors the structure of the old application,
provides some new features (gamification aspects) and bases on the newest technology stack. The application is
extended so that it is able to communicate with other existing platforms (e.g. platforms for central user management
and mobile apps).
The developed new web learning application keeps track of the user’s learning behavior. It presents each
pupil individually the result and the level of his/her knowledge. The children can earn points while answering the
questions and use them to activate or enable the game characters. The users also have the possibility to observe their
game activities and thereby revise the gained knowledge presented during the game.
The teachers and the administrators can access and evaluate the data produced by the pupils. In order to
have an overview over the classes and groups, the data is presented in a clustered form and can be easily scaled to a
more detailed level. For a better analysis, the information is highlighted with colors. Besides that, indicators and
charts are used to give the user a quick overview. The collected data is used to provide an individual learning
support as well as to give feedback to the children on their performance.
State of the Art
This section covers three main areas of this work, namely Learning Analytics, Usability and Gamification.
Learning analytics
Learning Analytics is an innovative research field that uses intelligent algorithms to process (searching,
filtering, mining and visualizing (Khalil & Ebner 2016)), exploit and retrieve useful and meaningful information
from the data gathered from the users.
The Society for Learning Analytics Research (SoLAR) defines Learning Analytics as: “the measurement,
collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and
optimizing learning and the environments in which it occurs” (Siemens & d Baker 2012).
Four aspects of learning analytics are introduced by (Daniel 2016). These are: what, who, why and how to
analyze. More exactly, What explains the data used for the analysis and the environment or context the date is
retrieved from, e.g. LMS2 and PLE1 (Ebner et al. 2010). Who: involves the stakeholders the learning analytics is
[1] http://yukaichou.com/gamification-examples/top-10-ecommerce-gamification-examples-revolutionize-shopping/ (last visit
20.10.2016)
[2] Learning Management System
Originally published in: Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative Learning Analytics
Application. In J. Johnston (Ed.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2017 (pp.
810-820). Association for the Advancement of Computing in Education (AACE).
applied for, e.g. teachers, students, tutors, institutions, and researchers. Why: explains the objectives of the analysis.
This could be monitoring the learning behavior, prediction of learning outcome or necessary interventions,
mentoring, recommendations, personalization and supporting adaptive learning systems. How: is about how the data
is analyzed, which patterns or methods are used; some examples are statistics, visualizations, data mining, Social
Network Analysis and machine learning.
Furthermore Khalil and Ebner (Khalil & Ebner 2015) proposed the Learning Analytics Life Cycle, which
reveals the relation between the four aspects mentioned above. The life cycle begins with the stakeholders within a
learning environment from which the data is gathered. This might be interaction data between stakeholders and/or
stakeholders and the learning environment (traces), the personal data or academic information in general. The data
needs to be processed (analyzed) in the next step using different methods and techniques; some examples were
mentioned above. The result of the analysis step leads to the data interpretation that is the objective.
Usability
Heuristics Evaluation (HE) is an informal method where evaluators can analyze usability aspects of a user
interface (Nielsen & Molich 1990). Alsumait and Al-Osaimi (Alsumait & Al-Osaimi 2009) described a set of
heuristics for e-learning applications. One aspect that is to be evaluated is the general user interface is hat it should
be very intuitive and appropriate for 6-10 years old children. The pedagogical aspect as well as supporting the
learners in the learning process is also very important. The sets of heuristics are divided into three categories, by
which the mentioned aspects are analyzed. These are: 1) Nielsen Usability Heuristics (NUH) (Alsumait & Al-
Osaimi 2009) that is concerned with general usability und design aspects; 2) Child Usability Heuristics (CUH)
(Alsumait & Al-Osaimi 2009) focused on the child abilities and preferences; 3) E-learning Usability Heuristics
(EUH) (Alsumait & Al-Osaimi 2009) focuses on the learner centered design. These three categories are briefly
described as follows:
1) Nielsen Usability Heuristics (NUH) (Alsumait & Al-Osaimi 2009, Nielsen 1994): According to NUH,
the system should keep the user informed about what is happening and give him feedback in order to motivate the
user to use the application and stay active. Additionally, the user should have the possibility to see his status (points,
level, etc.) in the application and also understand the terminology used. The system presented to the user should
have familiar concepts from the real world and also be built in an intuitive form, in order to guarantee a clear
functionality of the buttons and other interface elements. Furthermore giving the user the control, freedom and
knowledge to navigate and move in the system within defined frames is a very important aspect. Characteristics of
this aspect are for example the possibility to undo false input, to move easily in the application, to navigate through
and filter large amount of data, to save the state of the application, etc. The application should give the user feedback
and prevent errors by design. The application should provide the user with the possibility to correct his input (e.g.:
by asking questions such as “Did you mean…”). Another characteristic of this heuristic is “Recognition vs. Recall”.
It means that the user should not have to remember the information when navigating from one view to another. He
should have the possibility to get all the information he needs quickly from the current view. Additionally the tool
should integrate beginners but also help experts to speed up their interaction. Also showing the information in
different way is a very important aspect.
2) Child Usability Heuristics (CUH) (Alsumait & Al-Osaimi 2009, Sim et al. 2006, Barendregt et al.
2003): According to CUH, the screen layout should be efficient, attractive and also be presented as simple and
readable. All elements of the site need to be chosen suitable for the children. Game or application devices should be
chosen appropriately for the target user group. Buttons with no functionality should be disabled in order to prevent
input errors. The user should have enough information to use the application when he turns it on. The goals of the
application should be clear. It should also challenge but not frustrate the user. The application should be easy to
learn but hard to master. While playing, the application should reward the child to motivate and encourage him. The
user should get involved in the program and its topics as far as possible consequently. The tool should help the
children to use their imagination. The user should be able to use his imagination to interpret the game; the characters
are chosen appropriately for the target user and they should attract the child’s interest and curiosity.
[1] Personal Learning Environment
Originally published in: Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative Learning Analytics
Application. In J. Johnston (Ed.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2017 (pp.
810-820). Association for the Advancement of Computing in Education (AACE).
3) E-learning Usability Heuristics (EUH) (Alsumait & Al-Osaimi 2009): The terminology and the used
vocabulary should be chosen appropriately and suitable for the stakeholders. There are many examples and graphics
to illustrate abstract formulas or rules. The structure of the content should be well chosen, so that the rapid
understanding of the goal is possible. The application should keep the user concentrated and motivated throughout
the activities, stories and games situations.
Gamification
Gamification is a concept with an enormous potential in the field of education. The Gamification technique
should increase interest, curiosity, and fun. It should challenge the user, while using learning software (Martí-
Parreño et al. 2016, Scholtz et al. 2016, Brull & Finlayson 2016, van Roy & Zaman 2017).
In order to motivate the children to use a game-based learning application different works about rewards
and gamification were analyzed. Becker and Nicholson (Becker & Nicholson 2016) describe meaningful
gamification as a personal connection to a non-game setting. The use of specific elements in the application causes
the user to reflect on the situations he experienced during the game. Additionally the use of narratives helps the
users connect the context to their own experiences. Giving users the freedom to make decisions, to try other
alternatives and to explore the context is also very important. Some of the gamification characteristics described by
(Becker & Nicholson 2016 ) are (1) Reflection, (2) Exposition, (3) Choice, (4) Information, (5) Play, and (6)
Engagement – communication and engagement between the users.
Previous System
The application that is described here is based on the previous software and work made by (Schön et al.
2012). The applied algorithm in this application is based on the idea of the degree of competence (DoC), which aims
to select the appropriate questions depending on the user’s knowledge. At the beginning the pre-knowledge of the
user, his initial DoC respectively, is estimated from a pretest; this is done by asking two questions to estimate the
DoC of the user, as illustrated in (Fig. 1).
Figure 1. Pre-DoC estimation by (Schön et al. 2012)
The DoC or the so-called learn rate will be regulated, according to the user’s answers, within the interval of
[0, 1]. The questions are divided into two parts: the already-known ones (those that are below the user’s learn rate)
and the ones that are to learn or to practice (those above the user’s learn rate). The algorithm is programmed to avoid
giving the users very difficult or very easy questions in order to prevent frustration or boringness. The algorithm
presents a mix of known and unknown questions, within the range of user’s learn rate. In order to motivate the user,
the algorithm also selects questions from the so-called extended learning area, which is 25% above the user’s learn
rate (see Fig. 2).
Originally published in: Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative Learning Analytics
Application. In J. Johnston (Ed.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2017 (pp.
810-820). Association for the Advancement of Computing in Education (AACE).
Figure 2. Distribution of the questions based on the degree of competence (Schön et al. 2012)
The questions are classified into following three categories according to the users’ answers:
- Unknown questions (identified via incorrect answers or unanswered questions)
- Known questions (identified by correct answers)
- Well-known questions (known questions that are answered correctly more than once. Depending on
the probability of the question, the user has to answer a question more than twice right in order to
classify it as well-known).
The so-called “result type” is also a significant aspect of the categorization of the answers. It divides the
answers into “NR” (a new question, correctly answered), “NW” (a new question, incorrectly answered), “KR” (a
known question, correctly answered) and “KW” (a known question, incorrectly answered) categories.
The collected answers and their categories have provided a data basis for this work and many other previous related
works such as (Taraghi et al. 2015, Taraghi et al. 2014).
The Multiplication Table (1x1) Trainer
The so-called existing “Einmaleins-Trainer” has been rebuilt based on the new web-technologies.
The challenge, while implementing the application, was the combination of the modern technologies, the flexible
structure as well as applying the outcome of the research fields discussed in section “State of the art”. To make the
trainer available for as many users as possible the application was written as a web-platform. The application also
offers a SOAP-interface to mobile devices and other co-applications such as one responsible for the user
management. The application structure consists of a client and a server component. The structure and the chosen
design patterns guarantee that the application is extendable and flexible for further improvements and future work.
(Fig. 3) gives an overview about the structure of the new 1x1 trainer.
Originally published in: Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative Learning Analytics
Application. In J. Johnston (Ed.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2017 (pp.
810-820). Association for the Advancement of Computing in Education (AACE).
Figure 3. Structure of the 1x1 application
The application offers four user roles: supervisor, administrator, teacher and pupil. The Administrator is
allowed to see all registered entities and to gather grouped and clustered information, but also to zoom into the
detailed view of each entity. The administrator can also regulate the system settings. The Teacher can only observe
his own pupils and classes. He can also view the data in a clustered form and can scale down and filter the
information to see the individual answering behavior of each pupil. Using the new 1x1 application, the teachers can
quickly find an answer for many questions that could be of their interest, such as:
- Which pupils exercised mostly?
- Who never played?
- Which questions are/were very easy to answer by the pupils?
- Which questions are/were difficult to solve?
Figure 4. Clustered answers. Teachers and administrators can view at once the answers’ status. The colors vary
from “dark green = very easy” to “dark red = very difficult” symbolizing the difficulty of the question.
(Fig. 4) shows an overview of the clustered data. Through the colors and the information in the matrix, a
teacher can see which questions were answered easily (the green ones) and which were very difficult (the red ones).
The Pupil can play the game and answer the questions. (Fig. 5) shows the state of the game before the user
begins to play. Pupils can view their own activities and gain points for correct answers, which in turn can be used to
activate the game characters (see Fig. 6). The characters bring more variety and fun in the game (through changing
layouts, animations etc.).
Originally published in: Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative Learning Analytics
Application. In J. Johnston (Ed.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2017 (pp.
810-820). Association for the Advancement of Computing in Education (AACE).
Figure 5. Game view
Figure 6. Available characters
The answered questions will flow to the main statistic, so that the teacher can estimate the progress of the
user.
The application is based on responsive design and can be accessed from modern browsers of every device.
The symbols are simple and meaningful so that they are familiar for the users from other programs and everyday life
(e.g.: game symbol, play and stop button etc.). The pupils get a feedback after each answered question. They have an
overview over their answered questions. The characters are consciously designed to be gender neutral and be
familiar to children. The pupils have also the possibility to review their exercises and learn from questions
experienced in the past.
Teachers have all the information in form of a statistic with different diagrams and lists. They can quickly
reveal the strength and weaknesses of the class or a group of pupils on different dimensions. The application offers
the possibility to reveal the relation between question and answers, difficultness, and the relations within the
classified answers in different forms.
The visualizations can be filtered and scaled down in order to better target the point of interest and
eventually help teachers to intervene for improvement of their pupils.
Discussion and Outlook
We have gathered large amount of data through the 1x1 trainer application. The application is used by 18
schools, containing 83 classes and 7164 users (who have answered at least two questions). Totally 1033237
questions have been already solved1.
[1] Per 9 October 2016
Originally published in: Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative Learning Analytics
Application. In J. Johnston (Ed.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2017 (pp.
810-820). Association for the Advancement of Computing in Education (AACE).
The bar chart of (Fig. 7) depicts the distribution of the classified answers, described before. It is obvious
that the majority of the questions are well-known, followed by known and unknown questions.
The technology and structure used for the implementation of the application allows further extensions and
enhancements. There is the possibility to reach more devices and more operating systems.
The 1x1 trainer application is implemented using Zend Framewok 21 for the server component and
AngularJS2 for the client component. These frameworks allow a modular setup. They are also suitable for dividing
the application logic into smaller parts. The client component is suitable for building hybrid apps with platforms like
cordova3 or ionic4. In this way it is for example possible to build a Windows app with the same look and feel as the
web app. The server component could be extended to offer more web-services in order to provide more data to the
clients. Furthermore there is still potential to improve and extend the current visualizations.
Currently the exercises can be answered by typing the result in an input form. Other methods like multiple
choice options can be implemented as well. To challenge the user even further, the possible answers could be
presented close to each other.
The algorithm, which is responsible to select the appropriate question to be posed to the user, can easily be
exchanged. This allows the administrators to compare the results and data produced by the children via different
learning algorithms. Hence the effectiveness and performance of different algorithms on the learning goal can be
analyzed.
Figure 7. Classified answers. Well-kwon questions in the dark green bars, known questions in the light green bars
and unknown questions (wrong answers) in the red ones
References
Alsumait, A., & Al-Osaimi, A. (2009, December). Usability heuristics evaluation for child e-learning applications.
In Proceedings of the 11th International Conference on Information Integration and Web-based Applications &
Services(pp. 425-430). ACM.
[1] https://framework.zend.com/ (last visit 20.11.2016)
[2] https://angularjs.org/ (last visit 20.11.2016)
[3] https://cordova.apache.org/ (last visit 20.11.2016)
[4] http://ionicframework.com/ (last visit 20.11.2016)
Originally published in: Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative Learning Analytics
Application. In J. Johnston (Ed.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2017 (pp.
810-820). Association for the Advancement of Computing in Education (AACE).
Barendregt, W., Bekker, M. M., & Speerstra, M. (2003, September). Empirical evaluation of usability and fun in
computer games for children. In Proc. IFIP Interact (Vol. 3, pp. 705-708).
Becker, K., & Nicholson, S. (2016). Gamification in the Classroom: Old Wine in New Badges. Learning, Education
and Games, 61.
Berg, M. (2011). Wie lernen Kinder das Lernen?. GRIN Verlag.
Brull, S., & Finlayson, S. (2016). Importance of Gamification in Increasing Learning. The Journal of Continuing
Education in Nursing, 47(8), 372-375.
Daniel, B. K. (Ed.). (2016). Big Data and Learning Analytics in Higher Education: Current Theory and Practice.
Springer.
Ebner, M., Scerbakov, N., Taraghi, B., Nagler, W., Kamrat, I., & Maurer, H. (2010, March). Teaching and learning
in higher education–an integral approach. In Proceedings of Society for Information Technology & Teacher
Education International Conference (pp. 428-436).
Kapp, K.M. (2012). Using game-based mechanics, aesthetics and game thinking to engage people, motivate action,
promote learning, and solve problems. In The gamification of learning and instruction: game-based methods and
strategies for training and education. John Wiley & Sons. p 10
Khalil, M., & Ebner, M. (2015, June). Learning analytics: principles and constraints. In Proceedings of World
Conference on Educational Multimedia, Hypermedia and Telecommunications (pp. 1326-1336).
Khalil, M., & Ebner, M. (2016). When Learning Analytics Meets MOOCs-a Review on iMooX Case Studies.
In Innovations for Community Services: 16th International Conference, I4CS 2016, Vienna, Austria, June 27-29,
2016, Revised Selected Papers 16 (pp. 3-19). Springer International Publishing.
Martí-Parreño, J., Seguí-Mas, D., & Seguí-Mas, E. (2016). Teachers’ Attitude towards and Actual Use of
Gamification. Procedia-Social and Behavioral Sciences, 228, 682-688.
Nielsen, J. (1994, April). Usability inspection methods. In Conference companion on Human factors in computing
systems (pp. 413-414). ACM.
Nielsen, J., & Molich, R. (1990, March). Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI
conference on Human factors in computing systems (pp. 249-256). ACM.
Scholtz, B., Raga, L., & Baxter, G. (2016). Design and evaluation of a “gamified” system for improving career
knowledge in computing sciences.
Schön, M., Ebner, M., & Kothmeier, G. (2012, April). It's just about learning the multiplication table.
In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 73-81). ACM.
Siemens, G., & d Baker, R. S. (2012, April). Learning analytics and educational data mining: towards
communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and
knowledge (pp. 252-254). ACM.
Sim, G., MacFarlane, S., & Read, J. (2006). All work and no play: Measuring fun, usability, and learning in
software for children. Computers & Education, 46(3), 235-248.
Taraghi, B., Saranti, A., Ebner, M., Müller, V., & Großmann, A. (2015). Towards a Learning-Aware Application
Guided by Hierarchical Classification of Learner Profiles. J. UCS, 21(1), 93-109.
Originally published in: Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative Learning Analytics
Application. In J. Johnston (Ed.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2017 (pp.
810-820). Association for the Advancement of Computing in Education (AACE).
Taraghi, B., Saranti, A., Ebner, M., & Schön, M. (2014, June). Markov chain and classification of difficulty levels
enhances the learning path in one digit multiplication. In International Conference on Learning and Collaboration
Technologies (pp. 322-333). Springer International Publishing.
van Roy, R., & Zaman, B. (2017). Why Gamification Fails in Education and How to Make It Successful:
Introducing Nine Gamification Heuristics Based on Self-Determination Theory. In Serious Games and Edutainment
Applications (pp. 485-509). Springer International Publishing.