Determining the Causing Factors of Errors for Multiplication Problems

Abstract

Literature in the area of psychology and education provides domain knowledge to learning applications. This work detects the difficulty levels within a set of multiplication problems and analyses the dataset on different error types as described and determined in several pedagogical surveys and investigations. Our research sheds light to the impact of each error type in simple multiplication problems and the evolution of error rates for different error types in relation to the increasing problem-size.

Full text

E-iED 2014 ImmersiveEducation.org
PROCEEDINGS 144
Title: Determining the Causing Factors of Errors for Multiplication Problems
Authors: B. Taraghi
1
*, M. Frey
1
, A. Saranti
1
, M. Ebner
1
, V. Müller
2
, A. Großmann
2
Affiliations:
1
Graz University of Technology, Münzgrabenstrasse 35/I, 8010 Graz Austria.
2
UnlockYourBrain GmbH, Französische Strasse 24, 10117 Berlin, Germany.
*Correspondence to: b.taraghi@tugraz.at
Abstract: Literature in the area of psychology and education provides domain knowledge to
learning applications. This work detects the difficulty levels within a set of multiplication
problems and analyses the dataset on different error types as described and determined in several
pedagogical surveys and investigations. Our research sheds light to the impact of each error type
in simple multiplication problems and the course of error types in problem-size.
One Sentence Summary: This work consists in the investigation of the various error types in
multiplication problems, as well as the problem-size effect.
Main Text:
1 Introduction: Learning simple multiplications is one of the major goals in the first years at
primary school education. Math teachers find pedagogically relevant to know which exercises
improve mathematical abilities, which errors occur repeatedly and on which steps they may
require teacher's intervention.
Applying math training applications can support the teachers in this regard and enhance the basic
math education at primary schools [1]. For example, the 1x1 trainer application [2] that was first
developed by Graz University of Technology, assists the training process of pupils and enhances
the pedagogical intervention of the teachers for learning one-digit multiplication problems at
schools. The application was used in several primary schools for training goals. In our previous
works [3, 4] we analysed the gathered data (about 500,000 calculations) to get insight about the
learners' answering behaviour within this application. We identified difficulty levels within the
set of one-digit multiplication problems. In this work we continue our research on another
dataset generated by the Android application UnlockYourBrain, which poses different basic
mathematical questions to the learners. The focus is drawn first to the multiplication problems.
We perform the same analysis steps as in our previous work to identify the difficulty levels. We
primarily want to shed light to the reasons of the incorrect answers. Therefore, based on the error
rates driven from the first part of the analysis, for each multiplication problem we detect different
error types known from the literature. We present the probabilities of occurrence of the various
error types in detail and explain them individually, for each specific multiplication problem.
Draft - extended version originally published in: Taraghi, B., Frey, M., Saranti, A., Ebner, M., Müller, V.
Großmann, A. (2015) Determining the Causing Factors of Errors for Multiplication Problems. In:
Immersive Education. Ebner, M., Erenli, K., Malaka, R., Pirker, J., Walsh, A. (Eds.). Communications in
Computer and Information Science 486. Springer. pp. 27-38

E-iED 2014 ImmersiveEducation.org
PROCEEDINGS 145
Section 2 describes the dataset that is used for analysis purposes. Section 3 covers the difficulty
levels of the multiplication problems, the findings and interpretations based on the difficulty
probabilities. Section 4 describes the detected error types and proceeds to the analysis of error
types in section 5.
1.1 Related Work: There are two major arithmetic models of fact retrieval that deal with errors
in simple multiplications; the modified network interference theory by Campbell [5] and the
interacting neighbours model by Verguts and Fias [5]. Both models introduce some common
error types and their cause in simple multiplication problems.
One of the most occurring error types in simple multiplication problems are the operand errors.
They happen whenever the failed result is the product of one of the neighbouring operands
instead of the given ones; e.g. 48 = 6 * 8 for the given problem 7 * 8. The survey done by
Campbell [5], shows that the majority of errors can be classified in this category. The operand
error rates differ for each multiplication problem and are not uniformly distributed [7].
Operand intrusion error happens when at least one of the two operands matches one of the digits
of the result; e.g. answering 74 to the posed question 7 *8. Campbell argued that reading the
operands as if they were two-digit numbers causes this error. This argument is supported from
the fact that the first operand is observed in the decade digit's place and/or the second operand
appears at the unit digit of the result [7, 8].
One of the initial findings in solving arithmetic problems is the so called problem-size effect.
The problem size is defined as the sum of the operands [9]. The error rates increase as the
problems get larger and the response time evolve correspondingly. The only exceptions are five
problems (problems involving 5 as operand e.g. 5 * 7) and tie problems (problems with repeated
operands e.g. 4 * 4), that do not exhibit this error to a large extend. These problems can be
answered faster in comparison to other problems of the same category [10].
The interacting neighbour model of Verguts and Fias [11] introduces the concept of consistency
of multiplication problems. The concept of consistency was formerly known from the language
literature [12], where it was proposed that the reaction time to pronounce a given word depends
on the consistency of the word to its neighbours, with respect to pronunciation. In the context of
simple multiplications, each problem has a set of neighbouring problems. The operands that are
used in these problems, are the neighbours of the operands (in the multiplication table) of the
original problem. Two arbitrary problems are consistent if their solutions have the same decade
or unit digits; e.g. 56 = 7 * 8 and 36 = 4 * 9 are two consistent problems with respect to their unit
digit. The authors argue, that the consistency measure explains the problem-size effect as well as
the tie effect. Tie problems have less neighbours and they are inconsistent rather than consistent.
Hence less competition exists for tie problems. For all five problems there are consistent
neighbours with distance 2 (they share 5 as unit digit). Although the neighbour distance is far, it
is assumed to be the reason for smaller error rates. Altogether, multiplication problems that have
a higher consistency with their neighbours can be answered faster with higher accuracy [13].

E-iED 2014 ImmersiveEducation.org
PROCEEDINGS 146
2 Dataset Description: The learning application that was used to provide insight for the
characterisation of learning difficulties is UnlockYourBrain. Android users are confronted with
basic mathematical questions each time they attempt to unlock their screen. The application
provides for each posed question a list of possible answers; only one of them is correct. The list
has variable length, meaning that it can vary from trial to trial between two and five possible
answers, even if the posed question is the same. The answering process evolves as follows: the
learner can either attempt to answer or chooses to skip the usage and continue with unlocking the
screen. In case of an answering attempt, either the correct answer is chosen and the application
finishes, or a wrong answer is selected. In the latter case the application indicates the mistake and
repeats the question with the remaining possible answers. The user reattempts to answer the
question with less possible answers or chooses to skip.
The dataset was cleaned to remove noise and was reduced to a minimum number of occurrences
of entities in order to ensure a high degree of confidence in the statistical results. The methods
used can be read at [14]. The final dataset contained 268 questions that were posed totally
1191450 times to 46357 users.
3 Answer Types and Difficulty Levels of Multiplication Problems: A measure of the
difficulty of a question is the answering manner of the learners. The possible answer types are
gathered in the following set {R, WR, W, WWR, WW, WWWR, WWW, WWWW} where W means
“wrong” and R “right”. A question that was posed with three answering options (see [14]) can
have three answering types: R which denotes that the user found the correct answer in the first
answering attempt, WR that the first attempt was wrong but the second right and WW that both
attempts failed. The set of answer types is the classification algorithm's dimensions. Every
multiplication lies in an eight-dimensional feature space where the value in each dimension is the
probability that the question was answered as the corresponding answer type. By applying the K-
Means algorithm [15] in this space we classified the problems in 11 clusters; each of them
contains problems that were answered in similar means from the learners.
Figure 1 depicts the computed difficulty probabilities (error rates) of all provided multiplication
problems within the dataset. A low probability indicates a rather easy problem whereas a high
probability implies a relatively difficult one.

E-iED 2014 ImmersiveEducation.org
PROCEEDINGS 147
!
Figure 1. Difficulty map of multiplication problems. Axes stand for the two operands A and B of
a multiplication problem A * B. Low probabilities imply lower error rates, hence rather easy
problems. High probabilities indicate relatively difficult problems.
It can be observed that the difficulty values appear to a great extent symmetric. The error rate of
problems A * B and B * A seem to be strongly correlated, therefore the order of operands
probably does not have a decisive influence on the error rate. One-digit multiplication problems
are considered easier than the two-digit multiplications. Looking further into the set of one-digit
multiplications (the top left quadratic area in figure 1 where both operands are less than ten) we
achieve the same results as we gained in our previous research work [3] in the one-digit
multiplication problems. 5 and 10 problems are relatively easier to solve. The problems
involving operands 6, 7, 8 and 9 are rather difficult problems.
Looking into two-digit problems, we observe the influence of the 5 and 10 operands in the
simplicity of the question containing them. As in one-digit problems, the unit digits 1, 2 and 5
show the lowest error rates. The same is true for difficult operands. It can be seen that specially
the unit digits 6, 7 and 8 make the two-digit problems extremely difficult relative to the other
operands. Considering the problems containing $5$ as unit digit, the combination with difficult
operands as decade digit leads to a higher error rate, compared to the other decade operands.
The tie effect is also visible. The problems containing repeated operands have lower error rates
compared with other neighbour problems, but the problem-size must be also taken into account.
While the tie problems in the interval of one-digit problems are relatively easy, they become
more difficult for two-digit problems. The provided dataset in our case contains tie problems no

E-iED 2014 ImmersiveEducation.org
PROCEEDINGS 148
greater than 17 * 17. In figure 1, problems 11 * 11 and 12 * 12 seem easy due to their unit digits
(1 and 2 effect); 15 * 15 shows relatively a lower rate than the other tie problems with operands
greater than 12 * 12. It can be argued that the use of 5 as one of the operands could explain this
phenomenon.
4 Error Types: The complete list of analysed error types with a short explanation can be found
in table 1. For a sample given multiplication problem 56 = 7 * 8 an example is given to clarify
how to interpret an error type.
Error type
Description
e.g. 56 = 7 * 8
Operand errors
A neighbouring operand is taken
Split 1
The neighbouring distance is 1
48 = 6 * 8
Split 2
The neighbouring distance is 2 for an
operand or 1 for both operands
40 = 5 * 8
Which operand?
Is the smaller or larger operand affected?
Ties were ignored.
Which neighbours?
Are smaller or larger neighbours taken?
Operand intrusions
A digit of the result matches an operand
First operand
Decade digit matches the first operand
74 ! 7 * 8
Second operand
Unit digit matches the second operand
68 ! 7 * 8
Unit consistency
Only the unit digit is correct
76 ! 56
Decade consistency
Only the decade digit is correct
51 ! 56
Table 1. The analysed error types and their descriptions.
5 Results and Discussion:
5.1 Operand Errors: The majority of errors can be categorized as operand errors. The operand
error rates differ for different multiplication problems (see section 1.1). Figure 2 depicts the
probabilities of an operand error for each simple multiplication problem where each square
represents a specific problem. The first operand can be read off the X-axis, the second operand
off the Y-axis. The color of the square indicates the probability of an operand error occurrence
for the corresponding problem; red color indicates higher probabilities and blue color a very low
probability. As it can be seen, the problems that are rather difficult (see section 3) are more
affected by operand errors than the easy ones.

E-iED 2014 ImmersiveEducation.org
PROCEEDINGS 149
Figure 2. Probabilities (error rates) of an operand error for each simple multiplication problem.
Figures 2A and 2B compare the error rates of an operand error with the split of 1 and 2
respectively. Figures 2C and 2D depict errors with decremented and incremented operands,
respectively. These are restricted to the error rates of operand errors with a split of 1. Figures 2E
and 2F compare errors caused by smaller and larger operands respectively. These are restricted
to the error rates of operand errors with a split of 1 incremented.

E-iED 2014 ImmersiveEducation.org
PROCEEDINGS 150
Figures 2A and 2B show the probabilities of an operand error with the split of 1 and 2.
Comparing the two heatmaps, it is visible that the shortest neighbour distance (split 1) contains
the most operand errors; e.g. for a given problem 7 * 8 the errors such as 48 = 6 * 8 are more
probable than 40 = 5 * 8. It is observable that the most difficult problems have the highest
operand error rates. Relatively easy problems comprised by operands 2, 5 and 10 show the
lowest error rates. This is also true for operand errors with split 2. One can verify that the error
rates are also not uniformly distributed over all problems. Five problems are less affected from
the effects that were described above. We can observe a slightly higher error rate for five
problems that involve an operand greater than 5 though. It can be argued that the difficult
operands account for this effect.
Looking further into operand errors with split 1, that account for the majority of errors, it can be
observed that the larger operand neighbours are more frequently responsible for the cause of
errors than the smaller ones. In other words, learners tend to choose a value greater than the true
result of a problem rather a decremented one; e.g. for a given problem 4 * 8 the errors such as
36 = 4 *9 are more probable than 28 = 4 * 7. Figures 2C and 2D show this finding. We
emphasize that this is a valid prediction for all simple multiplications. Looking further through
each multiplication individually, we observe some exceptions such as 9 * 7 and 7 * 9 where a
decremented operand is rather due. Furthermore the tie problems seem to follow the same rule as
can be seen in figure 2D.
Considering the operand errors with split 1 and incremented operands, the next step was to
analyse which operand accounts for the error. More specifically, to investigate which one is
incremented: the larger or the smaller operand. Our analysis shows that the mean probabilities
for the set of larger and smaller operands are extremely close to each other, so that we can not
claim that the relative size of operands plays an important role. Figures 2E and 2F show this
comparison for each multiplication problem. As an example it can be seen that 8 * 4 or 4 * 8
show a very high error rate, meaning that the most probable false answer in this case was
36 = 9 * 4.
5.2 Operand Intrusions and Consistency Errors:!Operand intrusion errors occur when an
operand intrudes into the result. Figure 3 depicts the error rates for the first operand A and the
second operand B respectively. In general the probability of an intrusion for the second operand
B is higher than for the first operand A. While no specific pattern can be found within the set of
simple multiplication problems, it can be observed that some operands reveal a higher
probability relatively to other problems. For instance in case of the first operand intrusion,
specially the operand A = 4 shows a probability over 10% while multiplied by difficult operands
B ∈ {7,8, 9}
. Interestingly
A ∈ {4, 6, 7,8}
are more often intruded to the results while multiplied
by B = 9. In case of second operand intrusion, B = 6 reveals a probability of 12% while being
multiplied by difficult operands
A ∈ {7,8, 9}
. It is followed by
A ∈ {3, 4}
multiplied by B = 8. In
both cases, first operands
A ∈ {6, 7,8, 9}
play a stronger role in operand intrusion compared with
other operands.

E-iED 2014 ImmersiveEducation.org
PROCEEDINGS 151
!
Figure 3. Probabilities (error rates) of an operand intrusion error for each simple multiplication
problem. Figures 3A and 3B show the error rates for the first operand A and the second operand
B respectively.
Considering the decade and unit consistency errors, we could find no clear pattern in the
multiplication table. The probability of error occurrence related to decade consistency is
relatively higher than unit consistency. Decade consistency errors are specially more probable if
both operands are greater than 5 and are unequal. The reason for this could be explained by the
problem-size effect.
5.3 Problem Size Effect:!Problem-size is the sum of the operands and expresses how large the
problem is. Figure 4A shows the error rate (of any type) against increasing problem size. As the
problem size increases, the error rate has also a tendency to increase. However there is no
continuous ascending course of error rate. As predicted in [5, 7, 10] the tie problems can be
answered faster and more accurate compared with other problems, also while the problem size
increases. We see here that the tie problems have a different course by ascending problem-size.
While the error rates for all other error types increases, in tie problems a decrease is observed.
This can be claimed only upto problem size 25, due to the fact that the provided dataset for the
analysis is restricted. Furthermore, the error rates for the tie problems have a local minimum at
5 * 5, 10 * 10 and 15 * 15, which can be argued by the 5 effect and the easy 10-problems.
We analysed each error type described in table 1 against the problem size individually. Decade
and unit consistency errors increase by ascending problem-size. Figures 4B and 4C depict the
unit and decade consistency errors against problem size respectively. All other analysed error
types do not reveal any increasing course and stay constant within a close probability interval. As
an example, the operand error with split 1 is depicted against the problem size in figure 4D. It
can be observed that the error rate varies between 5% and 10% and comes even down to about
3% at problem size 25. In sum, considering the set of analysed error types, the problem-size
effect can be defined according to the unit and decade consistency errors.

E-iED 2014 ImmersiveEducation.org
PROCEEDINGS 152
Figure 4. Error probabilities (fraction of errors) against problem size. Problem size is calculated
as the sum of the operands A + B. The largest considered operand is 15. Ties and non-ties are
depicted separately. Figure 4A shows the general round error probability; that is the fraction of
rounds where at least one error of any type has been made against problem size. Figure 4B
depicts the unit consistency errors and 4C shows the decade consistency errors against the
problem size.
References and Notes:
1. M. Ebner, M. Schön, in Why Learning Analytics in Primary Education Matters!, C.
Karagiannidis, S. Graf, Eds. (Bulletin of the Technical Committee on Learning Technology
2013) vol. 15, issue 2, pp. 14-17.
2. M. Schön, M. Ebner, G. Kothmeier, It's Just About Learning the Multiplication Table
(Proceedings of the 2nd International Conference on Learning Analytics and Knowledge
ACM, New York, NY, USA 2012) S. Buckingham Shum, D. Gasevic, R. Ferguson (Eds.) pp.
73-81. DOI=10.1145/2330601.2330624.
3. B. Taraghi, M. Ebner, A. Saranti, M. Schön, On Using Markov Chain to Evidence the
Learning Structures and Difficulty Levels of One Digit Multiplication (proceedings of the 4th
International Conference on Learning Analytics and Knowledge, Indianapolis, USA 2014)
pp. 68-72.

E-iED 2014 ImmersiveEducation.org
PROCEEDINGS 153
4. B. Taraghi, A. Saranti, M. Ebner, M. Schön, Markov Chain and Classification of Difficulty
Levels Enhances the Learning Path in One Digit Multiplication, P. Zaphiris, A. Ioannou,
Eds. (Springer LNCS 2014), vol. 8523, pp. 322-333.
5. J. I. D. Campbell, Mechanisms of simple addition and multiplication: A modified network-
interference theory and simulation. Mathematical Cognition. 1, 121-165 (1995).
6. T. Verguts, W. Fias, Interacting neighbours: A connectionist model of retrieval in single-digit
multiplication. Memory & Cognition. 33, 1-16 (2005).
7. J. I. D. Campbell, On the relation between skilled performance of simple division and
multiplication. Journal of Experimental Psychology: Learning, Memory, & Cognition. 23,
1140-1159 (1997).
8. J. I. D. Campbell, J. M. Clark, in Cognitive number processing: An encoding-complex
perspective, J. I. D. Campbell, Ed. The Nature and Origins of Mathematical Skills
(Amsterdam: Elsevier Science 1992) pp. 457-491.
9. N. J. Zbrodoff, G. D. Logan, in What everyone finds: The problem size effect, J. I. D.
Campbell, Ed. Handbook of Mathematical Cognition (New York: Psychology Press 2004)
pp. 331-346.
10. J. A. LeFevre, G. S. Sadesky, J. Bisanz, Selection of procedures in mental addition:
Reassessing the problem size effect in adults. Journal of Experimental Psychology:
Learning, Memory & Cognition, 22, 216-230 (1996)
11. T. Verguts, W. Fias, Neighborhood effects in mental arithmetic. Psychology Science, 47,
132–140 (2005)
12. M. S. Seidenberg, J. L. McClelland, A distributed, developmental model of word recognition
and naming. Psychological Review, 96, 523-568 (1989)
13. F. Domahs, M., Delazer, H., Nuerk, What makes multiplication facts difficult: Problem-size
or neighborhood consistency? Experimental Psychology, 53, 275–282. doi:10.1027/1618-
3169.53.4.275 (2006)
14. B. Taraghi, A. Saranti, M. Ebner, V. Müller, A. Großmann, Towards a Learning-Aware
Application Guided by Hierarchical Classification of Learner Profiles. Journal of Universal
Computer Science, Special Issue on Learning Analytics, in print (2015)
15. C. Bishop, Pattern Recognition and Machine Learning (Springer Science and Business
Media, LLC, New York, USA 2006), pp. 424-430