Content uploaded by Marina Lepp
Author content
All content in this area was uploaded by Marina Lepp on Sep 22, 2015
Content may be subject to copyright.
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
1
INTEGRATING RULE-BASED AND INPUT-BASED
APPROACHES FOR BETTER ERROR DIAGNOSIS IN
EXPRESSION MANIPULATION TASKS
R. PRANK∗, M. ISSAKOVA, D. LEPP, E. TONISSON and V. VAIKSAAR
Institute of Computer Science, University of Tartu,
Liivi 2-306 Tartu, 50409, Estonia
∗E-mail: rein.prank@ut.ee
T-algebra is a project for creating interactive problem solving environment
for basic school expression manipulation exercises: calculation of the values of
numerical expressions; operations with fractions; solving of linear equations,
inequalities and linear equation systems; operations with monomials and poly-
nomials. This article describes and motivates solution step interface and error
diagnostics developed in T-algebra.
Keywords: Problem solving environment; Error diagnostics; Elementary alge-
bra.
1. Introduction
Expression manipulation skills are important for solving tasks in practically
all fields of mathematics. Therefore the students solve in the school hun-
dreds of technical exercises with fractions, monomials, polynomials, equa-
tions and systems of equations. However, we are often not satisfied with
the results of learning in this area. Even many university students at the
faculties of mathematics or computer science mix up elementary conversion
rules and are unable to avoid numerous mistakes. One of the reasons for
this seems to be the fact that the students do not get necessary feedback
about their work during the school mathematics classes. The solutions of
expression manipulation exercises are long and contain many details. The
teachers are unable to supervise the students’ training in real time, discover
all the mistakes and correct them. The need for quick analysis of large vol-
umes of information indicates that the training and testing of expression
manipulation skills could be improved by using computerized environments.
This paper describes the design ideas and error diagnosing possibilities
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
2
of interactive learning environment T-algebra developed at the University
of Tartu beginning from 2004. The goal of this project is to create an
environment for solving expression manipulation tasks in four areas of basic
school mathematics:
•calculation of the values of numerical expressions
•operations with fractions
•solving of linear equations, inequalities and linear equation systems
•operations with monomials and polynomials.
T-algebra implements more than 50 problem types. Some of them are
integrated types that summarize the content of an entire chapter in text-
book (“Solve linear equation”), some others are devoted to one single step
in algorithm (“Move terms with unknown to left and constants to right”).
Our aim is not to design a comfortable conversion machine to help strong
students to solve advanced problems. Some existing programs1,2 do this
well enough. We try to support the weaker part of the students who often
do not know the algorithms for solving the standard tasks, have problems
with understanding the syntax of expressions, and make many mistakes
when they execute the algorithm steps. Designing the solution dialogue
for T-algebra, we have been guided by the didactical principle that all the
necessary decisions and calculations at each solution step should be made by
the student, and the program should be able to give hints and to understand
mistakes.
In Section 2 of this paper we discuss suitability of rule-based and input-
based dialogue schemes for diagnostic purposes. Section 3 describes our
three-component solution step and input modes in T-algebra. Section 4
describes how the information entered by the students is used for diagnosing
mistakes made at different stages of a solution step and what difficulties
we have with diagnostics. In Section 5 the results and problems for further
research are summarized.
2. What Type of Learning Environment Do We Need?
The history of learning environments for expression manipulation is consid-
erably shorter than the history of elementary counting drills or the success
story of dynamic geometry. Good programming tools for writing WYSI-
WYG expression editors did not exist for a long time and even the computer
algebra systems (CAS) had linear command-line input. Today we have al-
ready some more or less commonly accepted expression editing standards
and these standards are sufficiently well supported by programming envi-
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
3
ronments. As a result, practically usable programs for students are avail-
able.
We can point out two types of existing interactive environments with
essentially different expression manipulation dialogue:
(1) Rule-based manipulation environments.
(2) Input-based manipulation environments.
A typical rule-based environment is MathXpert.1Using this program,
the student begins each conversion step by marking some subexpression to
be changed. The program displays then a menu with a list of rules, which
are applicable to the marked subexpression. Then the student selects a
rule and the program applies the rule to the marked subexpression (or to
suitable parts of the marked subexpression) and writes the result together
with unchanged parts of the previous expression to next line. MathXpert
has hundreds of rules of different capacity and enables to construct solu-
tions of different granularity. A similar scheme for conversion steps is used
in programs developed in the framework of the Stanford Educational Pro-
gram for Gifted Youth,2but this program asks in addition the student to
fill sometimes some input boxes with coefficients, exponents, etc. The cur-
rent version of Cognitive Algebra Tutor by Carnegie Learning3is likewise
built on applying rules. AlgeBrain4is a small Intelligent Tutor for solv-
ing equations in rule-based interface. The use of computer algebra systems
belongs also to rule-based work. But computer algebra systems have a seri-
ous disadvantage in comparison with programs written for educational use.
They have very powerful commands for solving most task types of school
mathematics in one step but do not have sufficiently detailed commands
for construction of stepwise solutions. Such choice of rules is characteristic
even for systems like WIRIS5that are designed and advertised especially
for using in schools.
Input-based systems use paper-and-pencil-like dialogue design where a
transformation step consists mainly of entering the next line. For exam-
ple, Aplusix6copies the content of previous line (expression, equation or
system of equations) to the next line and the student should edit it into
the result of the step. The program displays between two lines the indi-
cator of equivalence, giving the student feedback about correctness of the
step. MATH-TEACHER7is another example of input-based solution envi-
ronment but in this program the expressions are entered in old-fashioned
linear form. An early version of Cognitive Algebra Tutor8used also purely
entering of the result and the program tried to figure out what step was
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
4
performed.
Let us now compare the dialogues of both types with the range of deci-
sions that the student should make at each conversion step. At each solution
step the student should
(1) Choose a transformation rule corresponding to a certain operation in
the algorithm for current task (or some simplification or calculation
rule known earlier).
(2) Select the operands (certain parts of expressions or equations) for this
rule.
(3) Replace them with the result of the operation.
Some more “creative” tasks (such as factorization or integration) are
taught in less algorithmic style but the solutions are expected to consist of
operations of the same structure.
When solving a task in the rule-based system, the student’s role is to
know or to invent the solution strategy. The program executes all opera-
tions as a black box. In such scheme the learning of details of operations is
passive and many typical mistakes are simply impossible. In order to learn
to execute new operations and to test the expression manipulation skills
we need an environment where all the decisions and calculations would be
made by the student. It is clear that for such aims the environment should
include input of new expression (or of the changed part of the expression)
by the student.
In case of input-based dialogue the whole solution step is performed by
the student. The student has the possibility to undertake whatever steps
and to make arbitrary mistakes. His input is restricted only by syntax of
expressions. The computer is now in the same situation as the teacher who
should check a solution script on the paper: there is no explicit information
about the student’s decisions at the first two stages of the steps. Without
knowing what operation was applied to what part(s) of previous expression
and without restrictions on the number of operations applied during one
step, it is very hard to diagnose errors more precisely than “expressions are
not equivalent”.
Experimental systems for diagnostics of algebraic errors in such situa-
tion were created in Intelligent Tutoring community already in the seventies
of the last century. The programs used libraries of correct and buggy con-
version rules (extracted by teachers from students’ written solutions). If
the entered expression was incorrect, then the program tried to model its
underlying derivation, combining different buggy rules with correct ones. A
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
5
good presentation about the ideas of that period is given in Ref. 9. Some
subsequent trials are described in Ref. 10 and Ref. 11. It is clear that in
expression manipulation the students really use some mixture of correct
and buggy rules and therefore this approach seems to be adequate. Nev-
ertheless we have not yet seen completed programs producing sufficiently
detailed analysis of mistakes. Beside difficulties in detection of the source of
the error, the systems with unrestricted extent of the step have also prob-
lems with providing intelligible feedback. The authors of the input-based
version of Cognitive Algebra Tutor complained that “. . . the student’s error
might well have occurred at some intermediate step that the students were
no longer fixated upon. It was very difficult to communicate to the stu-
dent what the problem was.” (Ref. 8, p. 42). After a period of silence quite
serious new attempts of application of classical AI methods in expression
manipulation exercises were reported at ITS 2006. The team of APLUSIX
has started building rule-based diagnostics.12 C.Zinn13 presented the main
ideas of rule-based reasoning and diagnostics of another system, SLOPERT,
developed in Saarbr¨ucken for tutoring of differentiation.
The task of error localization/diagnostics can be simplified when the
program sets constraints to student’s input. Some programs prescribe the
form of entered expression, using less or more detailed textual instructions
and/or splitting the input area in more than one specialized input boxes
(for instance, separate boxes for numerator and denominator of fractions).
Such measures are sometimes used already in quite primitive commercial
products that then compare the student’s input piecewise as strings with
‘correct answer’. But such interface can be combined with more intelligent
analysis. Similar effect can be achieved when the program tells what rule
should be applied or the student has to choose the rule from the menu.
For example, interactive exercises on addition of fractions in WIMS server
(Ref. 14, OEF fractions/Guided addition) use both ways. For each step the
student has to choose first the rule from the menu that contains together
with correct choice also some buggy and some unsuitable rules. After that
he has to enter different parts of the fractions into different boxes. But it
seems that at every step the menu proposes only one correct way to continue
the solution.
Several successful attempts have been made to incorporate computer al-
gebra systems in error diagnosis and automated creation of solutions in ex-
pression manipulation tasks. The abovementioned Stanford program2used
Maple. Several recent quiz systems are built on CAS. For example, AIM15
uses Maple and STACK16 uses Maxima. The quiz systems ask the stu-
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
6
dent to enter the final answer of the task and therefore in general they are
not the environments for stepwise solution. But the methods developed for
questions about conversions can be suitable for arbitrary conversion steps.
It is clear that the CAS can provide for quiz system the tools for checking
of equivalence (equality) of student’s answer and correct (CAS-computed)
answer. Computer algebra systems have also utilities for checking whether
the answer belongs to required syntactic category of expressions. For exam-
ple, the factorization questions in AIM and STACK are designed to include
checks whether the entered expression has completely factorized form, and
they denote it even when the answer is completely wrong. Using the pro-
gramming environment of corresponding CAS, it is possible for the authors
of questions to implement nontrivial diagnostics for more precise assess-
ment and feedback. But the required detailed work on a large number of
question (task) types is only in the beginning stages.
Some design ideas of T-algebra originate from our earlier work in Tartu.
In 1988–91 we developed a program package for exercises in Mathematical
Logic.17 One of the programs was interactive environment for stepwise solu-
tion of formula manipulation exercises in Propositional Logic (expression of
formulas through {&,¬},{∨,¬} or {⊃,¬} and finding normal forms). The
first version of this program worked with a pure input interface. At every
step the student typed on the next line a new formula (having some copy-
paste possibilities). The program checked the syntax, equivalence with the
previous line and whether the target form of the expression was reached.
We saw that the errors of misunderstanding of the order of operations were
most dangerous. We were generally unable to diagnose them without ex-
plicit information about the ob ject of conversion. In the second version18
the step dialog was built using an Ob ject-Action scheme. The student had
to mark some subformula and to convert it then to the result of the step.
The strings before and after the marked subformula were copied automat-
ically. For the second substep the program had two working modes: input
and selection of a conversion rule from the menu. As a result, the program
was able to verify separately the selection of operand and the performed
conversion. This addition of a marking phase gave us a level of feedback
that was sufficient for that group of users (second-year students) and we
did not have any further need to make it more precise.
3. Solution Step Dialogue in T-algebra
Our main concern in designing the dialog for T-algebra19 has been to create
preconditions for being able to diagnose and to give understandable feed-
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
7
back about mistakes at all three stages of a solution step. We have done this
rather straightforwardly: the student enters not only the result of the step
but also his first two decisions. At each solution step the student performs
three actions:
(1) Selects an operation from the menu.
(2) Marks the operand(s) in expression.
(3) Enters the result of operation.
Unlike many other programs, T-algebra requires precise marking of
operands for diagnostic purposes. For example, for the operation “Com-
bine like terms” the student should mark only those terms that will be
actually combined. (The program allows the preceding pluses and minuses
to be marked or not.) For reducing the fraction where numerator and/or
denominator consist of several factors, only those factors that will be actu-
ally reduced should be marked. Accordingly, the editor of T-algebra enables
to mark more than one piece of the expression. Precise marking requires
from the student more work at the second substep. However, leaving the
parts that will not be changed unmarked means also that these parts will
be copied onto the next line automatically and this reduces the amount of
work at input.
Figure 1 demonstrates the solution window where the student has se-
lected the rule Multiply/Divide monomials and has already marked two
terms for multiplying.
Fig. 1. Solution window during execution of the first two stages of a step.
The first two substeps are not separated in T-algebra. At the begin-
ning of our project we presumed that the three actions should be made in
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
8
the abovementioned order and we named our interaction scheme “Action-
Object-Input”. When we started our first experiments with students, we
wrote the order of substeps on blackboard. However, the program acciden-
tally enabled to mark one subexpression already before selection of the rule
(for copying). In the classroom we saw that the students often wanted to
mark the operand(s) before selecting the rule from the menu. They sus-
pected that the impossibility to mark the second operand before selection
of the rule was a mistake in the program (we had told them that the pro-
gram is not ready yet). Analyzing this experience, we decided to change the
dialogue and to amalgamate the first two substeps. The student can mark
the operands before, after, or even before and after selection of the rule.
During the marking process the program checks only that each marked
piece is a syntactically correct subexpression. Proper checking of Action
and Object begins only when the student confirms both stages by clicking
the green check button on the virtual keyboard. The student has to correct
his first two decisions until they are accepted by the program.
When T-algebra accepts Action and Operands, the unmarked parts from
the previous line will be copied onto the next line and the program asks
the student to enter the result of applying the selected operation to the
marked operands. T algebra has three input modes for entering the result of
the step: Free input,Structured input and Partial input. In different input
modes T-algebra offers different number and types of boxes for entering
the result. The three different modes of an input stage following from the
situation presented in Fig. 1 (after the third monomial is also selected for
multiplying) are pictured in Fig. 2. The colored boxes are initially empty
and the student enters their content during the stage.
Fig. 2. Three input modes.
In the free mode, the entire resulting subexpression is entered into one
single box. In case of operations where the result is not a single subexpres-
sion (some operations with fractions and equations) T-algebra offers two
boxes. Free mode is implemented in principle in the same way for all the
rules of T-algebra. In the free mode the student has almost the same degree
of freedom as when writing solution steps on paper. All decisions about the
structure of the result and about the values of component parts are made
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
9
by the student. The student has the possibility to make almost arbitrary
mistakes. Such mode is indispensable for testing the knowledge of students,
even if the amount of input is high.
Besides free input we have also sought working modes where the pro-
gram would support the student with some suggestions and/or would reduce
the amount of keyboard work by displaying some obvious parts of the result
automatically. In his Master thesis in 2003 D. Lepp tried to design natural
dialogues for different operations with monomials and polynomials.20 We
saw that if we follow the essence of each operation too punctually we get a
program where every rule has its own user interface. Therefore we decided
to design two standardized additional input modes.
In the structured mode, the program prescribes the structure of the
resulting subexpression(s), offering a set of boxes that enable to enter spe-
cific parts of the expression: signs, numbers, variables, monomials, powers,
etc. For this T-algebra uses the received information about the intentions
of the student, calculates the correct result and produces corresponding
boxes for filling out. Nevertheless it is still possible not to write unary plus,
coefficient or exponent 1, and even to leave all boxes empty if the member
has coefficient zero. In some cases we do not want to prescribe the com-
plete structure of the result. For example, in the case of multiplication of
polynomials we do not want to predict the number of resulting monomials.
The program creates only some monomial-structured groups of boxes and
the virtual keyboard contains a button for adding the next group. Input in
prescribed structure helps the student to some extent. On the other hand,
it gives the possibility to formulate more precise error messages and to
indicate the position of errors.
In the partial mode, T-algebra fills some parts of the structured result
automatically the user has to enter only the critical parts of the result. For
example, in the case of combining like terms, only the sign and coefficient
of the resulting monomial should be entered; variables and their powers are
filled automatically. Input modes of T-algebra are described more precisely
in Ref. 21.
There are a few transformations where the way they are performed in
the first school exercises necessitated some additions to our general input
scheme. For instance, in the case of addition of fractions in two structured
modes we decided to ask and check separately the common denominator
and extenders before entering the complete result (Fig. 3). The rule for
multiplication/division of both sides of equation asks about corresponding
factor, etc. A more detailed description of the solution dialogue together
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
10
with exceptions is given in Ref. 22.
Fig. 3. Input of intermediate result when adding/subtracting fractions with different
denominators.
In the current version of T-algebra, the input mode for each exercise is
predefined by the teacher/author in a problem file and cannot be changed
by the student. In the future it would be conceivable to change the mode
dynamically: if the student does not cope with work in the free mode the
program would deliver the tasks in more detailed modes for a certain period.
After description of the step dialogue we can appraise how much work
our interface requires from the student. It is obvious that in the rule-based
environments the student mostly does not enter the expressions and so the
amount of input is considerably smaller and the input itself is simpler. Let
us compare the work in T-algebra with pure-input interface.
In our interface the student should: select a rule, mark operand(s) in
initial expression and enter the changed part of the expression. The first
substep requires only one click of the mouse. The third substep is required
in a pure-input interface too. Consider our second substep. In pure-input
interface the student should form the next-line expression from empty input
box or from previous-line expression. In the first case he should mark and
copy the unchanged parts from the previous line one by one or enter them
from keyboard. In the second case he should mark and delete all changed
parts. So we see that marking of all bounds of operands is concealed in a
pure-input interface as well (if not replaced by total input of the whole new
line). Consequently our interface requires practically the same amount of
input work as pure-input interface.
The first experiments with students already demonstrated that our con-
cept of a three-component interface is understandable to students and
teachers. Some difficulties arise with students who are not confident in
terminology and have troubles with understanding the names of rules.
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
11
4. What Can We Diagnose in A-O-I-Interface?
4.1. What Can Be Decided about the Selected Operation?
It is quite easy to decide that the selected operation:
•cannot be applied to any subterm(s) of the current expression
•does not correspond to the algorithm to be learned (if such exists for
actual problem type).
However we prefer not to check the selected rule separately. When the
student confirms both the selection and marking then T-algebra checks the
possibility to apply the selected rule to the marked subexpression(s).
In fact, checking of the possibility is implemented in T-algebra for other
purposes. If the student selects an impossible rule and asks for help for
marking the operands then T-algebra responds that the application of the
selected rule is impossible. However after confirming an impossible rule
together with some marked subexpression(s) the message explains what
should be the form of the operand(s) for the selected rule.
Concerning the second issue we intend to add in the future warning
messages if the selected operation does not correspond to the “official”
algorithm and is not a simplification or calculation rule. After such message
the student can continue or cancel the rule.
4.2. Checking the Operation and Marking of Operands
Together
T-algebra diagnoses the following errors of selection of operands:
(1) Marked term is not a syntactically correct expression.
(2) Marked expression is not a proper subexpression (order of operations
misunderstood).
(3) Operand does not have the form required for selected rule.
(4) Operands do not satisfy the compatibility requirements (are not like
terms, etc.).
(5) Operands do not satisfy the location requirements (do not belong to
the same sum, fraction, product, etc.).
(6) Number of marked operands does not correspond to selected operation
(only one needed, at least two needed, etc.).
The first two issues are checked already when the student marks pieces of
the expression as operands. The next four depend on the selected rule and
are checked when the student confirms both rule and operands.
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
12
It is important to note here that our wish to diagnose errors 4–6 has lead
us to certain restriction in making expression manipulation steps: T-algebra
does not permit parallel use of rule in one step (collection of several groups
of like terms, multiplication of more than one group of terms, etc.). This is
in slight contradiction with mathematical practice on paper and gives rise
to some error messages of types 4–6 in the beginning of use of T-algebra but
these problems disappear quickly. Exclusion of parallel conversions makes
the solution longer but does not cause additional typing work because the
program copies unchanged parts of the expression automatically to the next
line. It also improves the readability of the resulting solution script.
Errors of types 1 and 6 can be often simply oversight errors where the
student has marked something else than he wanted (only one of two brack-
ets included, no operands marked at all, etc.). In the pure-input interface
where the student does not have to mark arguments he cannot make such
errors and therefore produces a correct step. The same can sometimes be
true for other error types as well but often the reasons are much more seri-
ous: the student does not understand the order of operations, the meaning
of some mathematical expression, or the nature of some expression manip-
ulation operation/rule. For example, some students understand −3(2x+ 3)
like addition between −3, 2xand 3, as if it would be written −3 + (2x+ 3).
This mistake is actually not a mistake in moving terms to other side, but
a mistake in opening parentheses. However, in T-algebra this mistake was
discovered during the selection of the parts for the rule Move terms to
other side (Fig. 4). In case of pure-input interface conversion of irrelevant
operands usually gives an expression that will be qualified as not equiva-
lent to the previous line. Yet even a very intelligent program could hardly
give a more precise diagnosis. Diagnosing the mistake already before the
input of the result tells the student where the error really is and prevents
meaningless input work.
Some error situations are caused by the student’s poor knowledge of
precise terminology or underestimation precise use of terms. Sometimes
they try to apply the rule called Multiply monomials to polynomials or the
rule Add numbers to the terms containing variables. On the one hand we
hope that demanding from the students use of correct names combined with
their need to understand the help texts and error messages leads to better
use of mathematical terms. On the other hand we understand that there is
yet a long way to standardization and the menus of the existing programs
are only starting-points. A possibility of dialogue simplification would be
to put in the menu only one item Multiply while applying different rules
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
13
Fig. 4. Student mistake and T-algebra error message in moving terms to other side of
equation.
depending on the marked expressions.
A second long-term standardization problem is the extent of al-
lowed/supposed preprocessing. When the student performs the conversions
on paper, our usual practice does not prohibit making some small con-
versions before the “main” operation of the step. For example, in case of
collection of like terms the student can convert the monomials to normal
form without doing this operation in writing. While we have never seen any
exact regulations what preprocessing is allowed for a particular operation
and what is not, any rule-based system should implement some decision
procedure for this. There exists a trivial solution to the approval problem
where the operations are just formal rewrite rules. In order to avoid long
chains of trivial steps we have tried to permit some reasonable prepro-
cessing. For example, T-algebra allows application of Collect like terms to
2xyx, 3x2yand −yxx. Automated solution procedure writes the result in
normal form. If the student enters the result in Free or Structured mode,
any monomial equivalent to the program’s answer will be accepted.
Different programs display the negative judgments of rule approval pro-
cedure in different form. If the student wants to collect like terms in Math-
Xpert and marks a subexpression then it is possible that MathXpert does
not display the command Collect ±terms in the pop-up menu or collects
only a part of expected terms. Then the student can rethink whether the
terms can be collected after some preprocessing by means of other rules or
they cannot be collected at all. If T-algebra diagnoses that the rule cannot
be applied to (some part of ) the marked subexpression then the feedback is
a bit more precipitous: an error message (usually explaining what form the
argument(s) of this rule should have) and increasing of the corresponding
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
14
error type counter. Therefore it is desirable to separate as clearly as pos-
sible the real mathematical mistakes from the breaches of some provisory
preprocessing conventions in the program.
Today we still need some time for experiments on both: making the con-
ventions more accurate and making the diagnosis more distinguishing. We
do not claim that our counters of mistakes can be used safely for assessment
purposes. At the current stage the program also saves the error situations
and the teacher has the possibility to weight the mistakes.
4.3. Checking Entered Result of Conversion
Knowing the actual operation and the actual operands enables to analyze
the input at the third stage much better. In many cases the correct input
is now uniquely or almost uniquely defined by known data. If the entered
result is not correct then the error messages can be formulated in terms of
a specific rule and its operands/results.
T-algebra diagnoses the following errors after input of a conversion re-
sult:
(1) Entered term is not a syntactically correct expression.
(2) Entered subexpression should be preceded by a sign.
(3) Entered subexpression should be put in parentheses (order of opera-
tions).
(4) Entered subexpression does not have the required structure (is not a
proper fraction, is not a monomial, has wrong number of members,
etc.).
(5) Entered subexpression is not equivalent with marked part.
(6) Concrete parts of input (sign, coefficient, variables, exponents, denom-
inator, etc.) do not have right value.
(7) Selected operation with marked terms is not performed (nothing re-
duced, terms are not moved to other side, etc.).
The first issue concerns only the entered expression. Issues 2 and 3
consider the syntactical compatibility of entered term(s) with unchanged
part of the expression. Error 3 can occur when the main operation of the
result has lower priority than the main operation of the marked part. For
example, (a+b)(c+d) = ac +ad +bc +bd but 3(a+b)(c+d)6= 3ac +ad +
bc +bd and thus in such context the parentheses should be put around the
result of multiplication. Following our didactic principles we do not want to
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
15
prevent this mistake automatically because this error is very frequent even
for university students.
Issue 4 is checked by T-algebra before issue 5 (equivalence). For example,
the result of multiplication of monomial with polynomial should be the sum
of monomials and the result of combining like monomials should be one
monomial. In case of structured and partial input modes such requirements
or part of them can be fulfilled automatically. The program allows the order
of the members of sums and products to be changed. Although the boxes
in structured and partial modes prescribe the structure of expression, it is
allowed to leave some boxes empty (unary plus, the coefficient or exponent
that equals 1, the monomial with coefficient 0, etc.).
The message about mere non-equivalence is issued by T-algebra in cases
when the program is unable to give more detailed diagnosis of category 6.
At the current stage in free input mode T-algebra checks for equivalence
or non-equivalence (issue 5) of the entered expression with the correct one.
In simplest cases (for example, when the result is a single monomial) T-
algebra checks also the correctness of specific parts of the result (issue 6).
In structured and partial input modes the result is of a required structure
and T-algebra is able to check for correctness of different parts (Fig. 5).
Fig. 5. Student mistake and T-algebra error message in opening parentheses.
For some rules the right syntactic structure and equivalence with previ-
ous line do not guarantee that the selected operation is really performed. In
such cases T-algebra checks this issue additionally. For example, T-algebra
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
16
does not accept reducing 6/12 to 60/120 or 5/10 or 6/12.
Issue 4 (right structure, number of members, etc.) is associated with
problems of allowed or prohibited postprocessing of the result of “main”
operation of the step. Working with paper and pencil, we try to economize
the steps. Strong students want to combine like terms already within the
step of multiplication of two polynomials or within moving the terms to
other side of equation.
Some additional information about error diagnosis in T-algebra can be
found in Ref. 23 and Ref. 24.
5. Conclusions
Developing a three-component expression manipulation interface, we have
created a solution dialogue that
(1) Is intuitively understandable for the students.
(2) Requires generally the same amount of keyboard/mouse work as a pure
input interface.
(3) Allows students to make practically all expression manipulation mis-
takes.
(4) Allows the program to locate the substep of solution step (choice of op-
eration, choice of operands, execution of operation) where the student
has made a mistake.
(5) Allows to point in error messages to the actual location of mistakes.
(6) Gives the teacher (in the future also the program) the possibility to
choose such input mode that supports the student’s actions.
Our project seems to be the first attempt to implement detailed and
substep-oriented error diagnostics for expression manipulation exercises.
Diagnostics in T-algebra is based on purely syntactic analysis of entered
information. We check the rule, operands and result separately, pair-wise,
and all together. Experiments with students demonstrate that this enables
formulating helpful feedback. A natural expectation would be to use this
automated detailed error diagnostics for assessment (assigning to each error
type some penalty and multiplying penalties by the counters of mistakes).
For this purpose we should learn to detect the cases when some error mes-
sage is caused not by real mathematical mistake but by missing knowledge
about the restrictions established in the program (for example, prohibition
to collect two groups of like terms in one step).
Obviously our decisions and ideas need some years of practical classroom
work before they can be finally confirmed. In addition, experiments are
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
17
needed to solve some problems concerning standardization (for example,
standardization of nomenclature of rules). Starting from the school year
2006–2007 tens of teachers in Estonian schools use T-algebra for practicing.
The results of school trials and teacher experiences will contribute to and
support further development of T-algebra.
Acknowledgments
Programming of T-algebra is financed by the Estonian School Computeri-
zation Foundation “Tiger Leap”. The authors are financed also by Targeted
Financing grant SF0182712s06 of the Estonian Ministry of Education and
Research and by Estonian Doctoral School in Information and Communi-
cation Technologies.
References
1. M. Beeson, Design Principles of Mathpert: Software to support education in
algebra and calculus, in Computer-Human Interaction in Symbolic Compu-
tation, ed. N. Kajler (Springer, 1998) pp. 89–115.
2. R. Ravaglia, T. Alper, M. Rozenfeld and P. Suppes, Successful pedagogical
applications of symbolic computation, in Computer-Human Interaction in
Symbolic Computation, ed. N. Kajler (Springer, 1998) pp. 61–88.
3. Cognitive Tutor by Carnegie Learning, Inc. http://www.carnegielearning.
com.
4. S. R. Alpert, M. K. Singley and P. G. Fairweather, Deploying Intelligent
Tutors on the Web: An Architecture and an Example. International Journal
of Artificial Intelligence in Education 10, 2 (1999) pp. 183–197.
5. WIRIS, http://www.wiris.com/.
6. J. Nicaud, D. Bouhineau and H. Chaachoua, Mixing microworld and cas
features in building computer systems that help students learn algebra, In-
ternational Journal of Computers for Mathematical Learning 5, 2 (2004) pp.
169–211.
7. MATH-TEACHER, http://www.mathkalusa.com/index.html.
8. J. R. Anderson, C. F. Boyle, A. Corbett and M. W. Lewis, Cognitive modeling
and intelligent tutoring, Artificial Intelligence 42, 1 (1990) pp. 7–49.
9. D. Sleeman and J. S. Brown (eds.), Intel ligent Tutoring Systems (Academic
Press, 1982).
10. M. Quigley, A Simple Algebra Tutor, Journal of Artificial Intelligence in
Education 1, 1 (1989) pp. 41–52.
11. H. U. Hoppe, Deductive error diagnosis and inductive error generalisation for
intelligent tutoring systems, Journal of Artificial Intelligence in Education 5,
1, (1994) pp. 27–49.
12. J. Nicaud, H. Chaachoua and M. Bittar, Automatic Calculation of Students’
Conceptions in Elementary Algebra from Aplusix Log Files, ITS 2006 Pro-
ceedings, LNCS 4053 (2006) pp. 433–442.
May 13, 2007 20:14 WSPC - Proceedings Trim Size: 9in x 6in sce4
18
13. C. Zinn, Supporting Tutorial Feedback to Student Help Requests and Errors
in Symbolic Differentiation, ITS 2006 Proceedings, LNCS 4053 (2006) pp.
349–359.
14. WIMS server, http://wims.unice.fr/wims/en_home.html.
15. C. J. Sangwin, Assessing mathematics automatically using computer algebra
and the internet, Teaching Mathematics and its Applications 23, 1 (2004) pp.
1–14.
16. C. J. Sangwin, Assessing Elementary Algebra with STACK. International
Journal of Mathematical Education in Science and Technology (forthcoming).
17. R. Prank, Using Computerised Exercises on Mathematical Logic, Informatik-
Fachberichte 292, (Springer-Verlag, 1991) pp. 34–38.
18. R. Prank and H. Viira, Algebraic Manipulation Assistant for Propositional
Logic, Computerised Logic Teaching Bulletin 4, 1 (St Andrews Univ, 1991)
pp. 13–18.
19. R. Prank, M. Issakova, D. Lepp and V. Vaiksaar, Designing Next-Generation
Training and Testing Environment for Expression Manipulation, ICCS 2006,
Part I, LNCS 3991 (2006) pp. 928–931.
20. D. Lepp, Program for exercises on operations with polynomials, 6th Interna-
tional Conference on Technology in Mathematics Teaching (2003), pp. 365–
369.
21. M. Issakova, D. Lepp and R. Prank, Input Design in Interactive Learning
Environment T-algebra,Proceedings 5th IEEE International Conference on
Advanced Learning Technologies (IEEE, 2005) pp. 489–491.
22. M. Issakova, D. Lepp and R. Prank, T-algebra: Adding Input Stage To
Rule-Based Interface For Expression Manipulation, International Journal for
Technology in Mathematics Education 13, 2 (2006) pp. 89–96.
23. M. Issakova, Comparison of student errors made during linear equation solv-
ing on paper and in interactive learning environment, in Proceedings DES-
TIME-2006: Dresden International Symposium on Technology and its Inte-
gration into Mathematics Education (2006).
24. D. Lepp, Error Diagnosis in Problem Solving Environment Using Action-
Object-Input Scheme, in ITS 2006 Proceedings LNCS 4053, eds. Ikeda, Ash-
ley and Chan (2006) pp. 769–771.
