Content uploaded by Dimitra Tsovaltzi
Author content
All content in this area was uploaded by Dimitra Tsovaltzi
Content may be subject to copyright.
An Approach to Facilitating Reflection in
a Mathematics Tutoring System
Dimitra TSOVALTZI
Department of Computational Linguistics
Saarland University
dimitra@coli.uni-sb.de
Armin FIEDLER
Department of Computer Science
Saarland University
afiedler@cs.uni-sb.de
Abstract. In this paper we present an approach which enables both reflection
on the student’s own line of reasoning and active learning. We combine the
categorisation of the student answer against the expert domain knowledge, a
hinting process that engages the student in actively reflecting upon his
reasoning in relation to the experts reasoning, and clarification subdialogues in
natural language initiated by the tutor or the student on the task.
1 Introduction
Despite empirical evidence that natural language dialogue capabilities are necessary
for the success of tutorial sessions [1], only few state-of-the-art tutorial systems use
natural-language style interaction that requires menu-based input or exact wording of
the input [2, 3, 4]. In the DIALOG project [5], we aim at a mathematical tutoring
dialogue system that employs an elaborate natural language dialogue component. To
tutor mathematics, we need a formally encoded mathematical theory, means of
classifying the student’s input in terms of the knowledge of the domain demonstrated,
and a theory of tutoring.
1
To that end, we enhanced the mathematical ontology of the state-of-the-art
theorem prover ΩMEGA [6] by making explicit relations that can potentially be used
in the tutoring. Moreover, to classify the student’s input, we developed a
categorisation scheme for student answers, which draws on the mathematical
ontology.
As far as tutoring is concerned, we aim at a mathematical tutoring system that
is able to tutor proving in a way that not only helps the student understand the current
proof, but also allows for a high learning effect. What is meant by the latter is the
ability of the students to better understand the problem at hand, as well as to
generalise and apply the taught strategies on their own later on. Adhering to the
psychological evidence for the high educational effect of hinting [7, 8], we propose to
establish those tutoring aims by making use of the socratic tutoring method
2
, whose
decisive characteristic is the use of hints in order to achieve self-explanation [2, 8, 9].
Our work builds on the, little, systematic research done to date in the area [11, 12,
13].
1
We will refer to the student as ‘he’ and to the tutor as ‘she’ for clarity. No gender preference is
assumed.
2
This method bears similarities to contingent help [10].
In order to model hinting, we have been developing a taxonomy of hints for
the naive set theory, which is based on the previously mentioned mathematical
ontology. This taxonomy is used by a hinting algorithm which models the socratic
tutoring method by means of calling different hint categories according to an implicit
student model [14].
The implicit student model of the hinting algorithm makes use of the student
answer categorisation and of information on previous hints produced. It is different
from standard user modelling, which we are not concerned with in this paper. Its aim
is to use information from the running tutoring and the current progress of the student
on the particular problem under consideration in order to produce hints. Those hints,
in turn, intend to address the cognitive state of the student and cause him to reflect
upon his answers and his reasoning.
3
In this paper, we first describe the domain ontology in Section 2. Then, in
Section 3 we look at a scheme for categorising the student answer and we outline the
use of the domain ontology and subdialogues in categorising the student’s input. Next,
in Section 4 we give a brief overview of the taxonomy of hints and of the hinting
algorithm. In Section 5 we discuss our work with regard to the notion of reflection.
Finally, we discuss some relevant work in Section 6 and conclude the paper.
2 A Domain Ontology for Tutoring
In this section, we first show a part of ΩMEGA’s [6] mathematical database. Then,
we give some of the relations we have defined for the needs of the hinting process, for
instance, for realising hint categories (cf. [16]) and for categorising the student’s
answer (cf. Sections 3 and 4).
A Mathematical Database In ΩMEGA’s database, assertions are encoded in a
simply-typed λ-calculus. In this paper, we concentrate on mathematical concepts from
the naive set theory. These concepts draw on the types sets and inhabitants of sets.
We give two examples of definitions of mathematical concepts in intuitive terms as
well as in more formal terms paraphrasing but avoiding the λ-calculus encoding of
ΩMEGA’s database.
Let U, V be sets and let x be an inhabitant.
Element: The elements of a set are its inhabitants:
x ∈ U if and only if x is an inhabitant of U.
Intersection: The intersection of two sets is the set of their common elements:
U ∩ V = {x | x ∈ U and x ∈ V}.
The existing database also includes lemmata and theorems that use concepts, as well
as inference rules. Since every lemma and theorem can be rewritten as an inference
rule and vice versa, we identify lemmata and theorems with inference rules
henceforth.
Enhancing the Ontology The mathematical database implicitly represents many
relations that can be used in tutorial dialogues. Further useful relations can be found
when comparing the definitions of concepts with respect to common patterns. We
3
DIALOG uses ACTIVEMATH [15] as an external facility for user modelling.
consider relations between mathematical concepts, between mathematical concepts
and inference rules, and among mathematical concepts, formulae and inference rules.
By making these relations explicit we convert the mathematical database into an
enhanced ontology that can be used in tutoring. Let us look at two examples of
relations that we defined between mathematical concepts.
Let
σ
,
σ
’ be mathematical concepts. We define the relations:
Antithesis:
σ
is in antithesis to
σ
’ if and only if it is its opposite concept (i.e., its
logical negation).
Examples: antithesis(∈, ∉), antithesis (⊂, ⊄)
Hypotaxis:
σ
is in hypotaxis to
σ
’ if and only if
σ
’ is defined using
σ
. We say,
σ
is a
hypotaxon of
σ
’, and
σ
’ is a hypertaxon of
σ
.
Examples: hypotaxon(⊆, ⊂), hypotaxon(∈, ∩)
3 Student Answer Categorisation
In this section we present a scheme for categorising student answers. We are only
concerned here with the parts of the answer that address domain knowledge. The
scheme provides a classification of the answer with regard to an expected answer,
which is always approximated for the student’s own line of reasoning. The line of
reasoning is the formal proof that the student is performing with all proof steps and all
transitions from one step to the next, as well as all potential explanations for the
particular proof and for each step. The system’s response is based on that
categorisation, as in all traditional tutoring systems. The scheme becomes relevant as
the output of the classification constitutes part of the input to the hinting algorithm,
which models the hinting process. The process itself aims at encouraging the student
to reflect on his line of reasoning and learn actively.
We have defined student categories based on their completeness and accuracy
with respect to the expected answer. The expected answer is the proof step that is
expected next by the student, according to the proof which has been produced by the
system for the problem at hand. It includes premises, inference rule and conclusion.
In the following we define completeness and accuracy, as well as the parts of
the answer that these terms apply to.
Proof Step Matching In order to allow the student to proceed with his own line of
reasoning, we make use of that reasoning in helping him with the task. In order to
follow the student’s line of reasoning, we match his answer to one of the possible next
steps in one proof of a set of possible proofs. Since the set of possible proofs might be
too large to be manageable, we achieve that by having ΩMEGA [6] check the
correctness and relevance of the student’s answer. If the answer was not correct,
ΩMEGA returns the step that is closest to the one the student intended to make. That
step constitutes our expected answer. We can then evaluate the student’s answer
against this expected answer. Hence, the system gives guidance according to the proof
that the student is attempting without super-imposing one of the alternatives.
Completeness vs. Accuracy We call an answer complete if and only if all parts of
the expected answer are mentioned. We call a part of an answer accurate if and only
if the propositional content of the part is the true and expected one.
From this definition of completeness it follows that completeness is dependent
on domain objects, but not on the domain ontology. That is, the expected answer,
which is the basis of the evaluation of the completeness of a student answer,
necessarily makes use of objects in the domain. However, the relations of the objects
in the domain are irrelevant to evaluating completeness. Completeness is a binary
predicate, and only the presence or absence of objects in the student answer is
relevant to it.
Accuracy, contrary to completeness, is dependent on the domain ontology. It
refers to the appropriateness of the object in the student answer with respect to the
expected object. An object is accurate, if and only if it is the exact expected one. We
are using accuracy as a binary predicate in the same way that we do with
completeness. However, we intend to extend the categorisation to include different
degrees of accuracy. The relation in the domain hierarchy, which holds between the
expected concept and the one the student provides, will constitute an additional
criterion for the accuracy of an answer.
Parts of Answers We now define the relevant units for the categorisation of the
student answer. A part is a premise, the conclusion and the inference rule of a proof
step. The two former are mathematical formulae and must be explicitly mentioned for
the proof to be complete. The inference rule can either be referred to nominally, or it
can be represented as a formula itself. In the latter case, we just consider the inference
rule as another premise. It is up to the student to commit to using the rule one way or
the other. A formula is a higher-order predicate logic formula. Every symbol defined
in the logic is a function. Formulae can constitute of subformulae to an arbitrary
degree of embedding. Constants are 0-ary functions that constitute the lowest level of
entities considered.
The Categories Let us now enumerate the categories of student answers based on
our definitions of completeness and accuracy and with regard to the expected answer.
Note, however, that although the following list is exhaustive further refinement is due.
Correct: An answer which is both complete and accurate.
Complete-Partially-Accurate: A complete answer with some inaccurate parts.
Complete-Inaccurate: An answer which is complete, but all parts in it are inaccurate.
Incomplete-Accurate: An incomplete answer, with accurate present parts.
Incomplete-Partially-Accurate: An incomplete answer with some inaccurate parts.
Wrong: An answer which is both incomplete and inaccurate.
Using the Domain Ontology The ontology we presented in Section 2 is evoked,
among other things, in categorising the student’s answer. That is, the algorithm takes
as input the analysed student answer. In analysing the latter, we compare it to the
expected answer and look for the employment of necessary concepts. These
necessary concepts are defined in terms of the ontology. The algorithm checks for the
student’s level of understanding by tracking the use of these concepts in the student
answer to be addressed next. The hint to be produced is then picked according to the
knowledge demonstrated by the student. Note that this knowledge might as well have
already been provided by the system itself, in a previous hinting turn when dealing
with the same proof step. Since the algorithm only checks for generic descriptions of
those concepts, we use the present ontology in order to map the descriptions onto the
actual concepts relevant to the particular context.
Subdialogues The tutor can initiate subdialogues either in case of ambiguity or in
case of inability to classify the student’s answer. The students are given the
opportunity to correct themselves, provide additional information on their reasoning
and give essential information about the way they proceed with the task. Let us look
at some examples.
Subdialogues are used to treat any irrelevant answers, which are not wrong in
principle but do not contribute anything to the current proof, since they do not address
the expected step. In these cases, the tutor informs the student of the status of his
answer.
Another instance of subdialogue initiation is in case of potential typos. Since
we do not want to count the latter towards the student model, we need to treat them
differently from conceptual domain mistakes, such as wrong instantiations. We can
prevent that by asking the student what he really meant to say. Our assumption is that
if the student made a typo and not a conceptual domain mistake, he will realise it and
correct it. If he does not correct it, we categorise the answer taking into account the
domain mistake. We identify this kind of subdialogue with alignment [17], and an
instance of it is the example in Figure 1. (The examples are taken from a corpus on
mathematics tutorial dialogues in German, which we recently collected [18].
Translations are included, where necessary.) The tutor could not make sense of the
student’s utterance unless he substituted A ∩ B = ∅ for A ≠ B and B ∩ K(B) = ∅ for B
≠ K(B). The student could not correct himself, although he realised from the tutor’s
question that he had used the wrong symbol.
S5: wenn A ⊆ K(B), dann A
≠
B, weil B ≠ K(B) [if A
⊆
K(B), then A ≠ B, because B ≠ K(B)]
T6: meinen Sie wirklich ≠ oder etwas anderes? [Do you really mean
≠
or something else?]
S6:
⊄
Figure 1: Subject 13
Every time the tutor cannot categorise the student’s answer because no
matching can be found, she initiates a clarification subdialogue. This is done up to
two times for each answer. If the answer cannot be matched to anything, it is
categorised as wrong. A clarification dialogue of this kind can be seen in Figure 2.
The student seems to be applying a rule, which is not recognised. Thus, the tutor asks
the student to specify the rule.
S2: P((A ∪ C) ∩ (B ∪ C)) = P(A) ∩ P(C) ∪ P(B) ∩ P(C)
T2: Welche Regel haben Sie hier angewendet ? [Which rule are you using here?]
Figure 2: Subject 21
Subdialogues can also be initiated by the student when, for instance, he is not
content with the evaluation of his level. In Figure 3, the tutor provides a
give-away-
inference-rule
hint, judging that the student does not have the knowledge that the hint
provides. The student initiates a subdialogue and informs the tutor of his erroneous
judgement. The tutor accepts that and encourages the student to apply his knowledge.
T5: Sie müssen die wenn-dann-Beziehung auflösen, indem Sie die Wahrheit der Voraussetzung
annehmen. [You have to eliminate the if-then-relation, by assuming that the presupposition is true.]
S5: schon klar [I already know.]
T6: Dann tun Sie das bitte. [Then, please, do it.]
Figure 3: Subject 20
4 Hinting
Hint Taxonomy We have defined hints, which are produced by the hinting
algorithm, based on the needs in the domain as they revealed through the domain
ontology. The hint taxonomy captures the underlying function of hints that can be
common for different surface realisations. This underlying function is mainly
responsible for the educational effect of hints. The structure of the hint taxonomy also
reflects the function of the hints with respect to the information that the hint addresses
or is meant to trigger. In order to capture the different functions of a hint in the hint
taxonomy we have defined hint categories across two dimensions.
The first dimension distinguishes between the active and passive function of
hints. The former refers to the information provided each time and the latter to the
information that the hint aims at triggering in the student’s current cognitive state, that
is, the information elicited. Since some information often needs to be given away to
elicit some other information, most hints have both the active and passive function.
The second dimension distinguishes between different classes of hints. Each
of these classes consists of single hint categories that elaborate on one of the attributes
of the proof step under consideration. Hint categories are grouped in classes according
to the kind of information they address in relation to the domain and the proof. The
classes are ordered with respect to the amount of information that they give away.
By and large, the hints of the passive function of a class in the second
dimension constitute the hints of the active function of its immediately subordinate
class. For example, the passive hint
give-away-relevant-concept of the class domain-
object
is also an active hint of its subordinate class, namely, inference-rule. In providing
this hint the system is trying to elicit the inference rule, which would help the student
proceed with the proof. A realisation of a
give-away-relevant-concept hint when the
tutor wants to point to the inference that eliminates an implication is given in Figure
4. This hint could precede the hint in Figure 3, which informs the student what to do
with the if-then-relation, as the latter gives more information away.
T2: Sie müssen als erstes die wenn-dann-Beziehung betrachten. [First you have to consider the if-then-
relation.]
Figure 4: Subject 23
For the complete hint taxonomy and a detailed discussion see [14].
A Hinting Algorithm A tutorial system ideally aims at having the student find the
solution to a problem by himself. Only if the student gets stuck should the system
intervene. There is pedagogical evidence [7, 8] that students learn better if the tutor
does not give away the answer but instead gives hints that prompt the student for the
correct answer. Accordingly, based on the work by Tsovaltzi [13] we have derived an
algorithm that implements an eliciting strategy that is user-adaptive by choosing hints
tailored to the students. Only if hints appear not to help does the algorithm switch to
an explaining strategy, where it gives away the answer and explains it. We follow
Person and colleagues [2] and Rosé and colleagues [8] in calling the eliciting strategy
socratic and the explaining strategy didactic.
The algorithm makes use of an implicit student model and takes into account
the current and previous student answers. The particular input to the algorithm is the
category that the student answer has been assigned, based on the student answer
categorisation scheme, and the domain knowledge employed. Moreover, the
algorithm computes whether to produce a hint and of which category that hint should
be, based on the number of wrong answers, as well as the number and kind of hints
already produced (cf. Sections 3 and 5).
We now examine the algorithm and the way the student’s level is taken into
account.
The Algorithm We present in this paper the main function of the algorithm which
implements hinting. We derived this algorithm from empirical data, namely from the
corpus collected in the BE&E project [8], with additional normalisations motivated by
educational theories [13]. The function
socratic produces hints directly or calls
several other functions, which we do not examine in this paper. The functions check
the domain knowledge available to the student and decide on producing the least
informative hint.
For a more detailed description of the algorithm see [14].
The Function
socratic The bulk of the work is done by the function socratic,
which we only outline here. The function takes as an argument the category C of the
student’s current answer. If the origin of the student’s mistake is not clear, a
clarification dialogue is initiated (cf. Section 3). Note, however, that the function
stops if the student gives the correct answer during that clarification dialogue, as that
means that the student corrected himself. Otherwise, the function produces a hint in a
user-adaptive manner.
In the following, H denotes the number of hints produced so far and C
-1
the
category of the student’s previous answer. Furthermore, the student answer category
inaccurate is shorthand for one of the categories complete-partially-accurate or
complete-inaccurate or incomplete-partially-accurate. A hint is then produced as
follows:
After having produced a hint the function socratic analyses the student’s answer to
that hint. If the student’s answer is still not right the function
socratic is recursively
called.
5 Reflection
In order to strike a balance between guiding the student and allowing him to proceed
with his own line of reasoning, we guide the student through hints that aim at causing
him to reflect on what is missing or wrong in his answers and line of reasoning. Proof
step matching makes it possible to follow the student’s line of reasoning by choosing
between different possible expected answers. The categorisation scheme provides the
necessary input for updating the implicit student model of the hinting algorithm. It gives
the degree that the student’s answer fits the expected answer. A second input for the
implicit student model is whether the student possesses certain domain knowledge
necessary for the task. The second dimension of the hint taxonomy captures that, as the
hints in it provide gradually more of that domain knowledge. They are produced based
on the input for the implicit student model.
The hinting process with the input described together with proof step matching
aim at making the student reflect about his own line of reasoning by pointing out what is
useful in it (relevant mathematical concepts, inference rules etc.) and by providing a hint
in order to advance in the task. Therefore, the hinting process gives him both the
opportunity to reconsider his answer in an immediate and encouraging mode, and to take
the system’s guidance into account in order to learn actively. The student can thus
benefit in three ways. First, he can reflect on his own reasoning by getting feedback
on it. Second, it becomes cognitively easier to learn since learning presumably takes
place based on the structures that gave rise to the particular line of reasoning. Third,
he gets the feeling of achievement, which is pedagogically encouraging.
6 Discussion
There are several approaches to categorising the student’s input. Some of them are
mostly relevant to parsing the input. We briefly discuss work similar to ours only with
respect to classifying the domain knowledge demonstrated by the student, as we are
not dealing with natural language processing in this paper.
AutoTutor [2] uses a sophisticated LSA (Latent Semantic Analysis) approach.
It computes the truth of an input based on the maximum match between it and the
training material results. It computes relevance by comparing it to expected answers.
The latter are derived from the curriculum scripts. It further uses the notions of
completeness and compatibility. Both these notions are defined as percentages of the
relevant aspect in the expected answer.
Statistical methods, however, are insensitive to recognising linguistic
phenomena crucial to the evaluation, such as negation, and LSA is insufficient for
evaluating the kind of short answers representative of our domain. Moreover, we need
a definition of completeness and accuracy that gives us insight to the part of the
student answer that is problematic and hence needs to be addressed by appropriate
hinting. Therefore, defining them in terms of percentages is not enough. We need to
represent the relevant domain entities that can serve as the basis for the hint
production.
Why-Atlas [19] uses theorem proving techniques based on abduction for
classifying student answers. Elaborate inferences can be made about the student’s
reasoning with this approach. This is similar to our approach with the difference that
we use deductive instead of abductive reasoning. Abductive reasoning, however,
could prove to be useful to detect misconceptions.
CIRCSIM [20, 21] and PACT-Geometry tutor [22] both use reasoning based
on domain knowledge, similar to the approach presented in this paper. They evaluate
the conceptual content of the student’s response by use of domain ontologies.
CIRCSIM uses a fine-grained classification. Unfortunately, the rationale of the
classification is not documented. That makes comparison difficult. However, a list of
ways of responding after a particular classification is available. Still no flexibility is
allowed of the kind we are attempting by automating hinting.
PACT-Geometry [22] tutor uses a cognitive model of the ideal student based
on production rules, which is used to estimate the student’s skills. A semantic
representation of the student’s input is built that makes use of a domain ontology in
order to resolve concepts used and their relations. The correctness of the reasoning
used is based on a hierarchy of explanation categories that includes legitimate
reasoning steps in the domain. Our work is similar to this approach in the use of
organised domain knowledge to evaluate the student’s answer and in providing hints.
PACT-Geometry tutor aims at helping students with self-explanation. It does not have
a structured approach of the kind that we have presented in this paper, but it uses a
more elaborate student model to understand what exactly, if possible, the student
needs help with.
Finally, STyLE-OLM [23] uses an interactive open student model. The learner
has access to his model and can negotiate about it and alter it. It uses a graphical
representation of the student’s beliefs and the theory of dialogue games for the
dialogue management. The learner’s beliefs are approximated by comparing his
behaviour to the domain knowledge required. We also aim at getting the student to
reflect on his answer and at engaging him in subdialogues that allow us to evaluate
the student answer in a more accurate way. However, our approach differs in that we
are building a natural language tutorial dialogue system that uses the process of
hinting as a method of making the student active in performing the task. Hints intend
to address the cognitive state of the student and cause him to reflect upon his
reasoning, compare it gradually to the desired reasoning and augment or correct it.
Automating the hinting process provides flexibility with the hope of fitting the
student’s needs better. It allows capturing discourse structure in the generation of
hints through natural language. This facilitates the student’s understanding further [1].
7 Conclusion and Future Work
We presented an approach which actively involves the student in the learning process
by making him reflect upon his reasoning and constitutes the basis for a high learning
effect. We examined some of the prerequisites for modelling this approach, namely (i)
the enhancement of a domain ontology for the naive set theory in mathematics, (ii) a
hint taxonomy, which was derived with the aim of automating the hint categories
defined in it and (iii) a hinting algorithm.
Moreover, we presented a student answer categorisation scheme which uses
the notions of expected answer, completeness and accuracy in evaluating the student’s
domain knowledge. This categorisation is further elaborated for evaluating the
student’s input by clarification subdialogues initiated by the tutor, as well as by
allowing the student himself to initiate them.
In the future we plan to make further developments in our domain ontology to
improve the evaluation of the student’s answer, for instance, by capturing different
degrees of accuracy of the parts in the answer. We will also refine our student answer
categories and model more subdialogues to better treat the refined categories. All
these improvements will, in turn, help us augment the hinting algorithm. To meet
these aims, we will use the empirical data from the corpus on mathematics tutorial
dialogues that we recently collected [18].
References
1. Johanna Moore. What makes human explanations effective? In Proceedings of the Fifteenth Annual
Conference of the Cognitive Science Society, pages 131-136. Hillsdale, NJ. Earlbaum, 2000.
2. Natalie K. Person, Arthur C. Graesser, Derek Harter, Eric Mathews, and the Tutoring Research
Group. Dialog move generation and conversation management in AutoTutor. In Rosé and Freedman
[24], pages 45-51.
3. Vincent Aleven and Kenneth Koedinger. The need for tutorial dialog to support self-explanation. In
Rosé and Freedman [24], pages 65-73.
4. Neil Heffernan and Kenneth Koedinger. Intelligent tutoring systems are missing the tutor: Building a
more strategic dialog-based tutor. In Rosé and Freedman [24], pages 14-19.
5. Manfred Pinkal, Jörg Siekmann, and Christoph Benzmüller. Projektantrag Teilprojekt MI3
DIALOG: Tutorieller Dialog mit einem mathematischen Assistenzsystem. In Fortsetzungsantrag
SFB 378—Ressourcenadaptive kognitive Prozesse, Universität des Saarlandes, Saarbrücken,
Germany, 2001.
6. Jörg Siekmann, Christoph Benzmüller, Vladimir Brezhnev, Lassaad Cheikhrouhou, Armin Fiedler,
Andreas Franke, Helmut Horacek, Michael Kohlhase, Andreas Meier, Erica Melis, Markus
Moschner, Immanuel Normann, Martin Pollet, Volker Sorge, Carsten Ullrich, Claus-Peter Wirth, and
Jürgen Zimmer. Proof development with ΩMEGA. In Andrei Voronkov, editor, Automated
Deduction—CADE-18, number 2392 in LNAI, pages 144-149. Springer Verlag, 2002.
7. Michelene T. H. Chi, Nicholas de Leeuw, Mei-Hung Chiu, and Christian Lavancher. Eliciting self-
explanation improves understanding. Cognitive Science, 18:439-477, 1994.
8. Carolyn P. Rosé, Johanna D. Moore, Kurt VanLehn, and David Allbritton. A comparative evaluation
of socratic versus didactic tutoring. In Johanna Moore and Keith Stenning, editors, Proceedings of the
23rd Annual Conference of the Cognitive Science Society, University of Edinburgh, Scotland, UK,
2001.
9. Dimitra Tsovaltzi and Colin Matheson. Formalising hinting in tutorial dialogues. In EDILOG: 6th
workshop on the semantics and pragmatics of dialogue, pages 185-192, Edinburgh, Scotland, UK,
2002.
10. Heather A. Wood and David J. Wood. Help seeking, learning and contingent tutoring. In Computers
and Education, 33:153-169, 1999.
11. Gregory D. Hume, Joel A. Michael, Rovick A. Allen, and Martha W. Evens. Hinting as a tactic in
one-on-one tutoring. Journal of the Learning Sciences, 5(1):23-47, 1996.
12. Armin Fiedler and Helmut Horacek. Towards understanding the role of hints in tutorial dialogues. In
BI-DIALOG: 5th Workshop on Formal Semantics and Pragmatics in Dialogue, pages 40-44,
Bielefeld, Germany, 2001.
13. Dimitra Tsovaltzi. Formalising hinting in tutorial dialogues. Master’s thesis, The University of
Edinburgh, Scotland, UK, 2001.
14. Armin Fiedler and Dimitra Tsovaltzi. Automating hinting in mathematical tutorial dialogue. In
Proceedings of the EACL-03 Workshop on Dialogue Systems: interaction, adaptation and styles of
management, pages 45-52, Budapest, 2003.
15. Erica Melis, Eric Andres, Andreas Franke, George Goguadse, Michael Kohlhase, Paul Libbrecht,
Martin Pollet, and Carsten Ullrich. A generic and adaptive web-based learning environment. In
Artificial Intelligence and Education, pages 385-407, 2001.
16. Dimitra Tsovaltzi and Armin Fiedler. Enhancement and use of a mathematical ontology in a tutorial
dialogue system. In Proceedings of the IJCAI Workshop on Knowledge and Reasoning in Practical
Dialogue Systems, Acapulco, Mexico, 2003. In press.
17. Jean Carletta, Amy Isard, Stephen Isard, Jacqueline C. Kowtko, Gwyneth Doherty-Sneddon, and
Anne H. Anderson. The reliability of a dialogue structure coding scheme. Computational Linguistics,
23(1):13-32, 1997.
18. Chris Benzmüller, Armin Fiedler, Malte Gabsdil, Helmut Horacek, Ivana Kruijff-Korbayová,
Manfred Pinkal, Jörg Siekmann, Dimitra Tsovaltzi, Bao Quoc Vo, and Magdalena Wolska. A
Wizard-of-Oz experiment for tutorial dialogues in mathematics. In Proceedings of the AIED
Workshop on Advanced Technologies for Mathematics Education, Sidney, Australia, 2003. In press.
19. Pamela W. Jordan and Kurt VanLehn. Discourse processing for explanatory essays in tutorial
applications. In Proceedings of the 3rd SIGdial Workshop on Discourse and Dialogue, Philadelphia,
USA, 2002.
20. Zhou Yujian, Reva Freedman, Michael Glass, Joel A. Michael, Allen A. Rovick, and Martha W.
Evens. What should the tutor do when the student cannot answer a question? In Proceedings of the
Twelfth International Florida AI Research Society Conference (FLAIRS-99), pages 187-191,
Orlando, FL, 1999. AAAI Press.
21. Michael Glass. Processing language input in the CIRCSIM-Tutor intelligent tutoring system. In
Johanna Moore, Carol Luckhardt Redfield, and W. Lewis Johnson, editors, Artificial Intelligence in
Education, pages 210-212. IOS Press, 2001.
22. Vincent Aleven, Ocav Popescu, and Kenneth R. Koedinger. A tutorial dialogue system with
knowledge-based understanding and classification of student explanations. In Working Notes of 2nd
IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems, Seattle, USA, 2001.
23. Vania Dimitrova, John Self, and Paul Brna. The interactive maintenance of open learner models. In
S. P. Lajoie and M. Vivet, editors, In Artificial intelligence in education, pages 405-412. IOS Press,
Amsterdam, 1999.
24. Carolyn P. Rosé and Reva Freedman, editors. Building Dialog Systems for Tutorial Applications
Papers from the AAAI Fall Symposium, North Falmouth, MA, 2000. AAAI press.
