Content uploaded by Rupert Wegerif
Author content
All content in this area was uploaded by Rupert Wegerif
Content may be subject to copyright.
Available via license: CC BY-NC-ND 4.0
Content may be subject to copyright.
Proceedings of the 6
th
International
Conference on Networked Learning
80
ISBN No: 978-1-86220-206-1
Facilitate the Facilitator: Awareness Tools to Support
the Moderator to Facilitate Online Discussions for
Networked Learning
Maarten de Laat, Mike Chamrada & Rupert Wegerif
School of Education and Lifelong Learning, The University of Exeter, m.f.delaat@exeter.ac.uk,
mc286@exeter.ac.uk, r.b.wegerif@exeter.ac.uk
Abstract
This paper is part of an ongoing European research project, called ARGUNAUT in which we
present some of our findings regarding the development of online awareness indicators aimed
at supporting moderators to facilitate online discussions. Firstly we discuss our theoretical
orientation towards online argumentation and dialogues. This results in a multidimensional
analytical framework to analyse synchronous discussions in order to develop awareness tools
for the ARGUNAUT system. Secondly we present an overview of this system and in particular
the moderators interface (MI). The MI will be used by the moderator to select awareness
visualisations about students’ discussions. The moderator can request for example various
descriptive statistical information about the number and type of contributions made by students
or more advanced visualisations that show live interaction patterns of the participants using
social network analysis (SNA) techniques and Deep Loop classifications of contributions
written by the students. The Deep Loop, is an AI-based indicator that can automatically detect
for example if students are talking of / on-task, critical reasoning and question-answer patterns.
This paper ends with a discussion of current discourse analysis done by our team aimed at the
identification of important critical moments in the discussion that feature particular dialogic
properties and might need the attention of the moderator.
Keywords
Online facilitation, synchronous, awareness,
Introduction
This paper is part of an ongoing European research project, called ARGUNAUT, in which we like to
present some of our initial findings regarding the development of online awareness tools supporting the
moderator to best facilitate online discussions. The ARGUNAUT system that is being developed during
this project is based on synchronous learning and embeds an integrated suite of tools in order to set up
and moderate synchronous disussions. The synchronous discussion are held by the students in either
Digalo or Freestyler (developed at the University of Duisburg) which are tools that support online visual
argumentation and dialogues. Our focus in this paper however is on the Moderators Interface (MI), which
is a tool especially designed for the moderator. This tool allows moderators to log on to one or more
ongoing discussions and presents the moderator a set of awareness indicators based on which the
moderator can remotely (using the MI) facilitate these discussions.
The central idea behind this project is to develop these awareness indicators, using data mining
techniques and artificial intelligence trained on pedagogically annotated events of teaching and learning
activities in online discussions, to inform the moderator about teaching and learning activities that occur
in online discussions. The challenge is to find ways in which pedagogical ideas of online argumentation,
teaching and learning processes can be articulated into rules used by the awareness tools to find patterns
or examples of particular networked learning behaviour that signify these rules.
In this paper we will present the use of the MI awareness indicators and some of the ongoing pedagogical
research.
Proceedings of the 6
th
International
Conference on Networked Learning
81
ISBN No: 978-1-86220-206-1
Facilitating online argumentation and dialogues
The conceptual pedagogical framework of the ARGUNAUT system is a combination of three main
theoretical orientations. Firstly argumentation and dialogic theory; secondly we integrate these with
current understandings of teaching and moderating online and thirdly we situate them both within the
paradigm of social learning, which influences how we frame collaborative learning and construction of
knowledge. The integration of these three aspects made us realise that we needed a multi-dimensional
analytical framework to analyse synchronous discussions in order to develop awareness tools for the MI.
From the argumentation point of view one might focus on the quality of critical reasoning. Dialogic
theory focuses more on the multiplicity of perspectives and the creative emergence of new ways of seeing
problems. Moderation aims to reflect on the impact of interventions made by the moderator to steer the
discussion in a desired direction and social learning theories, in principle, focus on the level of
participation and the conditions in which groups learn collectively. The analytical framework developed
in this project aims to address and synthesize these different orientations in a meaningful way.
Many coding schemes used for investigating the quality of online collaborative learning have been
derived from argumentation theory (see De Laat and Wegerif, 2006). This can be seen in the focus on the
key moves of argumentation: claim, counter-claim, warrant, grounding etc. However there is increasing
interest in real contexts of problem solving and learning, which the traditional argumentation model does
always fit well. According to Chi (1997), for example, learning new concepts in science involves the
‘creativity’ of seeing familiar things from a new theoretical perspective. According to us, although this is
not necessarily Chi’s position, the capacity to shift perspective, to see things from multiple alternative
points of view at once and to collaboratively construct together a new perspective all imply ‘dialogic’
skills. ‘Dialogic’ here implies not only ‘pertaining to dialogue’ but also that these skills relate to the
fundamental nature of dialogue as the co-presence together of different perspectives (Wegerif, 2007). In
the light of this Wegerif and De Laat (2006) have argued that ‘higher order thinking skills’ including
creativity and critical thinking need to be reconceptualised as aspects of engagement in dialogue, rather
than in terms of argumentation alone. This is not to do away with traditional accounts of argumentation
but to expand them to include more of the dialogic context of arguments. Our argument is that the quality
of dialogic engagement, or the capacity to see from more than one perspective at once, is one influence on
the quality of critical thinking as well being fundamental to creativity and to learning. The challenge for
our analytic framework then is how to capture the quality of dialogic engagement as this impacts on
creativity and on openness to learning.
A shift towards dialogic engagement and collaborative learning argues for active student engagement
with not only learning but also peer-tutoring activities. The traditional relationship between the teacher
and students, where the teacher is responsible for designing and evaluating learning activities for example
has changed into a more shared responsibility. Research on networked learning shows that the teacher in
an online discussion environments acts as a learner and teacher (De Laat, 2006), but is mainly concerned
with guiding and facilitating the group. The students on the other hand adopt or develop roles to deal with
coordinating and regulating their shared group task. An important strand of research in networked
learning environments is looking into the development of networked learning skills and competencies of
both the teacher and students. How do they design and moderate teaching and learning activities when
discussing and learning online?
Engaging in high quality online argumentation means a mixture of learning and teaching qualities
providing a context in which groups of learners participate in open ended dialogues in which they
challenge each others perspectives guided by a skilled moderator. Being able to see the wood from the
trees we needed to cut across the various theoretical lenses combined in this project in order to develop
awareness indicators to improve the quality of online discussions.
One strategy we have adopted is to analyse ARGUNAUT-based dialogues according to several different
dimensions at once:
1. Pedagogical setting and group dynamics dimension is aimed at understanding the conditions and
ways in which students are participating in their learning task. In some ways this dimension also
serves as a precondition needed to contextualise the data.
2. Critical reasoning is focused on the more argumentative dimension present in the maps. Based
on the more structured or grammatical approach used in this tradition we are concerned with
how students develop the syntax of their arguments.
Proceedings of the 6
th
International
Conference on Networked Learning
82
ISBN No: 978-1-86220-206-1
3. Dialogic reasoning and engagement dimension seeks to highlight the quality of student
interaction, perspective taking and mutual engagement through the discourse
4. Moderation dimension describes the impact of interventions made during the online discussion
aimed at moderating and facilitating the quality of the discourse.
The outcomes of this analysis serves two aims. Our first aim is to code and annotate ARGUNAUT
discussions in order to develop awareness indicators used in the MI by the moderator. The aim of coding
is to detect (reoccurring) patterns and actions. This is mostly done by applying a comprehensive coding
scheme aimed at identifying structural argumentative and dialogical events in the synchronous
discussions. These expert coded events are used to develop learned classifiers using artificial intelligence
techniques (McLaren, Scheuer, De Laat, Hever, De Groot & Rose, 2007), that will be able to detect and
classify these events automatically and inform the moderator. This will be done by a component called
the Deep Loop. Our second aim is to advance dialogic theory and pedagogical practices of teaching and
learning online.
ARGUNAUT system
The ARGUNAUT system is aimed at facilitating moderators by offering them with several awareness
indicators providing feedback about ongoing events in the online discussion. For this reason the project
has developed the moderators interface (MI) through which the moderator can remotely observe one or
more ongoing online discussions (see figure1). The MI consists of two main components. The first being
a set of awareness indicators providing visualisations of students’ engagement with the online
discussions. These awareness indicators are the core of this project and based on the pedagogical research
(as discussed above) a set of visualisations are currently embedded in the MI. Using the MI the moderator
can select a visualisation of the actual discussion graph as it is created by the students, in order to read
their contributions. Furthermore the moderator can request various descriptive statistical information
about the number and type of contributions made by students. More advanced visualisations for example
shows live interaction patterns (see for more information De Laat, Lally, Lipponen & Simons, 2007) of
the participants using social network analysis (SNA) techniques and the Deep Loop classification of
contributions written by the students. The Deep Loop can automatically detect for example if students are
talking of / on-task, critical reasoning and question answer patterns.
Figure 1. Screen shot of the Moderators Interface
Proceedings of the 6
th
International
Conference on Networked Learning
83
ISBN No: 978-1-86220-206-1
The moderator can not only use these visualisations to observe and interpret student behaviour and
engagement with the online discussion but can also develop a set of rules based on which the MI will alert
the moderator if certain events occur following the conditions set in these rules. For example the SNA
tool can be used to create rules about student participation and notify the moderator if a student did not
participate during the last 10 minutes for example. These rules can therefore be action-based but also
directly related to the discussion content by using the Deep Loop classifiers or a set of key words as
conditions.
The second component is called the remote control which can be used by the moderator to facilitate the
discussion. Based on observations made or alerts received (using a set of rules) the moderator can choose
to sent a pop-up message to a particular student, group of students or the entire class. The moderator can
also annotate student contributions to make more in-depth comments directly related to what a student is
saying. Further more the moderator can highlight messages in the discussion to direct attention to it or can
use a remote pointer to point out to the students particular contributions or links featuring the discussion.
Current pedagogical research - future directions
As presented in this paper the ARGUNAUT system can facilitate the moderator in many ways, but
besides pointing out content-based and behavioural aspects of the discussion we would like to push a little
further and see if we are able to automatically detect critical moments in the discussion that feature
particular dialogic properties and might need the attention of the moderator. Our research is currently
focused at detecting patterns within the discussion that show elements of deepening and widening. These
are important steps in a discussion as they on the one hand try to provide further argumentation for a
perspective that is currently being discussed (deepening) on the other hand widening means an attempt to
‘break away’ from a particular perspective by either questioning it or presenting a new perspective. When
thinking about acts of deepening and widening (critical reasoning and dialogic reasoning), from a
dialogical point of view the widening moves in particular are of great interest since not much is known
about how widening moves are triggered. A widening move is often a creative act, i.e. the ability to step
back and come with a new ‘solution’ not thought of before. Such moves however are very important in
discussions as they stimulate people to think ‘out of the box’, and or stimulate further creative thinking
amongst the group members. These ‘eureka’ or ‘aha’ moments are hard to plan or moderate but should be
encouraged in order to open up the discussion space into new and unknown territories providing real
learning opportunities by and for its members.
In our research focused on discourse analysis we are coding single contributions as well as sequences of
related messages. Our coding framework (see De Laat & Wegerif, 2007) utilises of multiple levels
(contribution, sequence of contributions, and the entire discussion map) and multiple dimensions (group
dynamics, critical reasoning, dialogic reasoning, dialogic engagement, and moderation). Our work has
been to try and find a way to develop a coding methodology that shows some kind of continuum between
the various levels. Because we are looking at dialogues one can often assume that what is said in a
particular message will relate to something that is said in a previous message. For this reason we are
trying to find a way where the coding at shape level will inform the coding that is done at sequence or
even map level. This is a crucial step in our analysis as it determines the individual discussion threads-
sequences that later on inform the analysis of emergence of critical dialogic moments in a discussion.
By sequence we mean a continuous thread of dialogue in which all messages are linked together by their
content. Although it would be natural to identify these sequences by the links between different shapes,
this method proved to be unreliable due to inconsistencies in using the links by the participants. The
process of identifying these sequences begins for example with the opening question asked usually by the
moderator, who posts the first message. In Digalo, each shape is numbered according to the order in
which it has been posted (see figure 2 for an typical example of a Digalo discussion map). This order is
chronological; therefore it is possible to identify the development of the map from the time perspective.
Proceedings of the 6
th
International
Conference on Networked Learning
84
ISBN No: 978-1-86220-206-1
Figure 2. Example of an average Digalo discussion map
The construction of these discussions is very organic and by using the spatial and visual properties of the
Digalo tool these discussion can become rather complex. When creating these sequences we try to
abstract this complexity in order to create a more accessible representation of the discussion. At this point
of the analysis, sequence (or tree) diagrams are created.
Sequence diagram (see figure 3 for an example) – a visual representation of an online discussion that
serves the purpose of having an instant abstracted overview of the following aspects of a discussion:
1. The number and length of sequences of messages. A new sequence starts at the top representing the
first contribution of this particular discussion thread followed by related (linked) contributions shown
directly below this contribution in a vertical layout. When a new sequence is developed it will be
placed next to existing sequences.
2. The branching of sequences at different moments during the discussion. This happens when a
message in a sequence has more than one linked message.
3. Identifies messages that are not part of any sequence. They will appear isolated at the ‘top line’ of the
diagram
Once all the contributions are coded individually, these sequence diagrams can be used to visualise the
multiple dimensions of our analytical framework (group dynamics, critical reasoning, dialogic reasoning,
dialogic engagement, and moderation). The tree can for example be used to visualise group dynamics
properties of the discussion showing student engagement with the dialogues. This is shown in figure 3
which presents the interaction patterns between the students. Here each dot (signifying discussion
contributions) in the tree is coloured to represent contributions made by each student. The blue dots
represent student A, the yellow dots student B, etc. From this diagram it is instantly clear who is talking
to whom.
Figure 3. Group interaction patterns shown by a discussion tree-based visualisation of the Digalo
map
Proceedings of the 6
th
International
Conference on Networked Learning
85
ISBN No: 978-1-86220-206-1
Our next and current step in the analysis is to detect critical dialogic moments in the discussion by
mapping out previously discussed widening moves in the tree diagrams. In the tree diagram below (figure
4) the coloured dots represent opening questions (coloured blue) and disagreements (coloured red)
suggesting either new perspectives or opinions being presented or sought for by the students. Our
preliminary results indicate that the presentation of these widening moments coincide with branching
activities in the map. Almost each time when the tree structure branches to the left, rather then directly
down, it appears that participants tend to widen the discussion rather then just deepen a given perspective.
This means that when finding critical moments in a discussion the moderator needs to pointed out to
messages related to branching activities. Instead of having to browse through the entire map (see figure 2)
our coding results suggest that the moderator can focus on the surrounding messages related to branching
events in the discussion.
Figure 5. ‘Critical’ moments in the discussion
Some Concluding remarks
We need to analyse a larger data set to find out to what to extend this behaviour could be some kind of
reoccurring pattern featuring the discussions. Secondly we are planning to have several critical event
recall interviews with some of the students of these maps to question them about these critical moments
as a way to validate (or falsify) our initial findings. This is an important step in our attempt to triangulate
our findings (see De Laat & Wegerif, 2006). If branching events might be an indication of critical
moments in the discussion, we need to see if can we somehow annotate these critical moments to see if
they can be patterned in some way. In the ARGUNAUT project we are currently discussing with our
DFKI partners about the possibility of using artificial intelligence to train classifiers that might be able to
detect these critical moments based on our patterns as a way to extent the current Deep Loop features in
our ARGUNAUT system. Being able to detect these moments will mean a major step forward for online
moderation and student engagement in rich dialogues representing multiple alternative points of view.
Proceedings of the 6
th
International
Conference on Networked Learning
86
ISBN No: 978-1-86220-206-1
References
De Laat, M. F. (2006). Networked Learning. Apeldoorn: Politieacademie.
De Laat, M.F., & Wegerif, R. (2006). Argunaut Deliverable D5.1: Perspectives/Rules to Evaluate
Discussion. Public deliverable. Retrieved 12 January 2008 from
http://www.argunaut.org/publications/Members/rakheli/publications/Argunaut%20deliverable%20D5.
1%20-%20Perspectives-rules%20to%20evaluate%20discussions.pdf
De Laat, Lally,Lipponen & Simons, 2007 IJCSCL
Chi, M. T. H. (1997). Quantifying Qualitative Analyses of Verbal Data: A Practical Guide. Journal of the
Learning Sciences, 6, 271-313.
McLaren, B. M., Scheuer, O., De Laat, M., Hever, R., De Groot, R., & Rose, C. P. (2007). "Using
Machine Learning Techniques to Analyze and Support Mediation of Student E-Discussions". In:
Frontiers in Artificial Intelligence and Applications (vol. 158), Artificial Intelligence in Education -
Building Technology Rich Learning Contexts that Work, ed. by Rosemary Luckin and Kenneth R.
Koedinger and Jim Greer, pp. 331-338, Amsterdam, IOS Press
Wegerif. R. (2007) Dialogic, Educational and Technology: Resourcing the Space of Learning. New York:
Springer-Verlag.
Wegerif, R., De Laat, M.F. (2006). "Reframing the teaching of higher order thinking for the network
society". In: S. Ludvigsen, A. Lund, & R. Saljo. (Eds.) Learning in social practices: ICT and new
artefacts – transformation of social and cultural practices.
.