Content uploaded by Rupert Wegerif
Author content
All content in this area was uploaded by Rupert Wegerif on Aug 14, 2023
Content may be subject to copyright.
Exploring creative thinking in graphically mediated synchronous dialogues
Rupert Wegerif
a,*
, Bruce M. McLaren
b
, Marian Chamrada
a
, Oliver Scheuer
b
, Nasser Mansour
a
,
Jan Mikšátko
b
, Mriga Williams
a
a
University of Exeter, Graduate School of Education, St Luke’s, Exeter, UK
b
Deutsches Forschungszentrum für Künstliche Intelligenz, Saarbrücken, Germany
article info
Article history:
Received 1 May 2009
Received in revised form 29 September 2009
Accepted 19 October 2009
Keywords:
CSCL
Discourse analysis
Dialogic
Creativity
Artificial intelligence
Graphical interfaces
abstract
This paper reports on an aspect of the EC funded Argunaut project which researched and developed
awareness tools for moderators of online dialogues. In this study we report on an investigation into
the nature of creative thinking in online dialogues and whether or not this creative thinking can be coded
for and recognized automatically such that moderators can be alerted when creative thinking is occurring
or when it has not occurred after a period of time. We outline a dialogic theory of creativity, as the emer-
gence of new perspectives from the interplay of voices, and the testing of this theory using a range of
methods including a coding scheme which combined coding for creative thinking with more established
codes for critical thinking, artificial intelligence pattern-matching techniques to see if our codes could be
read automatically from maps and ‘key event recall’ interviews to explore the experience of participants.
Our findings are that: (1) the emergence of new perspectives in a graphical dialogue map can be recog-
nized by our coding scheme supported by a machine pattern-matching algorithm in a way that can be
used to provide awareness indicators for moderators; (2) that the trigger events leading to the emergence
of new perspectives in the online dialogues studied were most commonly disagreements and (3) the spa-
tial representation of messages in a graphically mediated synchronous dialogue environment such as
Digalo may offer more affordance for creativity than the much more common scrolling text chat environ-
ments. All these findings support the usefulness of our new account of creativity in online dialogues
based on dialogic theory and demonstrate that this account can be operationalised through machine cod-
ing in a way that can be turned into alerts for moderators.
!2009 Elsevier Ltd. All rights reserved.
1. Introduction
This paper reports on research that took place within a European Commission Framework 6 Future and Emerging Technologies (FET)
project, called Argunaut (http://www.argunaut.org/). The Argunaut system developed during the project uses the graphical e-discussion
environment Digalo (dito.ais.fraunhofer.de/digalo/) which is similar to a dynamic online concept map, along with a Moderators Interface
(MI) for teachers, which includes a range of awareness indicators and tools for intervention designed to make the task of moderation easier.
The awareness tools in the Argunaut MI, discussed in other papers (De Groot et al., 2007; Hever et al., 2007) include little charts indicating
who is online and who is interacting with who and an indication of the kinds of messages that they are choosing to send based on ‘message
types’ set-up in the system. As well as these ways of visualising the progress of the collaborative learning the MI also enables the moderator
to ask for and receive alerts which are highly customisable messages about the quality of the online dialogue. At the simplest level the
moderator can set the alert messaging tool to alert him or her if there is any bad language online by inputting a list of words and asking
to be warned if they occur. At a more complex level an alert can let the moderator know if there is any creative or critical thinking occurring
in the discussion over a period of time, or indeed perhaps more usefully, if there has been no critical or creative reasoning for a period of
time. Such alerts can help moderation by suggesting when intervention might be fruitful, and also what kind of intervention is needed: if
there is no sign of critical thinking for a period than a question challenging assumptions might be called for or if there is no sign of cre-
ativity it might be an idea to suggest a brainstorm of ways to look at the problem.
0360-1315/$ - see front matter !2009 Elsevier Ltd. All rights reserved.
doi:10.1016/j.compedu.2009.10.015
*Corresponding author.
E-mail addresses: r.b.wegerif@exeter.ac.uk (R. Wegerif), Bruce.Mclaren@dfki.de (B.M. McLaren), m.chamrada@exeter.ac.uk (M. Chamrada), Oliver.Scheuer@dfki.de
(O. Scheuer), n.mansour@exeter.ac.uk (N. Mansour), Jan.Miksatko@dfki.de (J. Mikšátko), Mriga.Williams@Exeter.ac.uk (M. Williams).
Computers & Education 54 (2010) 613–621
Contents lists available at ScienceDirect
Computers & Education
journal homepage: www.elsevier.com/locate/compedu
It is in the context of developing these awareness indicators for moderators that we explored online creative thinking and developed a
coding for creative thinking which could be read by a machine in order to generate online awareness alerts for moderators telling them if
creative thinking is or is not occurring. This led to a number of original developments at the methodological level: a new way of con-
ceptualising and coding for creativity in online dialogues, a new way of ensuring the reliability of the coding of online dialogues by auto-
mating that coding using machine learning pattern-matching techniques and finally, a methodology for exploring meaning making in
online dialogues combining discourse analysis of the electronic traces of dialogue with ‘key event recall’ interviews taking participants
back to events in the dialogue to explore what they meant and how they felt from the inside. It also led to some significant findings,
firstly of course the fact that the automatic coding of creativity in online dialogues is possible at all is a significant finding given the
uncertainty about the nature of creativity in the literature, secondly some trigger events for the emergence of creative new perspectives,
particularly disagreement, were found and finally evidence was gathered to support the claim that dialogues using graphical environ-
ments like Digalo, which allow for multiple views on a problem, are particularly good at supporting the teaching and learning of crea-
tivity. The method of using pattern-matching techniques for automated coded has been reported in published conference proceedings
(McLaren et al., 2007) and the more detailed data matching incidents of creativity and criticality has been reported in a short paper
in conference proceedings (Wegerif et al., 2009) but this is the first journal paper integrating a number of strands of the Argunaut project
to explore at more length the significance of these findings for our understanding of creativity in online dialogues and how to support
teaching for creativity online.
2. Understanding creative and critical thinking in online dialogues
Creativity is now generally agreed by educational policy makers to be an important skill for the knowledge age and governments across
the world are seeking to promote the teaching of creativity (e.g., Bruns, 2007; Leitch, 2006). However creativity remains a disputed concept
with a lack of consensus as to what creativity really is, where it comes from and how best to encourage it (Craft, 2005). One tension can be
seen in a definition of creativity produced by a UK government commission on how to promote creativity in education. The group defined
creativity as: ‘Imaginative activity fashioned so as to produce outcomes that are original and of value’ (NACCCE, 1999: 29) neatly combining
in one sentence the two sides of creativity which often seem incompatible, ‘imaginative activity’ or just ‘playing around’ on the one side
and products with social ‘value’ on the other. Governments who seek to promote creativity are almost certainly more interested in the pro-
duction of socially valued products than they are in promoting more ‘imaginative play’ for its own sake in classrooms. Another tension is
between the individualistic approach in much creativity research where the focus has been on what factors made this or that individual
into a creative genius different from others and more recent approaches that have seen creativity as part of group processes. Sawyer, a
leading creativity researcher, rejects what he calls the romantic myth of individual inspiration in favour of a view of creative breakthroughs
as a product of what he describes as lots of little sparks of insight that occur as people respond to each other in dialogues (Sawyer, 2006).
Similarly, empirical research on lab conversations by Dunbar suggests that the use of imaginative analogies in lab conversations underlies
creativity in science up to and including award winning major breakthroughs, one of which he was fortunate enough to record (Dunbar,
1995).
The literature on the analysis of online dialogues is dominated by the application of argumentation theory in which creativity is mar-
ginal and more formalisable accounts of reason are foregrounded. In a recent review of the literature, Andriessen (2008) presents devel-
opments in argumentation theory as moving from abstract and formal studies towards taking the empirical reality of human dialogues
into account. However it is clear most schemes applied to analyze online argumentation (e.g., those developed originally by Toulmin,
1958; Van Eemeren & Grootendorst, 2004; and Walton, 2000; see Andriessen, 2008 for details) focus on explicit reasoning in the form
of claims, challenges to claims and reasons in support of claims. This approach is good at picking up critical reasoning but ignores more
creative forms of shared thinking. Knowledge construction is sometimes claimed to follow from reasoning in an essentially dialectical
manner where a claim followed by a question challenging the assumptions behind the claim leads to the establishment of new shared
ground. However, any form of explicit reasoning which unpacks the necessary implications of grounds is not creative because it could, in
principal, be reduced to formal procedures or algorithms (for arguments against claims to the creativity of algorithmic thinking see
Wegerif, 2007, p. 78).
A possible implication of this focus on explicit reasoning in knowledge construction is that there is one correct way forward: a correct
answer already implicit in the premises. This was certainly the implication of the dialectical reasoning of Hegel that Bakhtin opposed
when he developed his ‘dialogic’ alternative. This dialogic approach begins with embodied, situated ‘living’ dialogue where no-one
can predict which direction the dialogue will take (Bakhtin, 1986). Bakhtin claimed that real open-ended dialogues are essentially cre-
ative and multiple. Against the idea in argumentation that it is necessary to achieve common ground and a shared framework he points
out that the meaning that flows in a dialogue actually depends on a tension between different perspectives. For Bakhtin the aim of dia-
logue is not common ground or consensus but creative inter-illumination between different voices and perspectives. In dialogues voices
interact in unpredictable ways to produce new perspectives that enable participants to see the topic of the dialogue in a new way. This
dialogic understanding of shared thinking, as a dance of voices and perspectives, puts the emphasis on creativity. Here we are defining
creativity as the unpredictable emergence of new perspectives in a dialogue where a new perspective is a new way of seeing the problem.
This contrasts to critical thinking in which the focus is on criteria for testing filtering ideas. Critical thinking, with its focus on questioning
premises and unpacking assumptions, can be understood as ‘deepening’ the space of dialogue whereas creative thinking, by generating
new ways of seeing a problem, can be described as widening this space (a version of a distinction found in Baker, Quignard, Lund, &
Séjourné, 2003). Of course both creative and critical thinking are often usefully combined but since previous coding schemes and analyses
of online learning have focussed successfully on critical thinking the focus on this study is on whether and how we can assess the
creative moment in shared thinking.
The essence of creativity in dialogues is the emergence of new perspectives out of the tension of holding multiple perspectives together.
This leads to the hypothesis that graphical interfaces may have a greater ‘affordance’ or natural support for creative emergence within dia-
logues than more linear threaded discussions (Wegerif, 2007, p. 259). In this paper we explore the affordance of graphical interfaces for
creative dialogues through the application of a coding for ‘new perspectives’ supported by artificial intelligence graph matching techniques
614 R. Wegerif et al. /Computers & Education 54 (2010) 613–621
and critical event interviews with participants. The idea that creativity can be seen in new perspectives that widen the dialogue by bringing
in a new voice or a new way of seeing is a specifically dialogic way of understanding creativity. This is not an account of creativity as fash-
ioning a socially valued product although to code a new idea as a new perspective it has to be valued as offering potential insight. In other
words a new perspective that widens a dialogue is not simply the result of unpacking a list of features of a problem, it is more like a new
point of view from which a problem can be seen in a new way. Examples will be given below that will make this notion of a new perspec-
tive clearer.
3. The ARGUNAUT coding scheme
On the ARGUNAUT project we developed a coding scheme both for research purposes and to provide a basis (and data) for the AI tech-
niques discussed later all with a view to supporting the moderators of online teaching and learning dialogues. The scheme included the
more traditional focus on explicit reasoning but also looked for the taking of perspectives and the listening to different perspectives in
a way that allows for the emergence of creative new perspectives (insights) that expand the dialogue without necessarily being a resolution
to any problem. In other words it sought to combine critical thinking with creative thinking. Earlier studies of classroom talk by Wegerif
(2005) and by Nystrand (1997) suggest that the development of dialogic reasoning, is often signalled through the expression of openness to
other points of view, through changes of mind and through inclusion of multiple voices in one ‘utterance’. This led us to expand the dimen-
sions of coding from the traditional single dimension of critical thinking with its focus on claims, counterclaims and reasons (D1) to include
the dimension of creative reasoning understood as a sort of dance of perspectives (D2) in which each new perspective or point of view on a
problem is labelled and also the dimension of dialogic engagement which includes not only ‘addressivity’ (language explicitly addressing
the other such as pronouns) and expressions of empathy but also expressions of doubt, changes of mind, ‘ventriloquation’ (a term from
Bakhtin for the presence of another voice within an utterance) and elicitation of the views of others (D3). At the same time we are crucially
interested in how moderation influences and can improve the quality of dialogues, not only the moderation of those assigned the role of
moderator but also of students moderating each other through encouragement and the scaffolding support of recapitulations, reformula-
tions and evaluations (D4).
The key code for dialogic creativity was that of ‘new perspective’. This takes the term ‘perspective’ as a metaphor for a new way of seeing
the problem. In physical space a perspective is one way of looking at an objective scene but in a dialogue a perspective is more than this as
it helps to create the shared landscape within which participants locate themselves. A ‘new perspective’ then is more than just a listing of
alternatives but implies an embodied perspective from which to see the world in a unique and different way. Our aim was to capture Bakh-
tin’s notion of the illumination and insight that comes from seeing an issue through other eyes. While the application of this coding was
intuitive and difficult to make fully explicit in a way that could be shared between the international partners on the project the fact that the
clusters identified as containing the emergence of a new perspective could be recognized by machine learning, as we describe below, offers
a convincing measure of reliability for this coding.
To help us code the rather complex online discussions produced in the course of the ARGUNAUT project (called ‘‘e-discussions” or ‘‘e-
discussion maps” henceforth) we developed sequence diagrams (see Fig. 1 for an example) that are visual representations of e-discussions
that serve the purpose of having an instant abstracted overview of the following aspects of a discussion:
(1) The number and length of sequences of messages. A new sequence starts at the top representing the first contribution of this par-
ticular discussion thread followed by related (linked) contributions shown directly below this contribution in a vertical layout. When
a new sequence is developed it will be placed next to existing sequences.
(2) The branching of sequences at different points during the discussion. This happens when a message in a sequence has more than one
linked message.
(3) Identifies messages that are not part of any sequence. They will appear isolated at the ‘top line’ of the diagram.
Once all the contributions are coded individually, these sequence diagrams can be used to visualise the multiple dimensions of our ana-
lytical framework (group dynamics, critical reasoning, dialogic reasoning, dialogic engagement, and moderation).
Our next step in the analysis was to detect key dialogic moments in the discussion by mapping out ‘widening’ moves (new perspectives)
in the tree diagrams. In the tree diagram below (Fig. 1) the coloured dots represent opening questions (coloured blue) and
1
disagreements
(coloured red) suggesting either new perspectives or opinions being presented or sought for by the students. Our results indicate that the
presentation of these widening moments coincide with branching activities in the map. Almost every time when the tree structure
branches to the left, rather then directly down, it appears that participants are widening the discussion rather then just deepening a given
perspective. Our use of sequence diagrams gives visual reference to widening in debates and our qualitative analysis showed that this cor-
responds to the dimension of creativity where new insights emerge whereas deepening corresponds largely to critical reasoning and dia-
lectic reasoning which unpack assumptions.
This coding scheme and approach to analysis has been tested and further developed through the analysis of over sixty free-form e-dis-
cussions created by approximately 100 undergraduates and 12 post-graduate students in the UK. The outcomes of this analysis serve two
aims. One aim is to advance understanding of creative thinking and how technology can support it. A linked aim is to code and annotate
ARGUNAUT discussions in order to develop awareness indicators used in the Moderators Interface by the moderator. The aim of coding is to
detect patterns and actions. Applying a comprehensive coding scheme aimed at identifying structural argumentative and dialogical events
in the synchronous discussions mostly does this. These expertly coded events were then used to develop classifiers using artificial intel-
ligence techniques (McLaren et al., 2007; Mikšátko & McLaren, 2008; Scheuer & McLaren, 2008) that are able to detect and classify these
events automatically and inform the moderator. This is done by a component of the ARGUNAUT system called the Deep Loop. As we will
see in the next section of this paper, the preconditions of ‘widening’ creative thinking can be recognized by artificial intelligence techniques
in a way that can then be fed back to the Moderators Interface.
1
For interpretation of color in Fig. 1, the reader is referred to the web version of this article.
R. Wegerif et al. / Computers & Education 54 (2010) 613–621 615
4. An illustration of coding for new perspectives
We began with a very complex Digalo map produced by a group of five undergraduates in response to the question: ‘Will the Internet
bring the world together or deepen its divisions?’. To help the analysis we reduced the map to a sequence diagram that enabled us to see
the critical branching moments more clearly (see Fig. 1).
This e-discussion map showed us the key moments when new perspectives emerged (the dots with N) and these coincided with branch-
ing moves in the sequence diagram and seemed to occur shortly after oppositions (dots with horizontal line) and open questions (dots with
vertical line). Focusing in on each key incident we could pursue qualitative interpretation of the factors leading to the emergence of new
perspectives that we followed up with key event recall interviews with the participants. Fig. 2 below illustrates a specific example of the
emergence of a new perspective. In this short extract of e-discussion, the discussants exchange ideas about awareness of other cultures,
ethics, and religions. The ‘‘new perspective” emerges when one student suggests that we may ‘‘create a divide” by becoming aware of dif-
ferent cultures, ethics, and religions. This new idea that helps us see the whole issue in a new light is coded as a new perspective.
Once we had used this method to code the maps with a breakdown into clusters i.e., sets of graph nodes indicating the cluster, such as
15, 21, 23, and 36 in Fig. 2, this was subjected to computational analysis to see if artificial intelligence techniques could match the patterns
and discover new incidents of creativity in as yet uncoded maps.
5. A computational model to explore the emergence of new perspectives
A computational model to identify places within e-discussions in which students deepen or widen a conversation provides at least two
key benefits. First, it allows us to more easily analyze discussions after the fact, exploring our theory of the existence and importance of
creative reasoning dialogue moves. Given the complexity of online e-discussions, it is difficult for a researcher to manually find such dis-
cussion moves; having a computer program assist in finding these key discussion events is extremely beneficial to our research efforts.
Second, the computational model can be used in online, ‘‘live” fashion within our ARGUNAUT system to help a teacher or a moderator
Fig. 1. Sequence diagram of a Digalo map.
616 R. Wegerif et al. /Computers & Education 54 (2010) 613–621
identify when students are thinking more creatively or more critically. While the teacher in a classroom will always be the most important
resource to students as they engage in e-discussions, a computer-based tool that assists a teacher in identifying these kinds of thinking
would be a very valuable support.
On the ARGUNAUT project we have developed a computational model called Detection ofClusters by Example (DOCE)(Mikšátko &
McLaren, 2008) that allows us to identify places in e-discussions in which students may be critically deepening or creatively widening
the conversation, as well as other types of complex conversational moves. DOCE is one of a number of tools that have been developed
on the ARGUNAUT project to assist a teacher in monitoring the on-going simultaneous e-discussions of several groups of collaborating stu-
dents. The students used the collaborative software tool Digalo to communicate with one another, with each student working on his or her
own computer, while a tool called the ‘‘Moderator’s Interface” provides the teacher with a variety of important views of the on-going dis-
cussions. One of the ‘‘views” provided to the teachers is a set of alerts that point to critical aspects of the conversation, such as whether
students are staying on topic and supporting their claims with good justifications. Some alerts are supported by relatively simple calcula-
tions (e.g., how often each student has contributed to the conversation, whether students use swear words), some by machine-learned clas-
sifiers (McLaren et al., 2007; Scheuer & McLaren, 2008), and some by the DOCE algorithm.
In particular, the DOCE algorithm identifies clusters of contributions, for example, several contributions made by different students that
indicate critical deepening or creative widening of a conversation. DOCE is based on the idea of using cluster examples to find similar clus-
ters in new discussions, similar to the ideas from the subfield of artificial intelligence known as case-based reasoning (Kolodner, 1993;
McLaren, 2003). DOCE operates by a researcher or teacher selecting a cluster in an existing e-discussion that exemplifies an interesting
pattern (e.g., connected individual contributions that provide a good example of deepening). The example cluster (also called a ‘‘model
graph” in the following text) is then used as a search query for similar clusters across other discussion maps (called ‘‘input graphs”).
The algorithm uses both structural features (e.g., the types of contributions made by students – for instance, ‘‘claim” or ‘‘question” –
and types of links between contributions – for instance, ‘‘supporting” or ‘‘opposing”) and textual features (i.e., the text provided by the stu-
dents, unigrams, bigrams, and syntactic structures from that text) of the discussion map to find similar clusters. The output of the algorithm
is a list of matching clusters in the discussion map(s), sorted according to a similarity rating, as is done by web search engines, such as
Google. DOCE provides both of the benefits of a computational model discussed above. That is, it can be used as a tool to help researchers
find and analyze clusters, such as examples of deepening or widening. Or, it can be used as a ‘‘live” classifier of clusters – characteristic
example(s) representing a cluster of a particular type are stored in the database and used later as queries for automated cluster detection.
Details about the underlying DOCE algorithm are provided in (Mikšátko & McLaren, 2008).
6. Evidence of the effectiveness and usability of the computational model
To find out how well DOCE could detect instances of creative reasoning we conducted an experiment in which we took hand-annotated
examples of deepening and widening (annotated by the members of the Exeter team on the co-author list) from actual classroom discus-
sion maps, and tested whether DOCE was able to use those examples to find the other examples of deepening and widening in our data set.
More specifically, we took 30 annotated examples of both critical deepening and the creative emergence of new perspectives from 14 dis-
tinct discussion maps, and did the following:
!For each annotated example, we ran DOCE with that annotation as the model graph against all of the other 13 discussion maps.
!We considered a relevant match to be 70% overlap, e.g., the following annotated example and found cluster would constitute a rel-
evant match, since there is a 75% node overlap (bold-faced nodes overlap):
sAnnotated example (Node1, Node3, Node4, Node5),
sRelevant cluster match (Node3, Node4, Node5, Node6).
!We varied parameters, such as the number (N) of clusters that were returned by DOCE and the relative impact of structural and tex-
tual properties on the similarity score of cluster pairs (e.g., is it more important that texts or shape types are similar?).
Fig. 2. A cluster of shapes around the emergence of a new perspective.
R. Wegerif et al. / Computers & Education 54 (2010) 613–621 617
!We evaluated recall, precision, and recall + precision on each run of DOCE. These are metrics typically used in information retrieval
and were calculated as follows:
sRecall represents the number of relevant matches in the Top Ndivided by the count of annotations in the searched map (value
between 0 and 1.0).
sPrecision is the number of relevant matches in the Top Ndivided by N(value between 0 and 1.0).
Unfortunately, since there is no ‘‘gold standard” for performing the type of retrieval task done by DOCE, there was no other computa-
tional model to compare to DOCE in our experiment. However, in an earlier experiment, reported in (Mikšátko & McLaren, 2008), we com-
pared DOCE to a simple program that returned random clusters and found that DOCE performed significantly better. While the random
algorithm is, admittedly, a ‘‘low bar” to exceed, doing significantly better than random demonstrated that DOCE is clearly finding (at least
some) clusters of interest. We considered recall to be the most important metric in our experiment, as it was most important to us to find
all of the interesting clusters in a given discussion. The number of relevant matches (i.e., precision) has somewhat lower importance since
we as researchers, and humans in general, are typically clever enough to filter out irrelevant matches.
7. Results on the effectiveness of the computational model
The results of our experiment are summarized in Figs. 3 and 4. Note, first of all, that the best results for deepening and widening are
quite reasonable (the middle bar for recall, precision, and recall + precision in each of the figures), especially for recall, the metric we con-
sider most important. By ‘‘best” result, we mean the human-annotated cluster that led to the best recall and precision values when used as
a model graph to DOCE. For instance, notice that the best deepening model graph (the middle bar in each of the first two sets of three met-
rics in Fig. 3) led to a recall of 0.80 and precision of 0.52. The average results, calculated across all of the annotated clusters (the leftmost bar
for recall, precision, and recall + precision in each of the figures), are not good (e.g., the 0.42 recall and 0.27 precision in Fig. 3 are very poor).
However, focusing on the best results is more important because, by the nature of the DOCE algorithm, only the best examples of deepening
and widening will subsequently be used as model graphs to DOCE. That is, once one finds the best model for a particular cluster type – or
the best set of models – that model (or models) will then be used as a ‘‘search probe” for all subsequent searches.
We also tested whether combining the results of multiple runs of DOCE might further improve the results. That is, we wanted to answer
the question: can multiple, high-quality clusters lead to even better results than single ‘‘best” clusters in retrieving relevant clusters?. We
implemented this combination by ranking the results according to the average relevance scores of the three single-best models. The third
bar in each set of three bars in Figs. 3 and 4 depicts these results. Notice that for the deepening cluster results shown in Fig. 3 the com-
bination approach did marginally worse (i.e., recall + precision = 1.30 for the combination approach vs. 1.33 for the single best model),
but for the widening clusters shown in Fig. 4, the combination approach did a bit better (i.e., recall + precision = 1.49 for the combination
approach vs. 1.42 for the single best model).
It is also important to note that DOCE did better in identifying ‘new perspective’ clusters, one of the hallmarks of creative reasoning
discussed earlier in this paper, than in identifying critical thinking clusters. In particular, note that the best new perspective recall
(0.93), precision (0.59), and recall + precision (1.49) in Fig. 4 improves upon the best critical thinking recall (0.83), precision (0.52), and
recall + precision (1.33) from Fig. 3.
Interestingly, the results we achieved with this data improved upon most of the results reported on a different data set, with different
annotated cluster types (Mikšátko & McLaren, 2008). More specifically, the DOCE algorithm generally performed better in finding snippets
of ‘‘creative reasoning” than it did in finding more standard argumentation structures, such as ‘‘chain of opposition” (i.e., a chain of con-
tributions by students in which they go back in forth in argue for and against a given issue) and ‘‘argument + evaluation” (i.e., a student
makes an argument which is then evaluated by another student).
8. Key event recall interviews
The fact that an intuitive coding of new perspectives could be recognized and reproduced by the DOCE algorithm as described above
confirms the reliability of the coding but not its validity to the experience of the participants in the study. We had coded certain ideas
as ‘new perspectives’ but were they surprising to the participants?. Did they really represent ‘insights’? To explore this further we inter-
viewed five undergraduate students about maps which they had participated in focussing on specific messages by them which we had la-
belled as ‘new perspectives’ and asking about their feelings at the time and how these messages had come to be sent.
To illustrate this procedure we will take one example. The dialogue map created by five students responded to the task: ‘There is a com-
petition about who is going to organize a party for the end of the course for this year 2008. The budget for this part is £2500. The winning
Fig. 3. Results of applying DOCE to the critical thinking clusters.
618 R. Wegerif et al. /Computers & Education 54 (2010) 613–621
team will get an award of £500. You are working with a team to plan this party. Discuss with your team the strengths, weaknesses, oppor-
tunities and threats related to your plan’. This activity led to an extended map the overall shape of which is illustrated in Fig. 5.
Within this map we had coded a number of clusters for the emergence of a new perspective. One such cluster is illustrated visually in
Fig. 6 and in table form (a stage closer to the form in which it was input into the DOCE algorithm) in Fig. 7. Here the ‘new perspective’ we
focus upon is an idea for an Egyptian theme for the party that is being planned which was picked up by others in the dialogue and become
an organiser for the discussion.
Amy, the student who came up with this idea (not her real name) was then interviewed about her experience as a participant in the
dialogue that constructed this map and the interview was recorded. In the interview the ‘replay’ function of the Argunaut system was used
to walk her through the creation of the map and she was asked to comment on her feelings and motivations as the map progressed. Amy
described how the spatial arrangement made it easier to follow the whole discussion than her previous experience with linear text-based
conferencing environments. She explained how she was motivated to engage by her relationships with the other participants. She read
messages from her friends first and wanted to respond to them most but was concerned about responding to those she did not know
in case she was misunderstood. She was surprised by the ideas that ‘popped into her head’ prompted and triggered by the ideas put down
by others. She was aware of other strands and ideas in the discussion. The spatial nature of the relationship between messages, with
Fig. 4. Results of applying DOCE to the ‘new perspective’ clusters.
Fig. 5. Map of SWOT analysis.
Fig. 6. A cluster around the emergence of a ‘new perspective’.
R. Wegerif et al. / Computers & Education 54 (2010) 613–621 619
multiple themes and idea co-present in the space of the map, helped to trigger new ideas. In the case of the ‘new perspective’ in Fig. 6 a
previous posting triggered her childhood memory of ‘an amazing party’ with an Egyptian theme.
All the five students interviewed confirmed Amy’s account of how some of the ideas that came to them triggered by previous postings in
the discussion surprised them. All also agreed that one powerful motive for them in coming up with new perspective was disagreement
with ideas posted previously by others, thus offering an account of the relationship observed in the coding between oppositions and the
subsequent emergence of new ideas.
9. Discussion
There is currently a great deal of interest around the world in teaching for creativity which is widely seen as one of the core ‘21st century
skills’ required for flourishing in the emerging knowledge age (Wegerif & De Laat, in press). However, debates continue as to what crea-
tivity is, and even if it exists at all in a way that can be reliably recognized and so made use of in practical teaching and learning (Craft,
2005). In this study we explored the emergence of creativity in online graphically mediated dialogues. Our account of creativity is specific
to Bakhtinian dialogic theory focussing not, as some accounts of creativity do, on the social recognition of the value of a product, but on the
expansion of understanding that comes from seeing an issue from a new perspective. Our findings are suggestive that: (1) the emergence of
new perspectives in a graphical dialogue map can be coded for using an intuitive interpretation of the construction of meaning in a dia-
logue in a way which can be recognized reliably by a machine algorithm; (2) that the trigger events leading to the emergence of new per-
spectives are most commonly disagreements and (3) that the spatial representation of messages in a graphically mediated synchronous
dialogue offers a pedagogical affordance for creativity.
These results are potentially important. It is significant that the DOCE algorithm is capable of finding examples of creative reasoning,
given prior, annotated examples of such reasoning in earlier discussions. Therefore, the DOCE algorithm is a tool that either a researcher
or a teacher can use to pinpoint and evaluate creative reasoning in the context of real e-discussions. This technique has the potential to
inform moderators when creative thinking and critical reasoning is occurring in maps as well as when it is not occurring and it might
be a good time to intervene with a stimulus or a challenge.
The findings of this research broadly supports the value of a dialogic theoretical framework that sees thinking not as an imperfect
expression of any underlying logic or formal system but as a flow of meaning between embodied voices or perspectives. From this perspec-
tive it is crucial that research on online dialogue does not only analyze the externally visible traces of a dialogue in the form of the elec-
tronic data but also explores the internal perspective of the participants. We have illustrated how a methodology combining qualitative
interpretation of the flow of meaning in discourse with rigorous machine-learning techniques and key event recall interviews can be effec-
tive in illuminating some of the mechanisms of online dialogues.
Acknowledgments
We are grateful for the support of the EC and want to acknowledge the help of Maarten De Laat and the rest of the ARGUNAUT project
team, especially Rakheli Hever, Reuma DeGroot, Raul Drachman, and Ulrich Hoppe. The 6th Framework Program of the European Commu-
nity, Proposal/Contract No. 027728, sponsored this research.
References
Andriessen, J. (2008). Arguing to learn. In K. Sawyer (Ed.), Handbook of the learning sciences. Cambridge: Cambridge University press.
Baker, M. J., Quignard, M., Lund, K., & Séjourné, A. (2003). Computer-supported collaborative learning in the space of debate. In B. Wasson, S. Ludvigsen, & U. Hoppe (Eds.),
Designing for change in networked learning environments: Proceedings of the international conference on computer support for collaborative learning 2003 (pp. 11–20).
Dordrecht: Kluwer Academic Publishers.
shape text Critical reasoning Dialogic
reasoning
Dialogic
engagement
Moderation
50 Theme ideas –
circus, with lots of
different people
doing acts and we
could have a go at
doing them. –arabian
nights, we could
have shescha and
belly dancers and
magic tricks and
mind readers etc…
[claim(qualifier)] [opening
statement+
new
perspective]
0 0
54 Live bands would be
cool
[claim(support)] [build on] 0 0
59 I like those theme
ideas, what about
Egyptian with lots
of gold things? My
dad did it once for
an army summer
ball and it looked
amazing
[claim(support)+
question+
reason
(unpacking)]
[new
perspective]
[empathy] 0
63 Yeh that idea is
braaaap! Very cool.
[claim(support)] 0 [addressivity+
affection]
0
Fig. 7. Coding of the cluster.
620 R. Wegerif et al. /Computers & Education 54 (2010) 613–621
Bakhtin, M. (1986). Speech genres and other late essays. Austin: University of Texas.
Bruns, A. (2007). Beyond difference: Reconfiguring education for the user-led age. Paper presented at the ICE 3 (Ideas, Cyberspace, Education) conference at Ross Priory, Loch
Lomond, Scotland, 21–23 March 2007. <http://snurb.info/publications> Accessed 10.08.09.
Craft, A. (2005). Creativity in schools: Tensions and dilemmas. London: Routledge.
De Groot, R., Drachman, R., Hever, R., Schwarz, B., Hoppe, U., Harrer, A., et al. (2007). Computer supported moderation of e-discussions: The ARGUNAUT approach. In Clark
Chinn, Gijsbert Erkens, & Sadhana Puntambekar (Eds.), Mice, minds, and society – The computer supported collaborative learning (CSCL) conference 2007, Vol. 8 (pp. 165–167).
International Society of the Learning Sciences.
Dunbar, K. (1995). How scientists really reason: Scientific reasoning in real-world laboratories. In R. J. Sternberg & J. E. Davidson (Eds.), The nature of insight (pp. 365–395).
Cambridge, MA: MIT Press.
Hever, R., De Groot, R., De Laat, M., Harrer, A., Hoppe, U., McLaren, B.M., et al. (2007). Combining structural process-oriented and textual elements to generate awareness
indicators for graphical e-discussions. In The proceedings of the conference on computer supported collaborative learning (CSCL-07) (pp. 286–288).
Kolodner, J. (1993). Case-based reasoning. San Francisco: Morgan Kaufman.
Leitch, S. (2006). Prosperity for all in the global economy world class skills. Final report of the leitch review of skills. London: HMSO/HM Treasury.
McLaren, B. M. (2003). Extensionally defining principles and cases in ethics: An AI model. Artificial Intelligence, 150, 145–181.
McLaren, B. M., Scheuer, O., De Laat, M., Hever, R., De Groot, R., & Rosé, C. P. (2007). Using machine learning techniques to analyze and support mediation of student e-
discussions. In The proceedings of the 13th international conference on artificial intelligence in education (AIED 2007) (pp. 141–147). IOS Press.
Mikšátko, J., & McLaren, B. M. (2008). What’s in a cluster? Automatically detecting interesting interactions in student e-discussions. In B. Woolf, E. Aimeur, R. Nkambou, & S.
Lajoie (Eds.), Proceedings of the 9th international conference on intelligent tutoring systems (ITS-08), lecture notes in computer science, Vol. 5091 (pp. 333–342). Berlin:
Springer.
NACCCE, (1999). All our futures: Creativity, culture and education: National advisory committee on creative and cultural education. London: DfEE and DCMS.
Nystrand, M. (1997). Open dialogue: Understanding the dynamics of language and learning in English classrooms. New York: Teachers College Press.
Sawyer, K. (2006). Explaining creativity. Oxford: Oxford University Press.
Scheuer, O., & McLaren, B. M. (2008). Helping teachers handle the flood of data in online student discussions. In B. Woolf, E. Aimeur, R. Nkambou, & S. Lajoie (Eds.), Proceedings
of the 9th international conference on intelligent tutoring systems (ITS-08), lecture notes in computer science, Vol. 5091 (pp. 323–332). Berlin: Springer.
Toulmin, S. E. (1958). The uses of argument. Cambridge, UK: University Press.
Van Eemeren, F. H., & Grootendorst, R. (2004). A systematic theory of argumentation.The pragma-dialected approach. Cambridge: Cambridge University Press.
Walton, D. (2000), The place of dialogue theory in logic, computer science and communication studies, synthese. International Journal for Epistemology, Logic and Philosophy of
Science, 123, 2000, 327–346.
Wegerif, R. (2005). Reason and creativity in classroom dialogues. Language and Education, 19(3), 223–237.
Wegerif, R., & De Laat, M.F. (in press). Reframing the teaching of higher order thinking for the network society. In S. Ludvigsen, A. Lund & R. Säljö (Eds.), Learning in social
practices. ICT and new artifacts – Transformation of social and cultural practices. Pergamon.
Wegerif, R. (2007). Dialogic, educational and technology: Expanding the space of learning. New York: Springer-Verlag.
Wegerif, R., McLaren, B.M., Chamrada, M., Scheuer, O., Mansour, N. & Mikšátko, J. (2009). Recognizing Creative thinking in graphical e-discussions using artificial intelligence
graph-matching techniques. In C. O’Malley, D. Suthers D., P. Reimann, & A. Dimitracopoulou (Eds.), Computer supported collaborative learning practices.Proceedings of the
9th international conference on computer supported collaborative learning (CSCL-09), Vol. I (pp. 108–112). International Society of the Learning Sciences, Inc. ISBN: 978-1-
61584-137-0.
R. Wegerif et al. / Computers & Education 54 (2010) 613–621 621
