Content uploaded by Dimitra Tsovaltzi
Author content
All content in this area was uploaded by Dimitra Tsovaltzi on Jan 29, 2021
Content may be subject to copyright.
Note: This is a preprint of the paper to be published in C&E
Group Awareness Support and Argumentation Scripts for Individual
Preparation of Arguments in Facebook
Abstract.
This study investigates how group awareness support and argumentation scripts influence
learning in social networking sites like Facebook, which may be conducive to informal
learning, but often lacks argumentative quality. Supporting participants’ group awareness
about the visibility of the arguments they construct and about prospective future debate
with peers in order to promote argument quality may be particularly suited for learning in
Social Networking Sites. Additional argumentation scripts may directly foster
argumentative knowledge construction. In a 2 × 2 study (N = 81), we isolated and
investigated the effects of group awareness support and argumentation scripts during
individual preparation in a Facebook app on domain and argumentative knowledge. Our
results reveal that group awareness support of upcoming argumentative processes can be
counterproductive for learning in Social Networking Sites. Argumentation scripts in
Facebook may remedy possible negative effects of such awareness. Process analysis
showed that group awareness support promotes individual argument elaboration but
reduces broad analysis of the domain.
1. Argumentative Knowledge Construction in Social Networking Sites
Social Networking Sites (SNS)
1
, such as Facebook, LinkedIn, Google+, are rapidly growing communication
platforms. SNS provide easy platform-independent access and almost unrestricted interactivity for sharing ideas
and opinions. SNS can therefore be conducive to online dialogic exchange and potentially to argumentative
knowledge construction (AKC
2
; Weinberger & Fischer, 2006). AKC is the deliberate practice of elaborating
learning material by constructing formally and semantically sound arguments with the goal of gaining
argumentative and domain knowledge. Next to argument diagramming, argumentation scripts are among the
most prominent approaches to foster AKC in CSCL
3
environments (Noroozi, Weinberger, Biemans, Mulder, &
Chizari, 2013; Scheuer, Loll, Pinkwart, & McLaren, 2010). However, there is little known about the extent to
which these approaches can be applied to learning in SNS (McLoughin, & Lee, 2010; Tsovaltzi, Weinberger,
Scheuer, Dragon, & McLaren, 2012). SNS and the social web in general have arguably become a large arena of
informal learning (Greenhow, 2008; Greenhow & Robelia, 2009). However, a lot of concern has been expressed
right from the early days of such technology for the poor quality of argumentation, which tends to dismiss
conflicting opinions and inconsistencies, and of the quality of the emergent knowledge in that context (Kanuka
& Anderson, 1998). Currently prominent approaches to facilitate AKC in CSCL aim at technologically
enhancing group awareness during group work in order to support learners’ regulation of group processes and
enhance group work (Buder & Bodemer, 2008). At the same time, providing argumentation scripts that specify
and prompt learners to use formal structures of arguments were found to enhance the quality of AKC processes
and outcomes (Weinberger, Stegmann & Fischer, 2010).
1.1 Facebook for Academic Purposes
Dialogic exchange in social media has from the beginning lacked the argumentative quality that is needed
for learning, especially with respect to elaboration of arguments and critical evaluation of novel evidence
(Kanuka & Anderson, 1998). SNS in particular were created for interactions at the personal level with users
1
SNS: Social Networking Sites
2
AKC: argumentative knowledge construction
3
CSCL: Computer Supported Collaborative Learning
typically airing private opinions. However, there is research that supports the use of Facebook for academic
purposes and learning. Facebook and other SNS seem to foster social aspects of academic life including actual
friendships (Bennett, 2010; Pempek, Yermolayeva & Calvert, 2009; Ryan, Magro & Sharp, 2011; Selwyn,
2009). SNS also afford information sharing and learning (Dabner, 2011; Laru, Näykki, & Järvelä, 2011). A
connection has been found between social interaction and the exchange of study-related knowledge in SNS
(Wodzicki, Schwämmlein, & Moskaliuk, 2012) as well as between active information seeking and bridging
social capital, i.e. connecting to cooperation partners and thus, tapping into diverse human resources including a
larger potent public (Lampe, Vitak, Gray & Ellison, 2012).
1.2 Argumentation Scripts
The recognition that argumentation skills are sparse, but necessary to critically process information and
participate in civic society accentuates the value of such skills especially in higher education (Marttunen &
Laurinen, 2001). Collaborative learning research suggests that argument elaboration can promote knowledge
construction, and can greatly benefit from additional support through scripts, i.e. socio-cognitive structures that
specify what learners are to do in collaborative learning scenarios, activating existing internal scripts or
organising internally represented disperse elements of scripts (e.g. Baker & Lund, 1997; Fischer, Kollar,
Stegmann & Wecker, 2013; Stegmann, Wecker, Weinberger, & Fischer, 2012; Weinberger et al., 2010).
Learners, for instance, can be prompted by scripts to provide support or counterarguments for their claims, which
is often realized by sentence openers in the text boxes through which learners communicate. This can help them
elaborate the argument, consider multiple perspectives, and gain argumentative knowledge (Weinberger et al.,
2010). Learners can thus be guided to identify and productively resolve conflicting opinions and inconsistencies,
e.g. by providing concrete evidence and valid justifications for claims and counterclaims (Andriessen, 2006;
Baker, 2003; Belland, Glazewski & Richardson, 2008). Even though such processes are related in the literature
with attitude change (e.g. Eagly & Chaiken, 1995; Erber, Hodges, & Wilson, 1993) and may ultimately lead
learners to explore alternative perspectives and to re-consider initial standpoints, attitude change has rarely been
analysed in this context (Tsovaltzi et al., 2012). Supporting learners through structuring the construction of pro
and contra arguments can facilitate argumentative knowledge construction (Weinberger et al., 2010). Moreover,
scripts can foresee individual phases, in which learners prepare representations of their individual knowledge,
and subsequent collaborative learning phases, in which learners compare and combine their individual solutions
(e.g., Asterhan & Schwarz, 2007; Baker, 2009; van Boxtel, van der Linden, & Kanselaar, 2010; van Dijk,
Gijlers, & Weinberger, 2013). Such orchestration of different learning arrangements may give students the
opportunity to prepare in their own time and may result in different ways of processing and planning
argumentation (Marttunen & Laurinen, 2001). Argumentation scripts could also be applied to foster individual
learning only, although it has been shown to be less effective than for collaborative learning (Weinberger et al,
2010). One possible explanation is that scripts may not only activate or complement individual internal scripts,
but also raise awareness of the expected quality standards of the discourse and of argumentative group processes
defined, for example, by the learning partners’ scripted roles, like the analyst and the critic (Fischer, Kollar,
Stegmann & Wecker, 2013; Weinberger, 2011).
1.3 Group Awareness Support
Collaborative learners need to be aware of different aspects of the group (processes) to regulate group work:
For instance, knowledge awareness pertains to how knowledge is distributed and shared within a group. Social
awareness addresses information on how group members are involved in group processes emotionally or
motivationally. Group-structural awareness involves knowledge about group members’ roles and
responsibilities, hierarchies within a group, or processes a group is following (Gutwin, Greenberg, & Roseman,
1996). Any (combination) of these and other types of group awareness, e.g. behavioural, cognitive, and social
awareness, have been facilitated with tools that aggregate and visualize group data to compensate for the lack of
physical or conversational cues in CSCL environments (Bodemer & Dehler, 2011; Gutwin & Greenberg, 2002;
Gutwin et al., 1996). Moreover, awareness tools may provide additional group information beyond what is
possible in face-to-face interaction (Carroll, Rosson, Farooq, & Xiao, 2009).
Overall, the effects of awareness tools on learning processes and outcomes are inconclusive yet with a
majority of studies investigating effects on group processes (e.g., Kimmerle, Cress, & Hesse, 2007). Existing
research shows that group awareness may not necessarily result in argument elaboration and domain learning
gains, as students may rather concentrate on refutation of arguments or formal aspects of argumentation than on
reflecting multiple ideas (Baker, 2003). Group awareness does seem to impact – especially socio-emotional and
motivational – group processes. For example, Buder and Bodemer (2008) investigated an awareness tool that
visualized differences of opinion in a CSCL environment to strengthen scientifically correct minority opinions
and thereby promoted critical argument elaboration during collaboration. Group awareness is also positively
related to performance and to process satisfaction of computer-supported learning groups (Phielix, Prins, &
Kirschner, 2010). Still, only few awareness studies directly address motivational issues of group learning. For
instance, raising group awareness about future peer collaboration in individual preparatory phases can activate
motivational sets that are beneficial to learning (Benware & Deci, 1984).
Since SNSs have been designed to share social information, SNS could be particularly suited to foster online
learners’ group awareness. Moreover, given the mainly asynchronous mode of SNS discussions, one could
expect that discussants are aware of and may anticipate upcoming debate with peers involving, for instance,
critical evaluation, amendment, or refutation of one’s initial arguments. Yet, it is not clear how such group
awareness influences the learning processes and outcomes in SNS. In general, it is not known if and how these
results transfer to learning in social networks and how group awareness aligns with argumentation scripts to
foster AKC in SNS. Additionally, although scripting and awareness have been compared before, a need to
investigate their positive or negative synergetic effect has been advocated in order to understand how they
support CSCL as well as to offer practical advice for their implementation (Stahl, Law, & Hesse, 2013).
We investigate how group awareness support interacts with an argumentation script. We support group
awareness by highlighting that posted arguments are visible and will potentially be positively or critically
evaluated, or amended by a large group of peers. We realize the argumentation script through the use of an
ontology implemented in a Facebook App to allow learners to classify their arguments before posting them in the
App.
Our hypotheses are that group awareness support and argumentation scripts influence both, processes and
outcomes of collaborative learning. Learners with the argumentation script will produce (i) higher argument
quality and (ii) will exhibit better argument structure than those without the argumentation script since the script
suggests a specific argument structure. Learners with group awareness support (iii) will construct better
elaborated arguments, and (iv) will address multiple perspectives more than learners without group awareness
support to anticipate possible critiques and amendments of future readers. Based on these enhanced processes, we
assume additional effects of group awareness support and argumentation scripts on learning outcomes: (v) Group
awareness support and argumentation scripts are assumed to interact in fostering domain learning gains due to
higher elaboration of the learning material –group awareness support will motivate and argumentation script will
enable learners to elaborate knowledge. (vi) The argumentation script will foster learning gains on knowledge
about argument quality (rules about argument quality) and subjective learning gains on argument quality (how
much students think they produced arguments of better quality), since the script may activate or complement
existing internal scripts (Fischer et al., 2013). (vii) Group awareness support will foster subjective learning gains
on argument elaboration, corresponding to the more elaborated arguments they produce. Moreover, we
hypothesise (iix) that we will observe attitude change pertaining to open issues in the domain and (ix) far transfer
(retention) of domain learning gains.
2. Method
To investigate the effects of group awareness support in Facebook and of argumentation scripts, as well as
their interaction, on objective and subjective learning processes and outcomes, we did a comparative study where
students worked individually and their expectations about future argumentative group processes were raised. We
conducted a 2 × 2-factorial study with German university students (N = 81, Age: M = 22, SD = 5.5, Gender: F/M
= 70.4%/29.6%). Participants were taking part in a course for teacher trainees and joined the study optionally.
The study lasted approximately 120 minutes. Before the intervention, all participants took a knowledge pre-test,
filled in a pre-questionnaire, and read a text on behaviourism – a standard part of the teacher trainee curriculum –
including facts and definitions, as well as some main arguments for and against behaviouristic principles in
education. This text was intended to provide a basis for constructing high-level arguments. After that, and for 25
minutes, participants prepared arguments on the question “Should behaviouristic principles be applied in the
classroom?” with a different intervention depending on their condition. In the conditions with group awareness
support, the experimenter informed the participants that their arguments could be published in the standard
course forum after the completion of the experiment, where their peers can express their own opinions, comment
and amend the posted arguments. This is an officially maintained forum, where students can practice tests,
discuss with each other and get feedback from an assigned person – a student from a more advanced semester.
The groups without group awareness support did not receive this information and learned individually without
further ramifications.
An argumentation script was provided in the argumentation script conditions through a Facebook App with
an integrated ontology that helped students annotate their arguments with general argument types (claim,
counterclaim, evidence, rebuttal) as well as domain specific categories (ethics vs. effectiveness of behaviouristic
principles, example vs. every-day knowledge vs. empirical result) (Figure 1). Participants without the
argumentation script used a similar App without an integrated ontology. All participants had the opportunity to
familiarize themselves with the version of the App they used. Time on task was controlled. After the intervention,
all participants took a knowledge post-test and filled out a post-questionnaire.
Finally, in the control, there was no group awareness support, that is, participants were not told that their
arguments could be published after the experiment, and did not use the argumentation script.
2.1. Instruments
We collected control variables with a pre-questionnaire on demographics, ambiguity tolerance (a = .65),
interest in and attitude to collaborative learning (a = .574), subjective knowledge of learning theories (one item)
and lesson planning (one item), familiarity with SNS, Facebook, and PC (a = .548). The post-questionnaire used
Likert scales (1-5) and included a treatment check for group awareness support (two items, e.g., “When I was
formulating my arguments, I was constantly aware that they would be read by other people”), controlled for
motivation and interest (two items, e.g., “I was motivated to formulate arguments”) and included self-reports on
possible distractions, e.g. use of chat or email during the experiment. Moreover, we tested outcome variables
Figure 1: The Facebook-App with the integrated ontology for argumentation script (neuer Post
= new post, Ethik = ethics, Gefällt mir = like, antworten = respond. Forschungsergebnis =
research result, Beispiel = example; the symbols from top to bottom mean “claim”, “evidence”,
and “counterclaim”)
through a scale on subjective learning gains on argument quality (two item: “In this online learning session, I
learned to formulate and structure my arguments correctly”), and a scale on learning gains on knowledge of
argument quality (five items, e.g., “In a discussion, warrants are as important as claims”, a scale on subjective
learning gains on argument elaboration (one item, “In this online learning session, I learned to elaborate my
arguments better”).
Remotely building on knowledge sharing types by Ardichvili, Page, and Wentling (2003), we also collected
self-report data on learner’s attitude change. Attitude change was operationalised through two sub-constructs:
argumentation style and SNS interaction style, representing possible different aspects of discourse responsibility
in SNS discussions. Argumentation style captured students’ attitude in terms of argumentative caution,
confidence, and multi-perspectives, and SNS interaction style captured students’ attitude in terms of disclosure of
personal information and knowledge sharing. Argumentation style distinguishes between careful discussant (e.g.
“In discussions, I deliberate about my arguments”) and open discussant (e.g. “In discussions, I remain faithful to
my intuitive perspective”), confident discussant (e.g. “In discussions, my arguments are influential and
effective”) and flexible discussant (e.g. “In discussions, I adjust my arguments to those of others”), multi-
perspective discussant (e.g. “In discussions, one should hear as many arguments as possible”) and focused
discussant (e.g. “In discussions, presenting many arguments is non-constructive”). SNS interaction type
comprised the constructs personal communication oriented (e.g. “SNS are suitable for personal communication”)
and professional communication oriented (e.g. “SNS are suitable for establishing contact to colleagues”), and
exchange oriented (e.g. “SNS are suitable for communicating interesting personal views”) and information
oriented (e.g. “SNS are suitable for answering and posing questions”). The data was collected in the pre- and the
post-questionnaire for conditions with group awareness support, so that changes in the participants’ attitude can
be observed. For the other conditions, the data was only collected in the post-test so as not to raise the
expectation of measuring differences of argumentation and SNS interaction style as a result of the intervention.
We tested the participants’ domain learning gains with a knowledge test that included 24 closed items and
two open questions. The closed items tested knowledge on behaviourism and were extracted from the text on
Behaviourism that all participants read. After analysing the items regarding difficulty, discriminatory power and
learning gains, four items were excluded from the further calculation, because they did not seem to measure the
same as the rest of the items. The final internal consistency of the closed items is around acceptance levels (αpre =
.469; αpost = .567).The two open questions asked students to name weaknesses, respectively strengths, of
behaviourism. After creating a pool of model answers (10 for weaknesses and 9 for strengths, e.g.
“Behaviourism ignores high-order cognitive processes”, “Behaviourism focuses on well-defined, measurable
behaviours.”) two independent coders estimated the performance of each subject (number of answers that were
comparable to one of the model answers). At the pre-test the inter-rater reliability was sufficient for both
questions (first question: κ=.882; second question: κ = .816), as it was in the post-test (first question: κ = .833;
second question: κ = .732). The means of the two ratings were used for further analysis. A maximum score of
four answers was defined for each of the open questions, so that four correct answers represented 100%. In order
to create an overall score of domain-specific knowledge, closed and open questions were summed up and
divided by 28, which was defined as a total maximum of correct answers (20 + 8). The internal consistency of
the pre-test including the open questions is α = .543 and in the post-test α = .632. The discriminatory power in
the pre-test is r = .826 and in the post-test r = .856.
For the process analysis, we used a coding scheme that was developed based on the multi-dimensional
scheme from Weinberger and Fischer (2006) and further adjusted for learning in SNS, as well as the
particularities of the domain and the argumentation ontology. For this analysis we only included two dimensions
of the scheme. The other dimensions handle interactions in collaborative scenarios and did not apply. The inter-
rater reliabilities for both dimensions were good; karg =.69, p = .000; kepi =.66, p = .000. According to our
hypotheses we further operationalized the process variables argument quality, argument structure, argument
elaboration, and multiple perspectives. Argument quality is defined as the sum of points from the different
quality justifications produced; three points were given for scientific justifications with reference of scientific
results (e.g. “Shaping is another point to fostering learning. / At the beginning, all approximations are rewarded
and later explicitly only the important ones.”, which refers to Skinners experiments from the text on
behaviourism), two points for justifications with examples (e.g. “Grades are a form of behaviourism, the student
is rewarded with a good grade and penalised with a bad grade”, which refers to positive reinforcement and
positive punishment), and one point for every-day knowledge without source reference (e.g. “Everybody wants
to learn if he is expecting at the end a task that he likes doing”). Argument structure is defined as the mean
number of high quality justifications per statement, which is what the ontology taught. For example, an argument
with three justifications of which two are examples and one a scientific reference has higher argument structure
than an argument of three justifications all of which are examples, to capture the fact that formal structure has its
own value but cannot be separated from content. Argument elaboration is the mean number of relations between
different theories – the more relations the better the elaboration. For example, an argument like “Consciously
applied behaviourism can also lead to the exact opposite of what is expected” is well elaborated when it is
warranted by a number of justifications like “Other students might feel neglected or discriminated if the others
are always being praised and therefore even more intimidated”, as opposed to merely posting reproductions from
the text on behaviourism. Finally, multiple perspectives is the relation of pro and contra justifications. An
argument considers multiple perspectives if it has a balanced number of pro and contra arguments. For example,
a series of justifications like “Punishment often leads only to a short term suppression of the unwished
behaviour.” / “The more aversive a stimulus, the longer is the timeframe that the not wished behaviour does not
occur.” / “If a kid experiences a lot of pain after an unwished behaviour, this behaviour will not come about for a
long time” / “To inflict a lot of pain on a kid as punishment is morally not ok.”
Finally, the results of the course exam that was held 6-12 weeks after the experiment were at our disposal to
use as a test of far transfer for retention. This included multiple choice questions similar to the ones in our
knowledge test on diverse learning theories including questions on behaviourism. It also included four questions
from our knowledge test that were different from the other behaviourism questions in the exam. We used the
course exam as a measure of argumentative learning on Facebook (that was common in all conditions) against
the standard university teaching that also included the course forum. 62 out of the 81 participants took the final
exam, and 313 non-participants who took the standard course.
3. Results
There were no significant differences between the groups with respect to the socio-demographic data and the
knowledge test prior to the intervention. The conditions were also comparable regarding their prior knowledge:
There were no significant main effects, Farg_sup(1,77) = .00, p = .960, ηp² = .00; Fsoc_aw (1,77) = 1.61, p = .208, ηp²
= .02 or interactions F(1,77) = .19, p = .668, ηp² = .00, in the pre-test.
3.1. Domain Learning Gains
Regarding domain learning gains from pre-test to post-test, an ANOVA with the experimental factors and
the test time as independent variables showed a large significant main effect on learning gains, F(1,77) = 221.73;
p = .000, ηp² = .74 - all groups learned significantly between pre-test and post-test. The rather large effect size
indicates that learning in the Facebook App is possible independent of condition. . There were also tendencies for
an interaction effect between awareness and argumentation script, F (1,77) = 3.26, p = .075, ηp² = .04, and for a
main effect of group awareness support, F (1,77) = 3.14, p = .080, ηp² = .04, but no main effect for argumentation
script, F(1,77) = .41, p = .522, ηp² = .01 (Figure 2). Helmert contrasts showed that the control was significantly
better than the group awareness condition, t(77) = 2.52, p = .014, d = .856. There were no other significant
differences for the domain knowledge test.
The results for output variables measured in the post-questionnaire supported our hypothesis for the
argumentation script: There was a main effect for argumentation script on learning gains of knowledge about
argument quality, F(1,77) = 4.13, p = .046, ηp² = .05, and a significant and strong main effect of subjective
learning gains on argument quality, F(1,77) = 11.99, p = .001, ηp² = .14. A main effect for argumentation script
on subjective knowledge of argument elaboration was also found, F(1,77) = 9.52, p = .003, ηp² = .11, but not for
group awareness support, which was our hypothesis. Learners in the group awareness condition reported lower
subjective learning gains on argument elaboration than the control group, t(77) = -2.11, p = .042, d = -0.67.
Fig. 2: Domain Learning Gains: Mean difference from pre- to post-test
3.2. Process Analysis: Argumentative Quality, Structure, Elaboration, and Multiple Perspectives
Consistent with our hypotheses, process analysis showed a significant medium size main effect for
argumentation script on argument quality F(1,77)=4.689, p =.033, ηp² =.06, and a significant, but rather small
interaction effect F(1,77)=4.179, p =.044, ηp² =.05 (Figure 3). There was also, a significant large effect for
argumentation script on argument structure F(1,77)=10.462, p =.002, ηp² =.12 (Figure 4), but no interaction
effect. Although the analysis of variance showed no significant effects on argument elaboration (Figure 5),
Polynomial Contrasts showed significant differences between the groups (Table 1) supporting our original
hypothesis of a main effect for group awareness support. Contrary to our hypothesis, there were no interaction
effects on multiple perspectives (Figure 6).
Fig. 3: Argument Quality: Mean scores of conditions on argument quality
0
0.05
0.1
0.15
0.2
0.25
0.3
Arg-Script No Arg-Script
Mean Domain Learning Gains
Group Awareness
Support
No Group Awareness
Support
0
0.5
1
1.5
2
2.5
3
3.5
4
Arg-Script No Arg-Script
Mean Argument Quality
Group Awareness
Support
No Group Awareness
Support
Fig. 4: Argument Structure: Mean number of supportive arguments per main argument
Fig. 5: Argument Elaboration: Mean scores of conditions on argument elaboration
Table 1: Polynomial contrasts for argument elaboration
Polynomial Contrasts
T(df=77)
p
Cohens d
Script vs. Control
-.273
.786
-0.09
Script vs. Awareness
-2.143
.035
-0.67
Script vs. Combination
-.325
.746
-0.10
Awareness vs. Control
1.871
.065
0.62
Awareness vs. Combination
1.845
.069
0.56
0
0.5
1
1.5
2
Arg-Script No Arg-Script
Mean Argument Structure
Group Awareness
Support
No Group Awareness
Support
0
0.5
1
1.5
2
2.5
3
3.5
4
Arg-Script No Arg-Script
Mean Argument Elaboration
Group Awareness
Support
No Group Awareness
Support
Fig. 6: Means of conditions on multiple perspectives
Descriptive statistics of the process analysis show that the group awareness condition created the least
arguments, followed by the control, whereas the condition with script but without raised group awareness
support developed the most arguments (Table 2). This resulted in a significant main effect for the script, F(1,77)
= 6.51, p = .001, ηp² = .20.
Table 2: Number of arguments per condition
Group
awareness
support
No group
awareness
support
Argumentation
script
7.48 (4.72)
9.60 (3.35)
No argumentation
script
4.80 (.2.29)
6.85 (2.96)
There was a main effect for the treatment test for group awareness support, F(1,77) = 4.29, p = .042, ηp²
= .05; as expected participants in the group awareness conditions reported more awareness. There were no
differences regarding motivation or distractions.
Table 3: Principle component analysis
0
5
10
15
20
25
30
35
40
45
50
Arg-Script No Arg-Script
Mean Multiple Perspectives
Group Awareness
Support
No Group Awareness
1
2
3
4
Social Networks are suited for
…posing questions and responding questions by others.
.754
… keeping contact with colleagues.
.724
… exchanging ones’s interesting views.
.688
.337
… conducting personal discussions.
.594
… staying in touch with friends round the hour.
.585
… asking colleagues quick questions.
.566
… getting hold of facts.
.555
… experiencing the inspiring multiplicity and SNS-dynamci.
.523
.390
… taking advantage of collective intelligent for one’s own work.
.447
.331
… amusing oneself.
.440
In discursive discussions,
… I present my arguments openly.
.770
… my arguments are influential and effective.
.719
… I ponder well on my arguments.
.715
Table 4: Correlation of latent factors (PCA post-questionnaire) * theoretical factors (PCA pre-questionnaire)
Pre-test
Post-test
PC1:
confident and
careful
discussant
PC2: SNS
participator
PC3:
consumer of
information
PC4: efficient
communicator
PC1: SNS participator
r
0.054
0.921
0.687
0.137
p
.635
.000
0.000
0.223
N
81
81
81
81
PC2: confident and
careful discussant
r
0.839
0.023
-0.085
0.789
p
.000
0.840
0.448
0.000
N
81
81
81
81
PC3: multi-perspective
discussant
r
0.361
0.310
0.437
0.107
p
0.001
0.005
.000
0.340
N
81
81
81
81
PC4: reserved
discussant
r
-0.360
-0.040
0.457
0.020
p
0.001
0.725
0.000
0.863
N
81
81
81
81
3.3. Attitude Change
A principle component analysis (PCA) was done to find latent factors in the questionnaire on attitude
change. We first did an exploratory analysis on the pre-questionnaire data with 41 data points, which was more
than five times the number of variables (5 ∙ 5 = 35). After screening items that did not load on any factor with
more than 0.4 and applying varimax rotations we arrived at a model of four intermediate factors that fit our data.
Then we did a confirmatory analysis for those factors with the data from the post-questionnaire (N = 81). Two
items were eliminated in the end. The factor analysis resulted in four argumentation and SNS interaction styles
(Table 3). After validation comparing the factors used for the confirmatory analysis based on the pre-
questionnaire PCA and the latent factors of the post-questionnaire PCA (Table 4, white: expected correlation,
dark grey: unexpected correlation, light grey: no correlation), only the three first factors that had moderate
internal consistency were considered for further analysis: SNS participator (Table 3: marked leftmost column, V
=14.13) capturing the attitude that SNS have many advantages, apre=.811, apost = .810, confident and careful
discussant (Table 3: marked second to leftmost column, V =16.46) with 10 out of 18 items for the original
theoretical dimension argumentation, apre = .821, apost = .771, and multi-perspective discussant (Table 3: marked
second rightmost column, V =10.34) with a combination of items from both dimensions and the overall attitude
that information is sought and offered without expressing opinions and being confrontational, apre = .699, apost =
.677. Our hypothesis was that our intervention would cause attitude change. Therefore, we compared changes
… I deliberate on the arguments of others.
.672
.452
… one can explain one’s own arguments better.
.592
… one should elaborate on arguments by others.
.543
… one should listen to as many viewpoints as possible.
.742
Social Networks are suited for having access to an unlimited richness of information.
.495
.641
In discursive discussions, I deliberate about alternative arguments.
.462
.593
Social Networks are suited for getting informed.
.504
.514
In discursive discussions,
… I can check and improve my arguments.
.425
… I don’t enjoy sharing my arguments.
.765
… are many arguments less constructive.
.552
… my arguments are criticised and attacked.
.482
… are challenging arguments contra-productive.
-.332
.478
… I like remaining faithful to my intuitive views.
.380
.474
from pre to post-questionnaire for the two group awareness conditions which filled in the questionnaire prior and
after the intervention, using the same latent factors. A multivariate ANOVA with repeated measures gave
interesting results. The attitude of all participants changed significantly for two out of three factors: learners
became more confident and careful discussants, F(1;39) = 1103.408; p =.000; ηp²=.97, and multi-perspective
discussants, F(1;39) = 8.446; p =.006; partial η²=.18. For the factor SNS participator, there was a significant
interaction of time and condition, F (1;39) = 4.950; p =.032; η²=.1; the learners in the group awareness condition
acquired a more negative attitude, whereas the learners in the combination condition acquired a more positive
attitude. There was also a marginal interaction effect for learners of the combination condition towards becoming
an open-minded-discussant, F(1;39) = 3.600; p =.065; η²=.09. When comparing the argumentation and SNS
interaction styles of all four conditions as assessed by the post-questionnaire only, we found no significant
results.
3.4. Far Transfer - Retention
The results from the course exam revealed a significant effect for the extra questions when comparing
experiment participants to non-participants independent of condition, F(1,373) = 5.1, p =.001, ηp² = .052;
experiment participants were significantly better than non-participants, t(370) = 4.06, p = .000, d = .52. To test
the possibility that the results were confounded by general aptitude, we computed the correlation of the overall
exam score and the behaviourism questions. The correlation was not significant, r=.125, n=378, p =.129. We
also tested for specific aptitude in the topic behaviourism. The correlation between standard exam questions on
behaviourism and extra questions was significant for non-participants, r=.169, n=173, p =.026. As such, we can
conjecture that arguing on Facebook can increase domain specific learning in university courses. No significant
differences between the experimental groups could be found in the delayed post-test.
4. Discussion
Our hypotheses predicted learning processes and attitude change correctly. The argumentation script helped
learners to create arguments of higher quality and structure in accordance with previous results (Stegmann et al,
2012) and group awareness support fostered argument elaboration, supporting the idea that learners considered
possible critiques. The conjecture that multiple perspectives would also be addressed as part of the argument
elaboration was not supported, consistent with (Baker, 2003), but contrary to (Weinberger et al., 2010). There
seems to be a synergy of group awareness support and argumentation script, resulting in argument quality being
improved beyond sheer addition of both treatments, but again this is not true for multiple perspectives. Hence,
support on how to formally construct arguments may be particularly effective when combined with group
awareness support that arguments will be read and criticised or amended. Indeed, many scripts imply that learners
become aware of mutual responsibilities, roles, and commitments, which may explain some part of script effects
(Weinberger, 2011).
We observed attitude change, as hypothesised, but interestingly although all learners perceived themselves as
more confident and multiple-perspective discussants after than prior to the experience, group awareness support
assumed a less positive disposition towards SNS, contrary to learners of the combination condition who became
more positive, and who also perceived themselves as more open-minded after than before the learning session,
pointing to a positive synergy between group awareness support and argumentation scripts.
Regarding learning gains, we found the expected effects that learners with the argumentation script acquired
more knowledge about argument quality, and reported higher subjective learning gains on formal argumentation
(argument quality and argument structure), as well as on argument elaboration. We also found tendencies of a
beneficial interaction effect of script and group awareness support on domain learning gains as well as a
detrimental main effect of group awareness support. Further studies need to identify whether substantial benefits
on domain learning gains can be obtained by combining these treatments and potentially investigate prolonged
application of combined group awareness support and argumentation script. The unexpected detrimental result of
group awareness support on domain learning gains could be explained if the raised group awareness made
participants over-cautious (Wodzicki, et al, 2012) when formulating arguments, which could hinder learning.
Wodzicki and colleagues (2012) report that the participation of the lecturer in discussion forums and the
awareness that the interactions are graded can intimidate the participants and hinder the interactions. We may
have observed a similar effect in this study. Although this has to be tested in a separate study, there are some
more indications supportive of this interpretation in our current results. If cautious behaviour was induced by
group awareness support it would result in more careful preparation of arguments. Indeed, learners with group
awareness support posted fewer more elaborated arguments than participants without group awareness.
Participants with raised group awareness and argumentation script posted more arguments, suggesting that the
caution effect remains, but is reduced through the argumentation script. It seems like the argumentation script
provided through the Facebook App helped learners to develop more and better quality arguments that were,
however, not well elaborated. This is contrary to previous results which have shown higher elaboration when an
argumentation script is provided (Weinberger et al., 2010). Our results also implicate, contrary to our hypothesis,
that learners with group awareness support did not analyse multiple perspectives more than other conditions. This
can be the reason for the lower learning effect in the group awareness condition. Elaborating not only on few well
prepared arguments but on a larger breadth of perspectives presupposes dealing with and analysing larger parts of
the domain which can bring about a larger learning effect. It is also possible that the results on subjective learning
gains on argument elaboration (main effect for argumentation script but not for group awareness support as we
had expected) captured this breadthwise elaboration.
There are, in general, positive results for the use of arguing in SNS to foster learning. First, the results from the
far transfer (retention) support the use of SNS for learning domain specific subjects. Second, the results on the
attitude change indicate a general possibility to cultivate a culture of learning through SNS that shares
characteristics of productive dialogic exchange. Such characteristics include sharing information and being open
to alternative arguments, which support results of previous studies, but also grasping the possibility to elaborate
or adjust one’s own arguments, which is a novel finding. Although this is a general effect independent of
condition, our results show that group awareness support without an argumentation script may interfere with
arguing and learning, because it results in a rather general negative attitude to participating in SNS. On the
contrary, group awareness support enhances the attitude towards participating in SNS when an argumentation
script is provided. Argumentation scripts may foster learners’ confidence or simply guide learners on how to
structure and elaborate their arguments.
In sum, the results indicate that group awareness support of argumentative group processes can be
counterproductive for learning in social networks like Facebook. This group awareness is however granted in
SNS interactions, where individual preparation is not cut out from the overall asynchronous collaborative setting
as in our controlled experiment. Group awareness is also granted in all collaborative learning settings, where
individual preparation is included as an upstream phase in the overall learning scenario, preparing for
collaborative phases. To alleviate possible negative effects while harvesting positive effects of prevailing group
awareness on, e.g., argument elaboration, learners may be additionally supported in structuring their arguments
by an argumentation script. Apps, e.g. in Facebook, that implement argument ontologies are a feasible way of
implementing such argumentation scripts.
Another issue to consider is the novel use of Facebook as a learning platform that may cause uncertainty as
a result of not being acquainted with the audience (Mäkitalo, Weinberger, Häkkinen, Järvelä, & Fischer, 2005).
Group awareness support in a classroom may not result in similarly negative effects. Even outside the context of
an existing classroom, given more training with using SNS for learning may lead to the sense of community and
help minimize the negative effects of over-cautiousness.
Beyond developing specific strategies to turn over-cautiousness into argument quality, other promising
characteristics of SNS are worth investigating that may overrule or exploit effects of group awareness support.
For example, the effects of self-organization of groups in SNS, as well as the role of trust in that context that
may be necessary for harvesting learning opportunities are worth investigating. Our results in general and also
the marginally acceptable reliability of the closed items in the knowledge test indicate that these more socially
involved output variables are appropriate for measuring learning in SNS rather than standard declarative
knowledge tests. However, not all SNS exhibit this dynamic to the same extent, some are more successful than
others with a great variance, and it cannot be assumed that trying to utilize successful SNS to achieve specific
learning goals, e.g. related to a learning domain, will not interfere with this dynamic. Moreover, as a side
methodological remark based on the results on communication attitude change, to do justice to the possibility of
using SNS for academic purposes future research should additionally test effects over longer periods of time.
When measuring learning, we are in effect measuring change, be it change in conceptual knowledge or attitude.
We should therefore be wary of comparisons that take into account only one test time, the post-test. In our
results, we saw significant changes even with a small sample over a brief learning episode, even though no
differences between conditions show up when comparing post-test results.
5. Conclusion
This study applies CSCL research to the widely used social media Facebook. Simultaneously, it extends
research on SNS by redesigning Facebook for learning purposes and systematically looking into the influence of
group awareness support and argumentation scripts on learning. Moreover, the study aims to identify potential
synergistic effects of these different types of support. It thus fills in the gap in research that has compared
scripting and group awareness in isolation and has pointed out first insights in possible synergistic effects (Stahl
et al., 2013). The study also compliments research that looks into the interplay between internal and external
scripts, and adds to the knowledge on how situational information, in the form of group awareness, interacts with
scripts in the context of SNS (Fischer et al, 2013). Moreover, this study shed light into the role of argument
elaboration and multiple perspectives for learning and drew connections between attitude change in SNS and
learning, which support and extend previous research. Specifically with regard to attitude change, our results
show how even the most subtle interventions, like supporting group awareness through an oral instruction, may
change attitudes which can influence interactions during the learning process. However, the presented study
provided evidence that such group awareness support can also lead to (over-) cautiousness and the upheaval of
valuable dialogic processes like discussing multiple perspectives. Directions of future work with respect to group
awareness and argumentation scripts have been suggested. More research is needed to understand processes of
learning in SNS and develop a consistent body of knowledge that can help determine if and how informal
learning can be structured and leveraged for purposes of formal education. Long-term studies that will induce a
dynamic by mimicking or magnifying relevant socio-cognitive processes like the one presented in this paper are
indispensable in the study of SNS as learning platforms. Additionally, investigating learning in SNS may be a
pioneering possibility to understand learning in its social context and extent the theory of learning.
6. Acknowledgements
References
Andriessen, J. E. B. (2006). Arguing to learn. In K. Sawyer (Ed.), (Tran.), Handbook of the Learning Sciences (443-459).
Cambridge: Cambridge University Press.
Ardichvili, A., Page, V., & Wentling, T. (2003). Motivation and barriers to participation in virtual knowledge-sharing
communities of practice. Journal of knowledge management, 7(1), 64-77.
Asterhan, C. S. C., Schwarz, B. B. (2007). The effects of monological and dialogical argumentation on concept learning in
evolutionary theory. Journal of Educational Psychology, 99(3), 626–639.
Baker, M. (2003). Computer-mediated argumentative interactions for the co-elaboration of scientific notions. In Arguing to
learn (pp. 47-78). Springer Netherlands.
Baker, M. (2009). Intersubjective and intrasubjective rationalities in pedagogical debates. Transformation of knowledge
through classroom interaction, 145.
Baker, M., & Lund, K. (1997). Promoting reflective interactions in a CSCL environment. Journal of computer assisted
learning, 13(3), 175-193.
Belland, B. R., Glazewski, K. D., & Richardson, J. C. (2008). A scaffolding framework to support the construction of
evidence-based arguments among middle school students. Educational Technology Research and Development, 56(4),
401-422.
Bennett, G. (2010). Cast a wider net for reunion. Currents, 36(7), 28-31.
Benware, C., & Deci, E. L. (1984). Quality of learning with an active versus passive motivational set. American Educational
Research Journal, 21, 755–765.
Bodemer, D., & Dehler, J. (2011). Group awareness in CSCL environments. Computers in Human Behavior, 27(3), 1043–
1045. doi:10.1016/j.chb.2010.07.014
Buder, J., & Bodemer, D. (2008). Supporting controversial CSCL discussions with augmented group awareness tools.
International Journal of Computer-Supported Collaborative Learning, 3(2), 123–139. doi:10.1007/s11412-008-9037-5
Carroll, J. M., Rosson, M. B., Farooq, U., & Xiao, L. (2009). Beyond being aware. Information and Organization, 19(3),
162–185. doi:10.1016/j.infoandorg.2009.04.004
Dabner, N. (2011). “Breaking Ground” in the use of social media: A case study of a university earthquake response to inform
educational design with Facebook. The Internet and Higher Education, 15(1), 69–78.
Eagly, A., & Chaiken, S. (1995). Attitude strength, attitude structure and resistance to change. In R. Petty and J. Kosnik
(Eds.), Attitude Strength. (pp. 413–432). Mahwah, NJ: Erlbaum.
Erber, M. W., Hodges, S. D., & Wilson, T. D. (1995). Attitude strength, attitude stability, and the effects of analyzing
reasons. In R. Petty and J. Kosnik (Eds.), Attitude Strength. (pp. 433–454). Mahwah, NJ: Erlbaum.
Fischer, F., Kollar, I., Stegmann, K., & Wecker, C. (2013). Toward a script theory of guidance in computer-supported
collaborative learning. Educational psychologist, 48(1), 56-66.
Greenhow, C. (2008). Connecting informal and formal learning experiences in the age of participatory media: Commentary
on Bull et al. Contemporary Issues in Technology and Teacher Education, 8, 187–194.
Greenhow, C., & Robelia, B. (2009). Informal learning and identity formation in online social networks. Learning, Media
and Technology, 34(2), 119–140. doi:10.1080/17439880902923580
Gutwin, C., & Greenberg, S. (2002). A descriptive framework of workspace awareness for real-time groupware. Comput.
Supported Coop. Work, 11(3), 411-446.
Gutwin, C., Greenberg, S., & Roseman, M. (1996). Workspace awareness in real-time distributed groupware: Framework,
widgets, and evaluation. In People and Computers XI (pp. 281-298). Springer London.
Kanuka, H., Anderson, T. (1998). Online social interchange, discord, and knowledge construction. Journal of Distance
Education/Revue de l’enseignement à distance, 13(1), 1–19.
Kimmerle, J., Cress, U., & Hesse, F. W. (2007). An interactional perspective on group awareness: Alleviating the
information-exchange dilemma (for everybody?). International Journal of Human-Computer Studies, 65(11), 899–910.
doi:10.1016/j.ijhcs.2007.06.002
Lampe, C., Vitak, J., Gray, R., & Ellison, N. (2012, May). Perceptions of Facebook's value as an information source. In
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 3195-3204). ACM.
Laru, J., Näykki, P., Järvelä, S. (2011). Supporting small-group learning using multiple Web 2.0 tools: A case study in the
higher education context. The Internet and Higher Education, 15(1), 29–38.
Marttunen, M., & Laurinen L. (2001). Learning of argumentation skills in networked and face-to-face environments.
Instructional Science, 29, 127 – 153.
Mäkitalo, K., Weinberger, A., Häkkinen, P., Järvelä, S., & Fischer, F. (2005). Epistemic cooperation scripts in online
learning environments: fostering learning by reducing uncertainty in discourse? Computers in Human Behavior, 21(4),
603–622. doi:10.1016/j.chb.2004.10.033
McLoughin, C., Lee, M.J.W. (2010). Personalised and self- regulated learning in the Web 2.0 era: International exemplar of
innovative pedagogy using social software. Australian Journal of Educational Technology, 26(1), 28–43.
Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M., & Chizari, M. (2012). Argumentation-Based Computer
Supported Collaborative Learning (ABCSCL): A synthesis of 15 years of research. Educational Research Review, 7(2),
79-106. doi: 10.1016/j.edurev.2011.11.006
Phielix, C., Prins, F. J., & Kirschner, P. A. (2010). Awareness of group performance in a CSCL-environment: Effects of peer
feedback and reflection. Computers in Human Behavior, 26, 151–161.
Pempek, T. A., Yermolayeva, Y. A., & Calvert, S. L. (2009). College students' social networking experiences on Facebook.
Journal of Applied Developmental Psychology, 30(3), 227-238.
Ryan, S. D., Magro, M. J., & Sharp, J. H. (2011). Exploring educational and cultural adaptation through social networking
sites. Journal of Information Technology Education, 10, 1-16.
Scheuer, O., Loll, F., Pinkwart, N., McLaren, B.M. (2010). Computer-supported argumentation: A review of the state of the
art. International Journal of Computer-Supported Collaborative Learning, 5(1), 43–102.
Selwyn, N. (2009). Faceworking: exploring students' education‐related use of Facebook. Learning, Media and Technology,
34(2), 157-174.
Stahl, G., Law, N., & Hesse, F. (2013). Reigniting CSCL flash themes. International Journal of Computer-Supported
Collaborative Learning, 8(4), 369–374. doi:10.1007/s11412-013-9185-0
Stahl, G., Law, N., & Hesse, F. (2013). Collaborative learning at CSCL 2013. International Journal of Computer-Supported
Collaborative Learning, 8(3), 267-269.
Stegmann, K., Wecker, C., Weinberger, A., & Fischer, F. (2012). Collaborative argumentation and cognitive elaboration in a
computer-supported collaborative learning environment. Instructional Science.
Tsovaltzi, D., Weinberger, A., Scheuer, O., Dragon, T., & McLaren, B. (2012). Argument diagrams in Facebook: Facilitating
the formation of scientifically sound opinions. In: Ravenscroft, A., Lindstaedt, S., Delgado Kloos, C., Hernandéz-Leo, D.
(eds.) EC-TEL 2012: Seventh European Conference on Technology Enhanced Learning. Saarbrücken, Germany.
van Boxtel, C., van der Linden, J., & Kanselaar, G. (2000). Collaborative learning tasks and the elaboration of conceptual
knowledge. Learning and Instruction, 10, 311–330. doi: 10.1016/S0959-4752(00)00002-5
van Dijk, A., Gijlers, H., & Weinberger, A. (2013). Scripted collaborative drawing in elementary science education.
Instructional Science, 1-20. doi: 10.1007/s11251-013-9286-1
Weinberger, A. (2011). Principles of transactive computer-supported collaboration scripts. Nordic Journal of Digital
Literacy, 6(3), 189–202.
Weinberger, A. & Fischer, F. (2006). A framework to analyse argumentative knowledge construction in computer-supported
collaborative learning. Computers & Education, 46, 71-95.
Weinberger, A., Stegmann, K., Fischer, F. (2010). Learning to argue online: Scripted groups surpass individuals (unscripted
groups do not). Computers in Human Behavior, 26(4), 506–515.
Wodzicki, K., Schwämmlein, E., & Moskaliuk, J. (2012). “Actually, i wanted to learn”: Study-related knowledge exchange
on social networking sites. The Internet and Higher Education, 15(1), 9–14. doi:10.1016/j.iheduc.2011.05.008
