Content uploaded by Armin Weinberger
Author content
All content in this area was uploaded by Armin Weinberger
Content may be subject to copyright.
Measuring knowledge convergence: Achievement similarity and shared
knowledge in computer-supported collaborative learning
Armin Weinberger, Karsten Stegmann, & Frank Fischer
Knowledge Media Research Center, Konrad-Adenauer-Str. 40, 72072 Tübingen, Germany
Abstract
Learning in small groups may result in convergent knowledge outcomes or foster possible
prior differences between learners. Few studies, however, measure convergence or divergence
of knowledge as an outcome of small group learning. This contribution analyzes knowledge
convergence/divergence as an outcome of learning in small groups with the concepts of
achievement similarity and shared knowledge rooted in two different theoretical frameworks.
Achievement similarity means that learners acquired similar amounts of knowledge regardless
if they share similar knowledge. Shared knowledge means that learners acquired similar
knowledge. In order to test application-oriented knowledge, learners had to individually apply
concepts of a psychological theory in analyzing a problem case. Trained coders identified
correct relations between theoretical concepts and case information. These data were used to
compute both, achievement similarity and shared knowledge. A 2×2 factorial design was used
(number of triads = 32; N = 96) to examine the following questions: (1) can theoretical
concepts of achievement similarity and shared knowledge be measured independently and (2)
to what extend are these measures of convergence sensitive enough to indicate effects of
specific instructional interventions? We examined effects of two instructional interventions on
achievement similarity and shared knowledge. Achievement similarity was conceptualized as
difference between the amount of known theoretical concepts of an individual learner and the
average of his or her group. As a measure for achievement similarity, standard deviations of
learners within one group were used to indicate dissimilarity and multiplied by -1 to indicate
similarity. Shared knowledge was measured as number of pairs within one group, which
applied the same theoretical concept in the individual analysis of the problem case. Results
show that the different instructional interventions significantly and independently affect both
convergence measures. Validity and interpretability will be discussed against the background
of theoretical approaches to learning in small groups.
Summary
Learners in small groups may or may not achieve knowledge convergence (Fischer & Mandl,
2001). Even though knowledge convergence/divergence has been a major concern in
approaches to computer-supported collaborative learning (CSCL), it has been widely
neglected in quantitative studies on the subject. This contribution presents two different
conceptualizations and methods towards analyzing knowledge convergence/divergence.
Conceptualizing, measuring and facilitating knowledge convergence/divergence can be based
on different theoretical approaches towards learning in small groups that have been termed
cooperative and collaborative learning (Dillenbourg, Baker, Blaye, & O'Malley, 1995).
A major aspect of the cooperative learning approach is to distribute responsibilities for sub-
tasks to individual members of a learning group (e.g., Slavin, 1994). The different sub-tasks
will lead to different perspectives of learners, but typically, the grade of the individual group
member depends on the overall success of the group. Therefore, the focus of the cooperative
learning approach is to facilitate achievement similarity of learners, i.e. two or more learners
acquire similar amounts of knowledge, but do not necessarily know the same concepts.
hal-00197406, version 1 - 14 Dec 2007
Author manuscript, published in "11th Biennial Conference for Research on Learning and Instruction (EARLI 2005), Nicosia :
Cyprus (2005)"
Achievement similarity is an often discussed, but rarely measured concept in cooperative
learning approaches. Organizational psychology reviews, however, provide measures for
achievement similarity (Cooke, Salas, Cannon-Bowers, & Stout, 2000). According to Cooke
and colleagues, achievement similarity can be conceptualized as difference between the
amount of known theoretical concepts of an individual learner and the average of his or her
group. The collaborative learning approach supposes that learners negotiate and share their
knowledge resources with the goal to acquire shared knowledge (e.g., Roschelle, 1996).
Shared knowledge means that two or more learners possess identical knowledge concepts.
Against this background, CSCL research often faces the question how shared knowledge of
distributed learners can be facilitated. So far, however, measures for shared knowledge have
not been systematically investigated and applied.
In order to facilitate achievement similarity and shared knowledge we varied process-oriented
instructional interventions, namely computer-supported collaboration scripts. These scripts
pre-structure a specific set of activities in CSCL. An epistemic script pre-structures the
contents of learner’s individual contributions and aims to foster achievement similarity. A
social script pre-structures how learners negotiate and share their knowledge resources and
aims to foster shared knowledge.
We examine the following questions regarding this study: (1) can theoretical concepts of
achievement similarity and shared knowledge be independently measured and (2) to what
extend are these measures of convergence sensitive enough to indicate effects of specific
instructional interventions?
Methodology
A 2×2 factorial design with the factors epistemic script (with vs. without) and social script
(with vs. without) was used (number of triads = 32; number of participants = 96).
Data source. In order to test application-oriented knowledge, learners had to individually
analyze a problem case. This individual post-test was used as a data source for the knowledge
convergence measures. The written case analyses of the learners were segmented (87% rater
agreement) and analyzed by coders with sufficient inter-rater agreement (κ = .90). The data
was segmented and coded with respect to the specific concepts that learners knew to apply in
comparison to an expert solution.
(1) Achievement similarity. We counted the concepts that learners of one group knew to apply
in the post-test and the ones they did not know. Standard deviations of learners within one
group were aggregated (indicating achievement dissimilarity) and multiplied by -1 to indicate
achievement similarity. (2) Shared knowledge. We identified the single concepts that learners
knew to apply in the post-test. On grounds of single comparisons within the groups of three
the shared knowledge concepts were aggregated. When all three learners of one group knew
to apply the same concept, a shared knowledge value of 3 was credited to the learning group.
If only two learners knew to apply this concept, a shared knowledge value of 1 was credited.
In any other case, a shared knowledge value of zero was assigned.
Results
Achievement similarity. The scripts significantly influenced the achievement similarity of
learners. Learners supported with the epistemic script acquired a more similar amount of
knowledge concepts, whereas learners with the social script were more dissimilar than
learners without any script.
hal-00197406, version 1 - 14 Dec 2007
Shared knowledge. With regard to shared knowledge no significant effect of the social script
and no substantial interaction effect can be observed, but a strong negative effect of the
epistemic script. Groups who were supported with the epistemic script showed substantially
less shared knowledge concepts in the individual post-tests.
Conclusions
Results show, that the different scripts significantly and independently affect both
convergence measures in the intended way. Both convergence measures, achievement
similarity and shared knowledge, were sensitive to the effects of the script. The study has also
shown that the assumed theoretical differentiation between achievement similarity and shared
knowledge could be empirically supported. There are, however, also shortcomings of the
measures. Knowledge convergence/divergence measures can be misleading without
considering effects of systematic variation on individual knowledge acquisition. For instance,
exceptionally high or low achievement of learners may automatically lead to higher
convergence, simply because the likelihood of similarity will be higher. Therefore, we will
discuss the development of convergence measures, which regard likelihood and/or systematic
variation on individual knowledge acquisition.
References
Cooke, N. J., Salas, E., Cannon-Bowers, J. A., & Stout, R. (2000). Measuring team
knowledge. Human Factors, 42, 151-173.
Dillenbourg, P., Baker, M., Blaye, A., & O'Malley, C. (1995). The evolution of research on
collaborative learning. In P. Reimann & H. Spada (Eds.), Learning in humans and
machines: Towards an interdisciplinary learning science (pp. 189-211). Oxford:
Elsevier.
Fischer, F., & Mandl, H. (2001). Facilitating the construction of shared knowledge with
graphical representation tools in face-to-face and computer-mediated scenarios. In P.
Dillenbourg & A. Eurelings & K. Hakkarainen (Eds.), European perspectives on
computer-supported collaborative learning (pp. 230-236). Maastricht, NL: University of
Maastricht.
Roschelle, J. (1996). Learning by collaborating: Convergent conceptual change. In T.
Koschmann (Ed.), CSCL: Theory and practice of an emerging paradigm (pp. 209-248).
Mahwah, NJ: Erlbaum.
Slavin, R. E. (1994). Student Teams-Achievement Divisions. In Sharan, S. (Ed.), Handbook
of cooperative learning methods (pp. 3-19). Westport, CN: Greenwood Press.
hal-00197406, version 1 - 14 Dec 2007
