No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
An inventory of peer assessment diversity
Abstract
Since Topping published his literature review on peer assessment in 199846.
Topping , K. 1998 . Peer assessment between students in colleges and universities . Review of Educational Research , 68 : 249 – 76 . [CrossRef], [Web of Science ®]View all references, the number of studies on this subject has doubled, if not tripled. However, along with this expansion, the diversity of peer assessment applications increased equally fast. Based on recent literature, this contribution focuses specifically on the diversity that has come to exist in order to update Topping's typology, adding eight variables, extending eight others with extra sub‐dimensions, and including the implementation factors. Five original variables were absorbed into larger entities, leaving 20 variables which were finally grouped into five clusters. For teachers or advisors, this inventory may be of interest as a source of inspiration or as a checklist of important decisions to make when working with peer evaluation. For researchers, it may be a guide towards covering the full scope of particularities when introducing their peer assessment design. Moreover, the framework developed in this paper might relieve the possible confusion originating from the use of a single term to cover a multitude of sometimes incompatible practices.
... Scholars have identified gaps between theory and practice, and superficial implementation of CTW (Lawlor et al., 2018). Moreover, existing models predominantly focus on peer assessment in a cognitive context and therefore its direct and nuanced applicability to CTW is limited (Adachi et al., 2018;Gielen et al., 2011;Topping, 1998). To this end, we propose a framework specifically focussing on CTW and orienting it to specific peer assessment challenges and resolutions. ...
... Analysis of the literature reveals a dearth of focused frameworks specifically addressing peer assessment challenges in CTW context. For example, Gielen et al.'s (2011) typology explores the diversity of peer assessment in a broader context by extending Topping's (1998) typology classifying 20 variables into five clusters (peer assessment decisions, link between assessment and learning environments, peer interaction, composition, and management of procedure) with a single reference to peer assessment of behaviour. Adachi et al. (2018) framework extends this, incorporating 19 contextual elements covering broader peer assessment context, with peer assessment of process cited once. ...
... Peer assessment training has been found to increase perceptions of psychological safety which leads to increased confidence and trust in peer assessors (Cheng et al., 2015). Considering students' roles as assessee and assessor requires both emotional strength and resilience; training, monitoring and providing guidance in peer assessment is imperative (Gielen et al., 2011;Panadero, 2016). ...
Peer learning is an umbrella term covering diverse strategies supporting students to learn from each other. Studies highlight the power of combining two intertwined models of peer learning, namely peer assessment/feedback and collaborative team-based learning, to prepare graduates for the world of work and encourage acceptable social behaviours. Nevertheless, this approach comes with distinct challenges of marking bias, implementation difficulties, quality, trust and other issues. Studies addressing these challenges in the collaborative teamwork context are sparse and fail to consider the complex and intertwined challenges. Responding to this need, we propose a four-pillar framework comprising veracity, validity, volume and literacy to provide a strong footing on which to base future work in this area. Each of the pillars supports specific but overlapping aspects of peer assessment including assessment design (veracity pillar); implementation considerations (validity pillar); technology factors (volume pillar); and roles and responsibilities (literacy pillar). The framework aims to support educators, policymakers and scholars in mitigating challenges to reimagine and renew peer learning practices to effect positive change.
... Testament to its increased popularity is the accumulation of research studies on peer assessment and most notably more than thirty review studies covering a wide variety of topics. Among the topics investigated by these reviews are as follows: (i) the design and implementation of peer assessment practices (Adachi et al., 2018;Gielen et al., 2011;Luxton-Reilly, 2009;Topping, 1998Topping, , 2003Topping, , 2013Topping, , 2017Topping, , 2021aStrijbos et al., 2009;Van den Berg et al., 2006a, b;Van Gennip et al., 2009); (ii) reliability and validity of peer assessment (Falchikov & Goldfinch, 2000;Li et al., 2016;Speyer et al., 2011;Topping, 1998Topping, , 2003Topping, , 2013Topping, , 2017; (iii) quality criteria for peer assessment practices (Ploegh et al., 2010;Tillema et al., 2011); (iv) impact of social and interpersonal processes (Panadero, 2016;Panadero & Alqassab, 2019;Panadero et al., 2018;Strijbos et al., 2009;Topping, 2017;Van Gennip et al., 2009); (v) peer assessment of collaborative learning (Lejk et al., 1996;Dijkstra et al., 2016;Forsell et al., 2020;Meijer et al., 2020;Strijbos, 2016;Strijbos et al., 2017); and (vi) how the instructional conditions relate to peer assessment outcomes (Ashenafi, 2017;Double et al., 2020;Hoogeveen & Van Gelderen, 2013;Huisman et al., 2019;Li et al., 2020;Panadero et al., 2018;Sanchez et al., 2017;Sluijsmans et al., 1999;Topping, 1998Topping, , 2003Topping, , 2013Topping, , 2021aVan Popta et al., 2017;Van Zundert et al., 2010). Topping (1998) initially defined peer assessment as 'an arrangement in which individuals consider the amount, level, value, worth, quality, or success of the products or outcomes of learning of peers of similar status' (p. ...
... Although the peer assessment research community has taken up Topping's (1998) typology, subsequent refinements have not made the systematic description of peer assessment designs easier. Over time researchers (a) (re)ordered variables in conceptually motivated clusters (Adachi et al., 2018;Gielen et al., 2011;Van den Berg et al., 2006a, b) or removed (Van Gennip et al., 2009) and/or assigned variables to a different cluster compared to a prior refinement (Adachi et al., 2018;Gielen et al., 2011;Van Gennip et al., 2009), (b) revised or changed variable labels and/or descriptions (Adachi et al., 2018;Gielen et al., 2011;Van Gennip et al., 2009), (c) added more variables (Adachi et al., 2018;Gielen et al., 2011), and/or (d) subsumed and revised variables from Topping's (1998) initial typology under a new variable label (Adachi et al., 2018;Gielen et al., 2011) (Online Resource 1 provides an overview of these four refinements to Topping's typology; https:// osf. io/ z2vju/? ...
... Although the peer assessment research community has taken up Topping's (1998) typology, subsequent refinements have not made the systematic description of peer assessment designs easier. Over time researchers (a) (re)ordered variables in conceptually motivated clusters (Adachi et al., 2018;Gielen et al., 2011;Van den Berg et al., 2006a, b) or removed (Van Gennip et al., 2009) and/or assigned variables to a different cluster compared to a prior refinement (Adachi et al., 2018;Gielen et al., 2011;Van Gennip et al., 2009), (b) revised or changed variable labels and/or descriptions (Adachi et al., 2018;Gielen et al., 2011;Van Gennip et al., 2009), (c) added more variables (Adachi et al., 2018;Gielen et al., 2011), and/or (d) subsumed and revised variables from Topping's (1998) initial typology under a new variable label (Adachi et al., 2018;Gielen et al., 2011) (Online Resource 1 provides an overview of these four refinements to Topping's typology; https:// osf. io/ z2vju/? ...
The growing number of peer assessment studies in the last decades created diverse design options for researchers and teachers to implement peer assessment. However, it is still unknown if there are more commonly used peer assessment formats and design elements that could be considered when designing peer assessment activities in educational contexts. This systematic review aims to determine the diversity of peer assessment designs and practices in research studies. A literature search was performed in the electronic databases PsycINFO, PsycARTICLES, Web of Science Core Collection, Medline, ERIC, Academic Search Premier, and EconLit. using data from 449 research studies (derived from 424 peer-reviewed articles), design differences were investigated for subject domains, assessment purposes, objects, outcomes, and moderators/mediators. Arts and humanities was the most frequent subject domain in the reviewed studies, and two-third of the studies had a formative purpose of assessment. The most used object of assessment was written assessment, and beliefs and perceptions were the most investigated outcomes. Gender topped the list of the investigated moderators/mediators of peer assessment. Latent class analysis of 27 peer assessment design elements revealed a five-class solution reflecting latent patterns that best describe the variability in peer assessment designs (i.e. prototypical peer assessment designs). Only ten design elements significantly contributed to these patterns with an associated effect size R2 ranging from .204 to .880, indicating that peer assessment designs in research studies are not as diverse as they theoretically can be.
... According to Blooms taxonomy, lower cognitive levels include knowledge, comprehension, and application level, while higher-order thinking includes analysis, synthesis, and evaluation (Ramirez 2017) Students exposed to peer assessments also show structure and organisation regarding their written work and assessments (Vickerman 2009). Peer assessments allow students to participate actively in the learning process and feedback (Gielen et al. 2010). However, van den Berg et al. (2007) further argued that the effectiveness of peer assessments also depended on the combination of the design characteristics implemented. ...
... As a result, several frameworks and guidelines were formulated for implementing peer review assessments in higher education (K. J. Topping 1996; van den Berg et al. 2007;Gielen et al. 2010;Adachi et al. 2017;Liu and Carless 2007). Peer assessments need clear criteria to ensure that student feedback is accurate and valuable (Sluijsmans and Prins 2006). ...
... (Table 2, Q1). However, Gielen et al. (2010) indicated that performance improvement was not always related to the quality of the feedback but rather the attitude of the assess towards the peer feedback. Additionally, the efficacy of rubrics in improving student performance is not clear-cut (Francis 2018). ...
Moving towards online learning during the coronavirus pandemic presented challenges, such as identifying assessments for learning. Assessments for learning involve using assessments as part of the learning process. Alternative assessments, as opposed to traditional assessments, are favoured for promoting for learning. These assessments include peer assessments and using criteria-referenced tools such as a rubric. Online learning environments often favour automated grading tools such as multiple choice. However, essay-type probing questions help students adopt a deep learning approach. Peer assessments and rubrics can help with grading essay-type questions. However, while the benefits of rubrics and peer assessments are well documented, there is limited research on students’ perceptions in South Africa on the use of rubrics and peer assessments in online environments to facilitate a deep approach to learning. A mixed method approach using a Likert scale and an online qualitative questionnaire was undertaken to explore students’ perceptions of the use of peer assessments with a rubric in an undergraduate module at the University of Johannesburg. Despite a low response rate, four main themes emerged: a clear performance criterion, structured writing, and a deep approach to learning and critical thinking. However, the study also showed limitations of the peer rubric and peer assessments in helping students prepare for formal summative assessment. The results suggest that the rubric and peer assessments, with amendments, could help students adopt a deep approach in online learning environments.
... La EeP se fundamenta en el principio de igualdad de estatus, interpretado como mismo perfil y funciones (Adachi et al., 2018;Ashenafi, 2017;Escudero, 2018). De tal manera que igualdad de status puede ser interpretado como mismos años de escolaridad (para la evaluación que se realiza entre estudiantes) o mismo perfil y funciones (formación y tipo de cargo para el caso de docentes; Adachi et al., 2018;Gielen et al., 2011;Topping, 2009). En ese sentido se espera que la EeP funcione de manera adecuada si los evaluadores pueden valorar a sus pares docentes a partir de su propia experiencia, función y conocimiento del contexto, de modo que estén en posibilidad de dar una retroalimentación que resulte pertinente (Marquina, 2008). ...
... Por un lado, brinda una oportunidad para orientar y fortalecer el desarrollo profesional, porque los evaluadores pueden emitir juicios de valor pertinentes y facilitar una retroalimentación adecuada (Eather et al., 2019;Marquina, 2008). Por otro lado, la participación de docentes en la evaluación de sus colegas puede constituir un mecanismo que dote de confianza, credibilidad y legitimidad a los procesos evaluativos (Ahmad, 2020;Gielen et al., 2011;Ortega, 2015) y contribuye a identificar necesidades y fortalezas en la práctica docente (Bell & Thomson, 2018;Contreras, 2018;Whipp & Pengelley, 2017). ...
... Este conjunto de recomendaciones involucra a múltiples actores y distintos procesos que requieren de altos niveles de coordinación. La complejidad de la EeP se hace aún mayor cuando se implementa como política pública a gran escala, situación que no es analizada por Topping (Gielen et al., 2011). Sin embargo, los aspectos que éste plantea son útiles en la identificación de puntos nodales en la implementación de evaluación de pares como política pública los cuales fundamentan la revisión realizada en el caso mexicano. ...
Este artículo analiza las diferencias entre el diseño y la instrumentación de la política de evaluación entre pares (EeP) en México (2014-2017), creada con la Reforma Educativa de 2013. Se contrasta el proceso de diseño de la EeP, previsto en los lineamientos de la política, con su implementación en la práctica de evaluación docente. Esta comparación se realiza a partir de datos cualitativos obtenidos mediante análisis de gabinete, entrevistas y grupos de enfoque con actores clave que intervinieron en la política. Los hallazgos revelan la desvinculación entre el diseño e implementación de una política educativa de alcance nacional en donde destacan: poca claridad en las atribuciones de instancias involucradas, problemas de coordinación entre instituciones y niveles de gobierno, así como ausencia de algunos actores clave en la instrumentación.
... To highlight factors that should be considered when developing a peer assessment activity, Topping (1998) developed a typology of peer assessment in a post-secondary educational context that emphasizes 17 different sources of variation contributing to the success in a peer assessment activity. In later review articles, Topping (2001Topping ( , 2005 emphasized 12 organizational aspects of learning that instructors should consider in planning a peer assessment activity. 1 Building on this theoretical framework, Gielen, Dochy, and Onghena (2011) proposed organizing these different variables into five clusters. In the sections below, we focus on 15 of the variables most pertinent to in-class peer review of writing organized around three key categories (i.e., making decisions about peer review, linking peer review to other elements of learning, and managing the assessment process). ...
... Objectives for peer review can vary in substantial, yet subtle, ways. Gielen, Dochy, and Onghena (2011) note three main objectives for conducting activities such as peer review for learning: social control and accountability, preparation for self-regulation in lifelong learning, and active participation of all students. The goal of social control emphasizes making students accountable for making an effort to complete their work in a timely and thorough manner. ...
... For example, letting students know that their grades for peer review activity is not based on what their classmates comment on; rather, their participation and the work they put in towards their final product. Additional factors that deserve consideration include how feedback is directed, the level of privacy and anonymity in the feedback process, the form of the contact (i.e., inperson v. online), and the role of the reviewee (Gielen et al., 2011). ...
In ethics courses, students need to begin or continue their transformation from smart, motivated people to ethical, self-reflective professionals. This chapter focuses on some principles and strategies to achieve that goal, including backward design, developing an effective course atmosphere, the nature of teaching and learning ethics, and skill development. Rather than thinking first about classroom activities, reading assignments, and topics, instructors who use backward design start with consideration of the goals they wish their students to achieve, then move on to how they will assess whether students meet those goals. Only then can they design specific, effective activities and assignments for a course. We explore several other principles and assumptions relevant to the teaching of ethics. We do this by presenting an instructor’s observations and reflections on his graduate ethics course, followed by three graduate students’ views of (a) the course and (b) how the instructor attempted to actualize these principles.
... Вршњачка рецензија уведена је као допуна наставничкој рецензији у последње две школске године пратећи препоруке за примену вршњачке рецензије из стручне литературе (Falchinkov/Goldfinch, 2000: 317;Gielen, et al., 2011: 141-150, Huisman, 2018. Процедура примене задатка вршњачке рецензије у обе школске године у највећој мери била је иста (табела 1). ...
... Као главни недостатак прве примене овог задатка током претходне школске године студенти су у спроведеним интервјуима истакли недостатак припреме и обуке. Пратећи препоруке које су изнели бројни аутори (Falchikov, 2007;Seviour, 2015;Gielen, et al, 2011;Hansen/Liu, 2005;Min, 2005, итд.), студенти су у другој итерацији укључени у процес формулисања критеријума током једног двочаса наставе, након чега су имали прилику да примене дате критеријуме на одломцима додељених есеја. ...
... Keywords Architectures for educational technology system · Peer assessment · Peer review · Online systems · Systematic review · Framework CIT-supported peer-assessment systems. Several attempts have been made to systematize CIT-supported peer assessment by reviewing research literature (Alqassab et al., 2023;Fu et al., 2019;Topping, 2023) or existing applications (Luxton-Reilly, 2009), developing inventories of peer assessment diversity (Gielen et al., 2011), and proposing classifications of peer assessment emphases (Søndergaard & Mulder, 2012) (see the Literature Review section for further details). However, the need for a comprehensive, systematic survey of educational CIT-enabled peer-assessment systems that explores and generalizes affordances and constraints of these systems based on a structured research framework still has not been addressed (Alqassab et al., 2023). ...
... Luxton-Reilly (2009) also called for more usability studies and further evaluation studies of differences among the OPRA systems. Gielen et al. (2011) updated Topping's (1998 typology of peer assessment by reviewing studies on educational peer assessment published between 1997 and 2006. Specifically, they refined Topping's variables, identified new variables, dimensions and values, and extended variable clustering proposed by Berg et al. (2006). ...
Over the past two decades, there has been an explosion of innovation in software tools that encapsulate and expand the capabilities of the widely used student peer assessment. While the affordances and pedagogical impacts of traditional in-person, “paper-and-pencil” peer assessment have been studied extensively and are relatively well understood, computerized (online) peer assessment introduced not only shifts in scalability and efficiency, but also entirely new capabilities and forms of social learning interactions, instructor leverage, and distributed cognition, that still need to be researched and systematized. Despite the ample research on traditional peer assessment and evidence of its efficacy, common vocabulary and shared understanding of online peer-assessment system design, including the variety of methods, techniques, and implementations, is still missing. We present key findings of a comprehensive survey based on a systematic research framework for examining and generalizing affordances and constraints of online peer-assessment systems. This framework (a) provides a foundation of a design-science metatheory of online peer assessment, (b) helps structure the discussion of user needs and design options, and (c) informs educators and system design practitioners. We identified two major themes in existing and potential research—orientation towards scaffolded learning vs. exploratory learning and system maturity. We also outlined an agenda for future studies.
... For instance, experimental results depict that peer grades usually correlate with faculty-assigned grades (Liu et al., 2004), but the former may be slightly higher (Kulkarni et al., 2013). Furthermore, students may give lower grades than the faculty to the best-performing students (Sadler and Good, 2006), and peer assessment completed by undergraduate students may not be as reliable (Gielen et al., 2011). These effects usually stem from students having less grading experience than faculty. ...
... The association of agreeableness was still present when we compared the average difference between a post's weighted peer assessment grade and the one a student provides. As we mention, the accuracy of peer grades is often questioned (AlFallay, 2004;Gielen et al., 2011;Kulkarni et al., 2013;Yan et al., 2022) and some students believe it to be unfair because of these grade discrepancies (Kaufman and Schunn, 2011). Our results confirm previous work regarding the peer grades being slightly higher than those given by the professors (Kulkarni et al., 2013). ...
Recent research has leveraged peer assessment as a grading system tool where learners are involved in learning and evaluation. However, there is limited knowledge regarding individual differences, such as personality, in peer assessment tasks. We analyze how personality factors affect the peer assessment dynamics of a semester-long remote learning course. Specifically, we investigate how psychological constructs shape how people perceive user-generated content, interact with it, and assess their peers. Our results show that personality traits can predict how effective the peer assessment process will be and the scores and feedback that students provide to their peers. In conclusion, we contribute design guidelines based on personality constructs as valuable factors to include in the design pipeline of peer assessment systems.
... L'évaluation par les pairs permet également de décharger l'enseignant car les élèves reçoivent des rétroactions sans que l'enseignant doive lui-même évaluer les productions et formuler des commentaires (Double et al., 2020 ;Huisman et al., 2019). De plus, l'évaluation par les pairs étant interpersonnelle (Van Gennip et al., 2009), elle impacte également les compétences de collaboration et de communication des élèves (Gielen et al., 2011). ...
... Pour finir, un temps très conséquent a été utilisé dans la condition grille imposée pour gérer les interactions sociales au sein du sous-groupe et organiser son fonctionnement, ce qui pourrait avoir un effet positif sur les compétences communicationnelles des élèves (Gielen et al., 2011). Ces constatations appuient l'importance mise en évidence par Van Gennip et ses collègues (2015) de prendre en compte les variables interpersonnelles dans l'étude de l'évaluation par les pairs. ...
L’évaluation de l’oral est souvent mentionnée par les enseignants comme étant un obstacle à son enseignement. Les grilles critériées et l’évaluation par les pairs sont deux leviers identifiés dans les écrits scientifiques pour faciliter ce travail d’évaluation. La combinaison de ces deux méthodes est censée apporter certains avantages, mais ces derniers dépendent des caractéristiques de la situation d’évaluation. Cette étude vise, d’une part, à tester les effets de recours à une grille critériée dans un contexte d’évaluation par les pairs sur la compétence des élèves à communiquer oralement, et, d’autre part, à mieux comprendre les processus sous-jacents à son utilisation. Pour ce faire, une méthode mixte en milieu naturel a été suivie. L’échantillon est constitué de 47 élèves âgés de 10 à 12 ans partagés en trois conditions : une condition où les élèves s’appuient sur une grille critériée pour donner des rétroactions avec des consignes strictes et imposées, une condition où les élèves s’appuient sur une grille critériée avec une totale liberté d’action et une condition d’évaluation par les pairs sans grille. Les résultats quantitatifs indiquent une amélioration des composantes orales quelle que soit la condition. Les résultats qualitatifs révèlent des rétroactions plus précises lorsque les élèves utilisent la grille librement. À l’inverse, l’utilisation imposée de la grille implique un plus grand nombre de rétroactions, une posture plus active des élèves évalués et des interactions de groupe plus nombreuses. Par ailleurs, les résultats mettent en évidence la nécessité de former les élèves à donner des rétroactions de qualité.
... However, whether this is achieved depends on how and why peer feedback is implemented. To this end, Gielen et al. (2011) amassed an overview of the diversity of peer assessment and, building on the work of Topping (1998), formulated a list of variables that characterize this diversity. Moving from pedagogical and didactical theory to practice can be a challenging step, and models based on practical examples can help bridge this gap (e.g., May et al., 2016). ...
... In some cases, the feedback is anonymous, and in some cases, it is not. Thus, these examples encompass a variety of the diversity of peer feedback (Gielen et al., 2011), which allowed for a rich discussion of the effects of different choices in setting up and carrying out peer feedback. In some cases, the peer feedback process is voluntary (Urban development, Microbiology), and in others, it is a mandatory part of the course work (Didactics, Physics). ...
In this paper we conceptualize formative peer feedback principles by analyzing and comparing six empirical examples of formative peer feedback in a set of international STEM (science, technology, engineering, and mathematics) contexts. By discussing how these different approaches to peer feedback unfold in practice, we aim to facilitate the bridging of the gap between theory and practice in implementing peer feedback in Higher Education. As a group of researchers and teaching practitioners from six Higher Education institutions, we discussed, analyzed, and synthesized practical examples of peer feedback in STEM that cover a breadth of settings, aims, and forms. The analysis and comparison of these examples illustrate how principles in peer feedback emerge in practice. We discuss how the context in peer feedback influences its practical implementation, particularly with regard to its purpose and aims, the development and use of assessment criteria, and embedding and supporting the initiatives into the teaching and learning activities.
... o staff assessment, official weight, directionality, privacy, contact, year, ability, constellation (assessors and assessed), place, time, requirement, and reward. Thereafter, several researchers classified, modified, and extended the typology as a result of the ongoing increase in the diversity of peer assessment practices (see Adachi et al. 2018;S. Gielen et al. 2011;Van den Berg et al. 2006;Van Gennip et al. 2009). ...
... Identifying the characteristics of the peer assessment design is important for teachers and researchers who use or plan to implement it. Teachers should be aware that the dimensions of peer assessment might have differential effects on the assessment activity and researchers can use these characteristics as a guide to develop stronger research designs (S. Gielen et al. 2011). Overall, there are two main strands of research on peer assessment: as an assessment tool and as a learning tool. ...
... Using an analytical rating approach with criteria, students assess each criterion and may identify substandard work. Consequently, analytical rating is generally considered to be a more difficult task than comparing two works (Lesterhuis et al., 2016;Pollitt, 2012b), and requires training (Gielen et al., 2011;Tekian et al., 2017). ...
... For analytical rating, however, additional training (e.g. evaluating a few papers in class) has been recommended (Gielen et al., 2011;Norcini, 2003;Tekian et al., 2017) as it is believed to be beneficial for construct validity. Although research has shown that, even with such training, differences in leniency between student raters and overmarking persist (Aryadoust, 2016;Kakar et al., 2013), the lack of training may have worsened the clumping of marks and the insufficient diagnosticity in the present study. ...
The present study explores two rating methods for peer assessment (analytical rating using criteria and comparative judgement) in light of concurrent validity, reliability and insufficient diagnosticity (i.e. the degree to which substandard work is recognised by the peer raters). During a second-year undergraduate course, students wrote a one-page essay on an air pollutant. A first cohort (N = 260) relied on analytical rating using criteria to assess their peers’ essays. A total of 1297 evaluations were made, and each essay received at least four peer ratings. Results indicate a small correlation between peer and teacher marks, and three essays of substandard quality were not recognised by the group of peer raters. A second cohort (N = 230) used comparative judgement. They completed 1289 comparisons, from which a rank order was calculated. Results suggest a large correlation between the university teacher marks and the peer scores and acceptable reliability of the rank order. In addition, the three essays of substandard quality were discerned as such by the group of peer raters. Although replication research is warranted, the results provide the first evidence that, when peer raters overmark and fail to identify substandard work using analytical rating with criteria, university teachers may consider changing the rating method of the peer assessment to comparative judgement.
... van den Berg et al. (2006) grouped Topping's 17 variables into four clusters. Gielen et al. (2011) added new variables, augmented others, and subsumed five into larger entities, resulting in 20 variables grouped into five clusters. Adachi et al. (2018b, p. 454) proposed a typology of 19 'design elements' in six clusters. ...
The higher education shift to remote learning due to mobility restrictions imposed during the COVID-19 pandemic highlighted the need to improve the student learning experience using more active learning models. One model is peer assessment. Despite positively impacting student learning, peer assessment uptake remains low, partly because designing effective peer assessment processes is complex. Frameworks provide good coverage of the necessary design considerations; however, a systematic synthesis of the literature on how to design effective peer assessment processes is needed. We find strong evidence that peer assessment is most effective as formative peer feedback whereby students can apply feedback to support their performance and learning. Assessor training, multiple peer review iterations, assessment flexibility, collaborative assessment and providing resources to engage students and educators in peer assessment processes can also improve student experience and learning outcomes. Conversely, we find mixed evidence for the effectiveness of anonymity, online v offline settings and peer marking. Based on these findings, we provide guidance for educators in designing effective peer assessment processes, which, we hope, will drive greater uptake of peer assessment in higher education and support students to benefit from enhanced learning opportunities.
... L'évaluation par les pairs, quant à elle, implique deux ou plusieurs élèves qui, partageant un même statut, évaluent leurs productions ou démarches respectives ou conjointes (Allal, 1999 ;Topping, 2017). Il existe des pratiques diverses d'évaluation par les pairs, qui varient en fonction de différents facteurs, dont l'orientation de la relation évaluative (unilatérale, réciproque) ou le nombre d'élèves impliqué·es (Gielen et al., 2011 ;Strijbos et al., 2009). Des travaux sur cette modalité d'évaluation insistent sur son impact positif sur la progression des élèves (Double et al., 2020), et l'associent le plus souvent au fait que les élèves semblent accepter et comprendre plus facilement les commentaires (rétroactions) fournis par leurs pairs, comparativement à ceux proposés par l'enseignant·e (Black et al., 2004 ;Hadji, 2015). ...
L’approche de l’Évaluation-soutien d’Apprentissage favorise la motivation et l’engagement des élèves et participe au développement de leur capacité d’autorégulation. En cela, elle constitue un outil pédagogique précieux dans la perspective de l’inclusion scolaire puisqu’elle contribue au désir d’apprendre et a pour but de promouvoir les apprentissages des élèves. Dans cette conception, l’évaluation s’intègre à la planification même de l’enseignement, y compris pour permettre aux élèves de comprendre les objectifs poursuivis ainsi que les critères qui serviront à évaluer leurs productions. C’est sur cette base que les élèves pourront participer à l’évaluation de leur travail et recevoir des rétroactions utiles pour progresser. L’article présente une recherche collaborative, impliquant six enseignantes généralistes au degré primaire, pour concevoir une démarche et des outils d’évaluation des premiers apprentissages de l’anglais à l’école primaire. A partir d’un ensemble de données empiriques récoltées lors d’une formation continue combinée à la mise en place d’expérimentations dans six classes de 7ème année primaireP (élèves de 10 à 112 ans), nos résultats permettent de rendre compte de l’usage effectif des outils d’évaluation, avec leurs bénéfices et limites. Les analyses montrent, entre autres, une variété de pratiques pouvant être plus ou moins favorables au soutien des apprentissages. Des leviers et des obstacles à la participation des élèves dans le processus d’évaluation sont mis en évidence. Enfin, en posant un regard critique sur le dispositif de recherche collaborative, nous interrogeons les conditions nécessaires à une transformation des pratiques d’évaluation, pouvant favoriser un accès plus égalitaire aux apprentissages, dans une visée d’école inclusive.
... Later, van den Berg, Admiraal, and Pilot van den Berg et al. (2006) grouped Topping's variables into four clusters for peer assessment in course designs. Gielen, Dochy, andOnghena in 2011 Gielen et al. (2011) reviewed literature from 1997 to 2006 and revised Topping's typology to create an "Inventory of Peer Assessment Diversity". This inventory was developed based on published peer assessment models, and provides useful information for recognizing the details of a particular peer assessment design. ...
50 days free access to full paper: https://authors.elsevier.com/c/1iZVL1HucdZtcM
Abstract:
Peer assessment is a process in which students rate their peers which has many benefits for both the assessor and the assessed. It actively engages students, increases motivation by giving a sense of ownership of the assessment process, encourages autonomy and critical analysis skills, broadens their understanding of the topic, enhances problem-solving and self-assessment abilities as well as develops soft skills. Peer assessment is also beneficial to the teachers as it reduces the strain of the repetitive grading process thus opening more resources for teaching and the development of course materials, especially in large courses with hundreds of students. Sometimes, peer assessment is the only viable option, as in MOOCs (Massive Open Online Courses). Peer assessment has long been studied at all education levels and is gaining traction in recent years, where MOOCs and even the Covid pandemic, which instigated the development of digital competencies, had a positive influence. However, peer assessment is still a demanding process to carry out, but one that can be assisted by modern Internet technology. This paper has a twofold contribution: we present the peer assessment module of our open-source automated programming assessment system Edgar which has been heavily used and developed for the last six years, while peer assessment has been used for the previous three years. Additionally, we present a methodology used and a two-year case study of peer assessment of open-ended assignments in the undergraduate Databases course where 500+ students per season had to provide an entity-relationship model for a given domain and assess their peers' submissions. We discuss our grading methodology, provide in-depth data analysis, and present the students' opinions of the process acquired through an anonymous questionnaire. We find that the process is both demanding in terms of the design of assignments and assessment questionnaires and rewarding in the assessment phase, where the students’ grades turned out to be of high quality.
... Systematic ways of thinking, designing, and communicating about PCA, are important to promote conversations amongst practitioners and researchers. There are several typologies and frameworks already in existence to support the design of peer assessment activities, which might also work for collaborative assessment (Adachi et al., 2018a;Gielen et al., 2011;Panadero et al., 2023). This highlights the importance of considering the role of technology in PCA, which is highly relevant within the Virtual University. ...
... Consistent with the typologies used in the literature to characterise the diversity of approaches, applications, and contexts of peer assessment (Gielen et al., 2011;Topping, 1998), we examined different categories of variables that may influence peer assessment reliability and validity (i.e., a form of moderator analysis), including (1) variables related to the context; (2) variables related to the task itself; and (3) variables related to rating rubrics. From (1) to (3), variables ranged from measures of a broader context to those related to more granular procedures. ...
Background
For peer assessment, reliability (i.e., consistency in ratings across peers) and validity (i.e., consistency of peer ratings with instructors or experts) are frequently examined in the research literature to address a central concern of instructors and students. Although the average levels are generally promising, both reliability and validity can vary substantially from context to context. Meta‐analyses have identified a few moderators that are related to peer assessment reliability/validity, but they have lacked statistical power to systematically investigate many moderators or disentangle correlated moderators.
Objectives
The current study fills this gap by addressing what variables influence peer assessment reliability/validity using a large‐scale, cross‐context dataset from a shared online peer assessment platform.
Methods
Using multi‐level structural equation models, we examined three categories of variables: (1) variables related to the context of peer assessment; (2) variables related to the peer assessment task itself; and (3) variables related to rating rubrics of peer assessment.
Results and Conclusions
We found that the extent to which assessment documents varied in quality on the given rubric played a central role in mediating the effect from different predictors to peer assessment reliability/validity. Other variables that are significantly associated with reliability and validity included: Education Level, Language, Discipline, Average Ability of Peer Raters, Draft Number, Assignment Number, Class Size, Average Number of Raters, and Length of Rubric Description. The results provide information to guide practitioners on how to improve reliability and validity of peer assessments.
... The benefits obtained from empirical studies attest to the usefulness of peer-assessment in language education (Birjandi & Hadidi Tamjid, 2012;Sadeghi & Abolfazli Khonbi, 2015;Saito, 2008;Topping, 1998;Zhao & Gallant, 2012). Peer-assessment aids students in reflecting on their learning by observing other students' performance (Falchikov, 1986;Gielen, Dochy, & Onghena, 2011;Nulty, 2010;Somervell, 1993;Vickerman, 2009), generates positive attitudes in students ( Haaga, 1993;Murakami, Valvona, & Broudy, 2012;Saito & Fujita, 2004), develops a sense of shared responsibility among students (Saito, 2008), and increases a higher level of cognitive thinking (Cheng & Warren, 2005;Davis, 2009). ...
... Recently, some reviews were conducted to investigate the status of peer assessment. These reviews of peer assessment mainly addressed students' perceptions toward peer assessment (Chang, 2016), peer assessment diversity (Gielen, Dochy, & Onghena, 2011), the effectiveness of peer assessment (Topping, 2017), as well as reliability and validity of peer assessment (Speyer, Pilz, Van Der Kruis, & Brunings, 2011). However, none of the previous reviews systemically analyzed how technology-supported peer assessment activities were designed and implemented. ...
With the advancement of information and communication technologies, technology-supported peer assessment has been increasingly adopted in education recently. This study systematically reviewed 134 technology-supported peer assessment studies published between 2006 and 2017 using a developed analysis framework based on activity theory. The results found that most peer assessment activities were implemented in social science and higher education in the past 12 years. Acting assignments such as performance, oral presentations, or speaking were the least common type of assignments assessed across the studies reviewed. In addition, most studies conducted peer assessment anonymously and assessors and assessees were randomly assigned. However, most studies implemented only one round of peer assessment and did not provide rewards for assessors. Across studies, it was more often the case that students received unstructured feedback from their peers than structured feedback. Noticeably, collaborative peer assessment did not receive enough attention in the past 12 years. Regarding the peer assessment tools, there were more studies that adopted general learning management systems for peer assessment than studies that used dedicated peer assessment tools. However, most tools used within these studies only provide basic functionalities without scaffolding. Furthermore, the results of cross analysis reveal that there are significant relationships between learning domains and anonymity as well as learning domains and assessment durations. Significant relationships also exist between assignment types and learning domains as well as assignment types and assessment durations.
... Peer assessment can be a very advantageous pedagogy. However, it is not necessarily reliable, especially when completed by undergraduate student (Gielen et al., 2011). Logan,Pedagonal : Jurnal Ilmiah Pendidikan 151 Students' Reflection Toward the Application of Peer Assessment Activity | Abdul Rosyid, et.al. ...
This research is an endeavor to portrait an effort to empower learners’ autonomy through peer assessment activity by using Moodle-based workshop. The method employed in this study was descriptive case study. 30 students taken from one class were involved in this study as the participants. The data of the research was gathered through administering questionnaire and conducting focused group discussion. The instruments were designed to find the answer of the following question, i.e., to what extend does peer assessment activity using Moodle-based workshop empower learning autonomy? Peer assessment activity was applied in Text Based Structure Class focusing on the topic about English Clauses. Autonomous learning indicators that become the focus of this research are: a) determining the objectives, b) defining the content and the progress of learning, c) selecting appropriate methods and techniques, d) monitoring the applied techniques methods, and e) evaluating what has been achieved. The findings of this study reveal that peer assessment activity by using Moodle-based workshop can promote learners’ autonomy. It is also discovered that the learners are able to learn from the exchanging inputs activities with their peers and most importantly they can access and review the inputs anywhere and anytime they desire.
... L'évaluation par les pairs, quant à elle, implique deux ou plusieurs élèves qui, partageant un même statut, évaluent leurs productions ou démarches respectives ou conjointes (Allal, 1999 ;Topping, 2017). Il existe des pratiques diverses d'évaluation par les pairs, qui varient en fonction de différents facteurs, dont l'orientation de la relation évaluative (unilatérale, réciproque) ou le nombre d'élèves impliqué·es (Gielen et al., 2011 ;Strijbos et al., 2009). Des travaux sur cette modalité d'évaluation insistent sur son impact positif sur la progression des élèves (Double et al., 2020), et l'associent le plus souvent au fait que les élèves semblent accepter et comprendre plus facilement les commentaires (rétroactions) fournis par leurs pairs, comparativement à ceux proposés par l'enseignant·e (Black et al., 2004 ;Hadji, 2015). ...
L’école vaudoise (Suisse romande) oriente précocement les élèves à la fin des degrés primaire en fonction des résultats de l’évaluation sommative. La différenciation structurelle à laquelle elle conduit n’est pourtant pas compatible avec la visée inclusive désormais défendue par les politiques scolaires. Cette situation conduit à des intentions contradictoires – en même temps inclure et sélectionner – mais aussi à des changements progressifs dans les prescriptions, qui apparaissent comme des opportunités de transformer les pratiques d’enseignement et d’évaluation. C’est dans ce contexte que l’anglais est introduit dans le programme scolaire des dernières années du primaire (7-8P) comme deuxième langue étrangère.
Notre projet de recherche-action-formation se saisit du contexte actuel et se donne comme objectif de construire des démarches et des outils d’évaluation formative et sommative des apprentissages en anglais, correspondant à la fois aux nouvelles prescriptions légales et aux cadres théoriques mobilisés dans la formation à l’enseignement, à la Haute école pédagogique du canton de Vaud. Il implique un groupe de 12 enseignant-e-s des degrés concernés avec des chercheur-e-s spécialisé-e-s soit en didactique des langues étrangères, soit en évaluation des apprentissages. Les démarches et les outils sont conçus, mis en œuvre, analysés et réadaptés, en articulant étroitement les principes de l’approche actionnelle (Piccardo, 2014) et ceux de l’évaluation-soutien d’apprentissage (EsA – Laveault & Allal, 2016 ; William, 2011).
Dans une approche actionnelle, les élèves ont à réaliser des tâches qui simulent des situations de communication authentiques (proches de celles de la vie quotidienne) et porteuses de sens. L’articulation avec le cadre de l’EsA vise à élargir la conception de l’évaluation formative en y intégrant les démarches d’évaluation sommative susceptibles de soutenir les apprentissages. L’EsA ne se réduit donc pas à la passation de tests formatifs, mais englobe toutes les tâches quotidiennes et le recours régulier à l’observation et aux formes variées d’échanges sur les objets de savoir. Les élèves, principaux destinataires de cette évaluation, sont ainsi activement impliqués au travers de démarches d’autoévaluation et d’évaluation par les pairs.
A partir des séquences et des démarches d’évaluation co-construites en équipe, des entretiens avec les enseignant-e-s et les élèves, et des observations filmées dans les classes des participant-e-s à la recherche, nous pourrons étudier et documenter les conditions nécessaires à une transformation des pratiques d’évaluation, favorisant ainsi un accès plus égalitaire aux apprentissages en anglais, dans une visée d’école inclusive, et contribuant aux réflexions sur les enjeux de l’évaluation pour l’enseignement (Mottier Lopez, 2015).
... The types of peer evaluations that have been recorded are quite diverse. Gielen, Dochy, and Onghena (2011) built on Topping's (1998) work of developing a typology to describe the diversity of peer assessments. In their analysis, they categorized peer assessments into five clusters, including the following: decisions concerning the use of peer assessment, link between peer assessment and other elements in the learning environment, interaction between peers, composition of assessment groups, and management of the assessment procedure. ...
This research describes a new method of peer evaluation for faculty who implement team learning in their course. It explored whether students preferred this method to other methods created by the course instructor. After creating a peer evaluation instrument using the Student-Driven Peer Evaluation Method in a team-based learning course and completing midterm and end of course peer evaluations, students responded to an electronic survey. Survey results indicated that most students preferred the Student-Driven Peer Evaluation Method to other methods and cited reasons related to feedback, shared values, choice, accountability, and transparency.
... La première est une évaluation par les pairs. Elle implique deux ou plusieurs étudiants dans l'évaluation de leurs acquisitions, approches, progrès et/ou difficultés respectifs (Gielen, Dochy et Onghena, 2011 ;Wegmuller et Allal, 1997). L'évaluation de textes par des pairs, sous forme de relecture collaborative, invite chaque auteur à examiner le travail des autres, à identifier les aspects positifs et à faire des suggestions d'amélioration, et ce, sur base des mêmes critères que ceux de l'écrit final. ...
Dans le cadre de cette contribution, nous avons choisi de centrer notre attention sur l’accompagnement de l’écriture d’un portfolio à double enjeu, destiné à des jeunes formateurs engagés dans l’enseignement supérieur et amenés à produire un texte réflexif qui se veut un levier de développement professionnel et constitue en même temps un support à l’évaluation certificative d’un programme d’agrégation de l’enseignement supérieur. Les balises pour accompagner les étudiants dans la production d'un portfolio sont présentés dans le texte.
... For peer assessment to be effective, extant literature has shown that training students on using the tool provided is important (Gielen et al., 2011;Lindblom et al., 2006;Sluijsmans et al., 2001Sluijsmans et al., , 2002Topping, 2010;Vickerman, 2009). Hence, our assessment criteria clearly state what is required of students, the process by which they will assess each other's contributions during the period of working together as a group, and ongoing discussion on how to use the rating criteria at the end of the period. ...
This study investigates the impact of peer assessment on students’ engagement in their learning in a group work context. The study used regression analysis and was complemented by qualitative responses from a survey of 165 first-year undergraduates in a UK university. Findings suggest that students’ perception of their contribution to group work fosters engagement and enhances their learning in a group. Also, that students’ perception and the overall experience of rating their peers’ work impact their engagement within a group. The study contributes to the literature by focusing on the assessment of the entire learning journey within a group rather than the final group output. In particular, the study highlights the significant contributions of peer assessment in managing student engagement in modules and/or assessments for large cohorts.
... Peer feedback training can also improve students' attitudes towards peer feedback, which in turn may influence their behaviour during the peer feedback process (Huisman et al., 2020). Topping (1998) synthesised a typology of peer feedback, consisting of 17 variables that can vary in a peer feedback assignment, which was extended to 20 variables by S. Gielen et al. in 2011. Van den Berg et al. (2006a studied the outcomes of varying several of these variables and found that having sufficient time between peer and final teacher assessment, providing reciprocal peer feedback, and feedback groups of three to four students were most beneficial for effective peer feedback. ...
Public higher education in democratic South Africa comprises an intense ever evolving network of policy, structural processes and human factors collectively working towards improving student retention and enhancing student success. This chapter presents a case study of institutional leadership efforts at the different management levels aimed at student retention and success, presently in one public higher education institution in South Africa, namely the University of KwaZulu-Natal (UKZN), which has a diverse student population and five campuses. We unpack the various filaments, commencing with decision-making and structural initiatives from the penultimate governance level of the deputy vice chancellor, filtering down the hierarchy to school leadership. Interestingly, leadership efforts are not solely underpinned by the intention of progression through the system based on normative ideas such as increased resourcing. Conceptually, improved retention and student success at UKZN is complicated by a transformation agenda of widening access, increasing rural participation and the recruitment of students from disadvantaged backgrounds. The current policy ecology seeks to harness the various policies which guide teaching, learning and curriculum development, academic support and assessment. The policies are all encompassing and they dovetail with each other towards a single goal earmarking success, namely that a healthy student will be able to progress in the expected time frame. A critical layer lies in the support system outside of the academic focus which centres on student well-being: personally, and environmentally.
... O'Donnell and Topping (1998) described a typology of relevant variables. Gielen et al. (2011b) offered a more developed inventory. Further developments came from Topping (2018), outlining 45 variables. ...
Digital peer assessment (PA) is an arrangement for learners to consider and specify the level, value, or quality of a product or the performance of other equal-status learners, using computers, tablets, mobiles or other devices, and the internet. Digital PA is of increasing relevance as more educational establishments are moving toward online or blended learning. It has been widely used for some time, not only in elementary (primary) and high (secondary) schools but also in higher education. In this article, the purposes of PA are considered. Then, questions of effectiveness are briefly discussed. Then, the majority of the article describes in general terms how to do it. A review is offered for variations in types of PA and the underpinning theory, both of which have practical implications, irrespective of whether the PA is digital or face-to-face. Then, the use of different kinds of digital hardware in different kinds of PA will be considered. After this, the social and emotional aspects of digital PA are considered. As the contexts are so different, differences between primary school, high school, and higher education are reviewed. A conclusion summarises the strengths and weaknesses of digital PA, which can certainly be effective as a teaching and learning method and enhance student communication, problem-solving, and self-confidence.
... Avainsanat fenomenografia, käsityötiede, materialisointi, transmediaatio, vertaispalaute Johdanto Vertaispalaute on arkipäivää yliopisto-opinnoissa, ja siitä on tehty runsaasti tutkimuksia viime vuosikymmeninä (esim. Gielen, Dochy & Onghena 2011;van Popta, Kral, Camp, Martens & Simons 2017;Topping 1998). Oma tutkimuksemme sai alkunsa, kun ryhdyimme kehittämään käsityötieteen opetusta ja oppimista käänteisen ajattelutavan innoittamina (ks. ...
Tutkimuksen tavoitteena on analysoida ja kehittää käsityönopettajan maisteriopintoihin liittyvää Materialisointi käsityötieteessä -opintojaksoa, jossa opis-kelijat (n=22) tutustuivat uudenlaiseen tapaan syventää tieteellistä ajatteluaan materialisoinnin kautta. Ideoimme kurssille teoreettisiin ajatuskarttoihin ja vertaispalautteeseen liittyvän opetuskerran, jossa yhdistyi käsitteellisen ja materiaalisen artefaktin vuoropuhelu. Videokuvatut vertaispalautteet koostuivat sekä sanallisesta että materiaalisesta osuudesta. Videoaineiston fenomenografisen analyysin avulla muodostettiin käsityöpainotteisen vertaispalautteen neljä erilaista merkitystä: mukautuminen, muuntaminen, kokeileminen ja kartuttaminen. Tutkimuksen johtopäätöksenä todetaan, että materialisoitu vertaispalaute tuki merkittävästi opintojakson tavoitteita ja tarjoaa jatkossa opiskelijoille ja opettajille uudenlaisen pedagogisen mallin yhteisölliseen oppimiseen ja (käsityö)tieteen materialisointiin.
Avainsanat: fenomenografia, käsityötiede, materialisointi, transmediaatio, vertaispalaute
... O'Donnell and Topping (1998) described a typology of relevant variables. Gielen, Dochy, and Onghena (2011) offered a more developed inventory. Further developments came from Topping (2018, 12-13), outlining 43 variables in the context of a comprehensive theory of PA. ...
Peer assessment (PA) is generally effective, and especially important for school teachers, as the experience might lead teachers to use PA more skilfully with school students. Digital PA (using computers) becomes more important as universities switch to online learning. This systematic review of research literature on digital PA for pre-service and in-service teachers encompasses: online/web-based, video, Massive Open Online Courses, digital frameworks to organize/structure PA, e-portfolios, Personal Digital Assistants (PDAs), Facebook, iPads and wikis. It contained 43 papers and all but one reported mainly positive effects. Potential moderator variables were considered, but studies rarely reported many of them. Few studies had control groups, only two reported Effect Sizes, and none reported implementation fidelity or follow-up. There was little evidence for carry-over of PA practices into later teaching of school children. However, the potential moderator variables provide a template for future reviews of PA and the design of PA by teachers.
... These conditions involve many factors that may hamper the accuracy of the assessments as well as their effects on the learning outcomes. Gielen, Dochy, and Onghena (2011) defined 20 variables to describe the different features of peer assessment. Among these, according to the meta-analysis carried out by Falchikov and Goldfinch (2000), which included 48 studies published up to the year 2000, the main set of factors that seem to increase the accuracy of peer assessments are: (a) the provision of an assessment made on the basis of explicit criteria or guidelines, (b) student-owned criteria, (c) the nature of the assessment task (academic better than professional practice) and (d) the study has a high quality experimental design. ...
Maximising the accuracy and learning of self and peer assessment activities in higher education requires instructors to make several design decisions, including whether the assessment process should be individual or collaborative, and, if collaborative, determining the number of members of each peer assessment team. In order to support this decision, a quasi-experiment was carried out in which 82 first-year students used three peer assessment modalities. A total of 1574 assessments were obtained. The accuracy of both the students’ self-assessment and their peer assessment was measured. Results show that students’ self-assessment significantly improved when groups of three were used, provided that those with the 20% poorest performances were excluded from the analysis. This suggests that collaborative peer assessment improves learning. Peer assessment scores were more accurate than self-assessment, regardless of the modality, and the accuracy improved with the number of assessments received. Instructors need to consider the trade-off between students’ improved understanding, which favours peer assessment using groups of three, and a higher number of assessments, which, under time constraints, favours individual peer assessment.
... Another reason may be that peer assessment activities are very diversified. For example, Gielen, Dochy, and Onghena (2011) identified 20 variables to describe the peer assessment process, which they organized into five categories: the decision to use peer assessment, the link between peer assessment and other learning environment elements, the interaction between peers, the composition of assessment groups, and the management of the assessment procedure. Given this breadth of peer assessment practices, in the present meta-analysis, we explored the aspects of peer assessment activities that were likely to influence non-cognitive outcomes. ...
Peer assessment is increasingly being used as a pedagogical tool in classrooms. Participating in peer assessment may enhance student learning in both cognitive and non-cognitive aspects. In this study, we focused on noncognitive aspects by performing a meta-analysis to synthesize the effect of peer assessment on students’ non-cognitive learning outcomes. After a systematic search, we included 43 effect sizes from 19 studies, which mostly involved learning strategies and academic mind-sets as non-cognitive outcomes. Using a random effects model, we found that students who had
participated in peer assessment showed a 0.289 standard deviation unit improvement in non-cognitive outcomes as compared to students who had not participated in peer assessment. Further, we found that the effect of peer assessment on non-cognitive outcomes was significantly larger when both scores and comments were provided to students or when assessors and assessees were matched at random. Our findings can be used as a basis for further investigation into how best to use peer assessment as a learning tool, especially to promote non-cognitive development.
In this study, we aimed to explore the impact of different roles of feedback providers on their experience of providing feedback, as well as their influence on the feedback receivers’ responses to feedback. The students who performed the assignment (creating learning material) assumed the role of peer experts (N=22) in providing
feedback. The other students (N=22) took on the role of peer users as they used the developed learning materials. The data on the students’ experiences were collected using the Intrinsic Motivation Inventory. The response to feedback was explored via interviews with the peer experts. Findings indicate that the peer experts exhibited a higher sense of competence than the peer users. Their feedback was perceived as more clear, specific, and useful than that provided by the peer users. According to the Self-Determination Theory, we highlight the importance of different task design features for students’ sense of competence, relatedness, and autonomy.
This study aims to analyze face-to-face and non face-to-face presentations centered on the cases of college students who participated in speaking classes. Since COVID-19, non face-to-face education has become a form of university education, and learners' presentations have been made in a completely different context depending on whether they are made in face-to-face or non face-to-face classes. Therefore, based on the previous study showing that the context can affect communication methods and perceptions, the presentation and the peer feedback pattern in the face-to-face and non face-to-face situations were examined. To this end, the face-to-face presentations of 60 participants enrolled in speaking courses were collected, to which their non face-to-face presentations were compared. As a result, in terms of presentation, the language expressions, voice elements, and visual elements in their face-to-face presentations and non face-to -face presentations showed different aspects. In addition, peer feedback also showed differences according to class methods, such as evaluating the proportion and the adequacy of the presentation. As a result of this study, we learned that we need a differentiated educational strategy for face-to-face presentations and non face-to-face presentations. We also discussed how there is a need to improve the realism of learning and to use presentations to achieve this goal.
In this paper, we explore how an educational programme can be designed to support managers in learning systematic innovation practices, contributing to the Innovation Management community. Using the community of practice (CoP) concept, we developed a programme to educate managers in systematic innovation practices, including systematic innovation work, innovation climate, and the creation of innovation teams. Our contribution builds on prior studies on the CoP construct. Using a case study with a mixed-methods design, we have explored the educational experiences of ten managers and the practical outcome of their organisations’ participation in the programme. Based on mixed data, we conclude that the educational programme “Management Education on Systematic Innovation Practices” (MEoSIP) supports managers’ understanding of systematic innovation practices in theory and in practice. It also contributes to practitioners who can support clients in improving innovation-related abilities. Agencies can develop training programmes using CoP for other educational purposes. Further research is proposed.
Most conventional information literacy classes apply direct instruction where students focus on memorizing target knowledge. As this approach promotes little active learning, while digital storytelling could effectively involve students, we proposed a digital storytelling approach to enhancing students’ information literacy development in this study. In this research, we applied student-centered digital storytelling to enhance learners’ development of information literacy. As the literature indicates that appropriate scaffolding is essential for effective digital storytelling, and learners need guidance on how to produce digital stories with in-depth content to achieve satisfactory learning outcomes, we integrated peer assessment (PA) into digital storytelling to guide learners to reflect on and critique their videos based on assessment rubrics and peer feedback, and then further improve their videos. We also conducted a quasi-experiment with two groups of elementary students to test the effectiveness of this strategy. The target knowledge was copyright and public license issues. The experimental group learned the content through the PA-enhanced digital storytelling approach, and the control group through the teacher feedback-enhanced approach. The results showed that the PA-enhanced approach to digital storytelling significantly outperformed the teacher feedback-enhanced approach in promoting students’ learning of information literacy, self-efficacy, and critical thinking abilities; on the other hand, no significant differences existed between the two groups in terms of their learning motivation.
The aim of this chapter is to present, from both a theoretical-conceptual and technical-practical perspective, the characteristics that assessment must have within a teaching and learning process for it to be inclusive. To do this, we will move from the most general level of decision-making about assessment, namely the Assessment Approach, where the teacher plans and develops the assessment, to the most concrete level, namely the Assignment that is found within the assessment instrument. The chapter highlights the need to align educational practices with assessment practices within the same teaching and learning process in order to ensure that it remains inclusive, both with regards to the pedagogical function of assessment aimed at improving the formative action of the teacher and to the self-regulation process of the pupils, as well as to the social function of assessment that provides qualification or accreditation of the learning results of the pupils.
This study investigates the implementation of peer feedback in an advanced English-Chinese translation course. Teacher scaffolding was provided to 20 students who completed two translation-feedback-revision cycles. A survey was then administered to the class, followed by a semi-structured interview with six students. The study yielded rather positive results, with the overall accuracy rate of peer feedback comments and the incorporation rate surpassing 75% in both tasks; students were generally positive about the peer feedback processes; doing peer feedback triggered reflective comparisons and internal feedback mechanisms. The findings provide empirical evidence that peer feedback not only has great potential to enhance student performance, but can also help to promote subsequent action upon feedback, facilitate engagement with translation criteria, and foster the ability to make evaluative judgments.
La participación del alumnado universitario en su evaluación se entiende como una oportunidad para mejorar la implicación de los estudiantes en su aprendizaje y favorecer el desarrollo de habilidades esenciales como la responsabilidad, la reflexión sobre el propio aprendizaje y la autonomía. El objetivo de esta investigación es estudiar la posibilidad de permitir al alumno universitario escoger el medio por el que ser evaluado en una determinada unidad. Para ello se diseñó una tarea individual relacionada con el uso de las nuevas tecnologías para la enseñanza de la lengua extranjera y el alumnado de la asignatura de Metodología de la enseñanza del inglés como lengua extranjera del Grado en Lenguas Modernas de la Universidad de Las Palmas de Gran Canaria durante los cursos 2017/18, 2018/19 y 2019/20 pudo elegir cómo ser evaluado en este tema, escogiendo entre tres medios de evaluación diferentes con tres calificaciones diferentes. Contrariamente a lo esperado, los alumnos escogieron la tarea más compleja. Según los resultados de una encuesta final que contestaron los alumnos del curso 2019/20, las principales razones de dicha elección son la novedad y el desafío personal que implica realizar esta tarea y la conveniencia de diseñar una herramienta necesaria para su futura labor docente. Finalmente, se analizan las páginas web educativas diseñadas por los alumnos atendiendo al modelo para el estudio de géneros digitales de Askehave y Nielsen (2005).
Peer assessment has been found to have many advantages in learning particularly when viewing it through a socio-cultural lens. This study explored peer assessment across language teacher education coursework, following 15 teacher candidates through three semesters of TEFL coursework. This longitudinal approach is rare with peer assessment research. Surveys and interviews were collected to understand the students’ perception of peer assessment, including how it contributed to their learning and its summative use. Results indicated that repeatedly practicing peer assessment in coursework led to increased positive perception about its usefulness in learning and enhanced trust in peers as qualified assessors, but with a complex interaction of various factors in students’ perception, such as perceived task difficulty, interpersonal relationships, mode of feedback, etc. The participants’ hesitant attitudes about summative use of peer assessment (PA) was interpreted as largely based on their socio-culturally shaped perspectives of authority and academic achievement. Implications for using peer assessment in language teacher education are offered.
This chapter explores the role of assessment in improving student success. First, literature regarding assessment characteristics, formative assessment and (peer) feedback, assessment and feedback literacy, the use of technology in assessment, and assessment integrity is discussed. Subsequently, three case studies are presented. The first case study investigates the effect of digital peer feedback on video presentations. Students used the digital platform Pitch2Peer to upload their presentation and provide feedback to at least one peer’s presentation. Results indicate that students’ presentation skills improved after peer feedback, but that this was not always related to received feedback comments. The second case study explores students’ peer feedback beliefs, using the Beliefs about Peer Feedback Questionnaire (BPFQ), and shows that students generally have positive beliefs with regard to peer feedback. The third case study exemplifies a curriculum design focused on assessment, which indicates that deliberate assessment design can have positive effects on student engagement and results. The results from the case studies and the literature emphasise that peer feedback in particular is an important tool in designing assessment and feedback to improve student learning.
There is a paucity of literature investigating peer review as an aspect of accounting and auditing education. This study investigates students’ perceptions of peer review as a method to develop professional competencies. It reports on a peer review intervention, the so-called TUTBuddy, introduced in an undergraduate auditing course. A mixed method approach was followed that showed that students perceived the intervention as having positively influenced the development of their competence in nine areas. A positive relationship was found between the students’ perceived development of these competencies and their own academic performance. The study also draws attention to students’ interpersonal perspectives, and suggests Ubuntu dimensions that can be emphasised to promote interconnectedness between an individual student and their peers. The study contributes to the peer review literature by showing its application in the auditing discipline, and by suggesting that an interpersonal frame of reference be considered to strengthen peer review as a social process.
The 2001 U.S. Supreme Court Case of Falvo v. Owasso School System (Owasso Independent School District No I-011 v. Falvo) has focused national attention on the common classroom practice of peer-grading. In a unanimous decision the court reaffirmed the popular view that students grading each others' tests is valuable, saving teachers' time and augmenting student learning. Our study puts these presumed benefits to the test in 4 middle school science classrooms. We compared teacher-assigned grades to those awarded either by students to themselves or by their peers. By training students to grade with the help of a scoring rubric, a very high correlation was obtained between students and their teacher on test questions (r = 0.91 to 0.94). We found patterns of bias when students assigned grades. When grading others, students awarded lower grades to the best performing students than their teacher did. When grading themselves, lower performing students tended to inflate their own low scores. Performance on an unannounced, 2nd administration of the same test 1 week later measured the degree to which student-grading resulted in any increased understanding. Students who graded their peers' tests did not gain significantly more than a control group of students who did not correct any papers but simply took the same test again. Those students who corrected their own tests improved dramatically. Self-grading and peer-grading appear to be reasonable aids to saving teachers' time. Self-grading appears to result in increased student learning; peer-grading does not.
This paper reports an action research project to evaluate an intervention designed to increase students' confidence in an undergraduate peer assessment of posters in Psychology. The intervention set out to maximize the benefits of peer assessment to student learning by explicitly developing and working with marking criteria, and improving the fairness and consistency of students' marking through a trial marking exercise. Evidence from qualitative evaluation questionnaires suggested that students' initial resistance to the peer assessment was transformed by their participation in these processes. After the intervention the range of marks used by students increased at the same time as variability significantly decreased. Quantitative and qualitative data from module appraisal forms completed by students also demonstrated increased transparency and greater confidence in the peer marking process compared with the year before. The study raises issues for student support and staff development in using peer assessment.
Abstract Assessment practices in higher education institutions tend not to equip students well for the processes of effective learning in a learning society. The purposes of assessment should be extended to include the preparation of students for sustainable assessment. Sustainable assessment encompasses,the abilities required to undertake activities that necessarily accompany,learning throughout life in formal and informal settings. Characteristics of effective formative assessment identified by recent research are used to illustrate features of sustainable assessment. Acts of assessment need both to meet the specific and immediate goals of a course as well as establishing a basis for students to undertake their own assessment activities in the future. To draw attention to the importance of this, the idea that assessment always has to do double duty is introduced.
Peer assessment provides a useful mechanism to develop many positive qualities in students studying in higher education (HE). Potential influences on peer‐awarded marks include student qualities such as gender, HE background (e.g. university affiliation) and participation in the development of the assessment criteria. Many studies that have investigated peer assessment have placed great emphasis on marks from a single tutor, or very few tutors, from a single university. This study examined grades awarded by 11 tutors (affiliated with four universities) to oral presentations delivered on a residential field course by second‐year undergraduate students from two universities studying environmental or biological disciplines. Student assessors awarded marks of fairly high precision (correlating strongly with tutor grades) but averaged 5% higher than their tutors (i.e. of only moderate accuracy). Marginally higher marks (circa 1.6%) were awarded by student assessors to speakers studying at the same university. Gender influences were detected as males tended to grade other male speakers very slightly more highly than female speakers. Marks from females were unaffected by speaker gender. Students who participated in the development of the assessment criteria did not achieve higher grades for their presentations. However, when these ‘participants’ were assessing, they awarded lower marks than their peers (i.e. closer to, but not as low as, those awarded by tutors). Lower marks were also awarded during the middle of sessions, possibly resulting from factors associated with motivation and attention of speakers and markers. Overall, the potential biases in marking by naive assessors examined in this study may influence the validity of marks generated by peer assessment schemes, but the experience of this type of assessment had positive effects on those involved.
Although peer reviewing of writing is a way to create more writing opportunities in college and university settings, the validity and reliability of peer-generated grades are a major concern. This study investigated the validity and reliability of peer-generated writing grades of 708 students across 16 different courses from 4 universities in a particular scaffolded reviewing context: Students were given guidance on peer assessment, used carefully constructed rubrics, and were provided clear incentives to take the assessment task seriously. Distinguishing between instructor and student perspectives of reliability and validity, the analyses suggest that the aggregate ratings of at least 4 peers on a piece of writing are both highly reliable and as valid as instructor ratings while (paradoxically) producing very low estimates of reliability and validity from the student perspective. The results suggest that instructor concerns about peer evaluation reliability and validity should not be a barrier to implementing peer evaluations, at least with appropriate scaffolds. Future research needs to investigate how to address student concerns about reliability and validity and to identify scaffolds that may ensure high levels of reliability and validity. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Students need to understand assessment processes in order to succeed in higher education. However, recent research has identified how difficult it is for students to become absorbed into the assessment culture of their disciplines, with a recognition that providing written criteria and grade descriptors is not enough to make this tacit 'knowledge' transparent to novice students. This paper reports on an exercise where sports studies students used assessment criteria to mark their peers work coupled with an assessment of their peer marking and feedback comments. The latter was included to encourage students to engage seriously with the peer assessment. Analysis of the data indicates considerable benefits for the students in terms of use of criteria, awareness of their achievements and ability to understand assessment feedback.
The research on formative assessment and feedback is reinterpreted to show how these processes can help students take control of their own learning, i.e. become self-regulated learners. This refor-mulation is used to identify seven principles of good feedback practice that support self-regulation. A key argument is that students are already assessing their own work and generating their own feedback, and that higher education should build on this ability. The research underpinning each feedback principle is presented, and some examples of easy-to-implement feedback strategies are briefly described. This shift in focus, whereby students are seen as having a proactive rather than a reactive role in generating and using feedback, has profound implications for the way in which teachers organise assessments and support learning.
To report and document well, scientific and engineering professionals systematically collect, evaluate and synthesize the work of their peers with their own. The product of such effort is a technical report or scientific paper that reviews the existing body of knowledge, presenting it in a logical and ordered manner where claims are supported by evidence and referenced by citation. It is also usual in the scientific community that peer review then precedes the publication of such papers. This paper reports on a trial of introducing Computer Science students to the task of technical and scientific writing including the process of peer review. Upon completion of the trial, the students were surveyed anonymously. An overwhelming majority of respondents claimed to have initial reservations about participating. However, upon completion of the exercise, the majority reported that the entire process was intellectually simulating to complete and that it had enhanced their knowledge of subject matter.
While a large amount of interest has been shown in the use of peer and self-assessment, few studies have considered the effect of stress on the students involved. None have considered whether the resultant stress itself might account for any noticeable improvements in student performance. The research presented in this paper addresses this question. An experimental design measured the effects of type of assessment and gender on student stress levels and performance. Results suggest that females are more stressed by self-assessment than males and that being subjected to self- and peer assessment, while more stressful, leads to improved student performance in summative tasks. Yes Yes
Het proefschrift omvat drie theoretische bijdragen en drie empirische studies over peer assessment, een algemene introductie en afsluitende reflecties met een discussie van de resultaten, een discussie van de onderwijskundige implicaties en een discussie van enkele methodologische kwesties.
De eerste bijdrage beschrijft de rol die peer assessment kan vervullen in het versterken van de consequentiële validiteit van een assessment systeem. Ten eerste worden de verschillende soorten effecten op leren verhelderd die assessment in het algemeen kan hebben, en worden de ontwerpprincipes geformuleerd om de consequentiële validiteit van een assessment systeem te verhogen. Vervolgens wordt aangetoond dat peer assessment helpt om tegemoet te komen aan de geïdentificeerde ontwerpprincipes die de consequentiële validiteit van een assessment systeem doen stijgen. Meer specifiek toont dit proefschrift dat peer assessment het beter doenbaar kan maken om uitdagende en authentieke taken op te nemen in een assessment syteem; dat het kan helpen om de assessment eisen duidelijker te maken naar studenten toe; dat het kan zorgen voor een supplement voor of een vervanging van formatief assessment door de docent of leraar; en ten slotte, dat het de respons op de feedback van de docent of leraar kan ondersteunen.
De tweede bijdrage gaat verder dan de impact van peer assessment op de consequentiële validiteit, en behandelt het probleem dat de output van peer assessment tegen een verscheidenheid aan kwaliteitscriteria beoordeeld wordt in de literatuur, waardoor er een verwarrend beeld ontstaat. De verschillende conceptualisaties van kwaliteit die in de literatuur voorkomen worden geanalyseerd. Er wordt aangetoond dat de discussies over de meest geschikte kwaliteitscriteria voor de output van peer assessment teruggebracht kunnen worden naar de onderliggende verschillen in doelen. Het meest voor de hand liggende doel van peer assessment is het gebruik als assessment middel. Het leerdoel van peer assessment is ook breed gekend. Nader onderzoek van de literatuur levert drie bijkomende doelen op: vestigen van sociale controle in de leeromgeving; voorbereiden van studenten op zelfcontrole en zelfregulatie bij levenslang leren; en actieve participatie van studenten of leerlingen in de klas. Elk van deze doelen resulteert in andere kwaliteitscriteria. Er wordt beargumenteerd dat enkel de criteria die congruent zijn met het doel dat men nastreeft overwogen moeten worden wanneer men de kwaliteit van peer assessment evalueert.
De derde bijdrage start van de vaststelling dat, samen met de expansie van het peer assessment onderzoek in het laatste decennium, ook de diversiteit van de peer assessment toepassingen exponentieel gestegen is. Deze diversiteit stelt problemen voor zowel gebruikers als onderzoekers van peer assessment. Er wordt een inventaris voor peer assessment diversiteit ontwikkeld die interessant kan zijn voor gebruikers, als checklist voor belangrijke beslissingen die genomen moeten worden of als overzicht van mogelijke alternatieven voor een specifieke toepassing, en voor onderzoekers, als een leidraad voor welke informatie te verschaffen over de details van hun peer assessment ontwerp.
De vierde bijdrage vergelijkt de impact van peer feedback en leerkrachtfeedback op het leren en behandelt de vraag of peer feedback kan fungeren als een vervanging van expertenfeedback. Een pretest posttest controlegroep design onderzoekt de langetermijneffectenvan individuele peer feedback en collectieve leerkrachtfeedback op schrijfopdrachten in het secundair onderwijs (N=85). Bovendien onderzoekt het de meerwaarde van twee maatregelen om de repons van de feedbackontvanger op peer feedback te ondersteunen: een a priori twijfelformulier en een a posteriori repliekformulier. De studie toont aan dat er geen significant verschil is tussen de leerlingen in de conditie met enkel vervangende peer feedback en de controleconditie met leerkrachtfeedback wat betreft vooruitgang in punten op de opstellen. Beide groepen (enkel peer feedback én leerkrachtfeedback) blijken echter significant minder vooruitgang te maken dan de groepen in de ‘uitgebreide’ feedbackcondities met het twijfel- of repliekformulier.
De vijfde bijdrage onderzoekt een groep van 68 eerstejaars leerlingen in het secundair onderwijs die een formatieve peer assessment hebben meegemaakt voor drie opeenvolgende schrijftaken. Ze zijn opgedeeld in twee experimentele condities (dezelfde als de ‘uitgebreide’ feedbackcondities in de vorige bijdrage) en een controlegroep met enkel peer feedback. De vooruitgang van leerlingen in schrijfprestaties wordt onderzocht ten opzichte van de mate van constructief zijn van de peer feedback die ze gaven en ontvingen, en ten opzichte van de conditie waartoe ze behoorden. Het effect van het constructief zijn van feedback wordt bestudeerd vanuit twee richtingen: vanuit het standpunt van de ontvanger van peer feedback (‘assessment for learning’) en vanuit het standpunt van de feedbackgever (‘assessing for learning’). De resultaten van de analyse van herhaalde metingen tonen een significant positief effect van de samenstelling van de ontvangen peer feedback op leerlingprestaties. Het constructief zijn van de feedback die leerlingen zelf verschaften blijkt daarentegen geen positief effect te hebben op hun leren. Globaal genomen is de feedback echter weinig constructief. Mogelijke barrières die leerlingen hebben weerhouden om goede feedback te geven, en oplossingen ervoor, worden besproken in de paper. Tot slot kan de studie het effect van conditie dat gevonden werd in de vierde bijdrage van dit proefschrift niet repliceren.
De zesde bijdrage vergelijkt de sterktes en zwaktes van peer feedback en docentfeedback vanuit het perspectief van de student. De studie is gesitueerd in een universiteitsvak met 192 eerstejaarssstudenten in de pedagogische wetenschappen. Algemene, collectieve docentfeedback op de kladversies van een reeks cumulatieve opdrachten wordt gecomplementeerd met een formatief peer assessment systeem. Startend van een hypothetisch opgelegde keuze worden de gepercipieerde kenmerken van beide bronnen van feedback in de diepte onderzocht, evenals hun gepercipieerde bijdrage aan een leeromgeving die tegemoet komt aan de behoeften van de lerende. Deze perspectieven worden aangevuld met de redenen die studenten rapporteren om één van beide bronnen van feedback te prefereren. Gesloten vragenlijst-items worden getrianguleerd met kwalitatieve data van open vragen. De resultaten tonen dat bij benadering de helft van de studenten bereid is om de geloofwaardigheid van docentfeedback in te ruilen voor de specificiteit van peer feedback als ze moeten kiezen. Beide bronnen van feedback blijken echter hun eigen sterktes en zwaktes te hebben vanuit het perspectief van de student. Ze zijn complementair en ze creëren zelfs de voorwaarden waaronder de complementaire bron sterker wordt. The dissertation includes three theoretical contributions and three empirical studies on peer assessment, a general introduction and final reflections including a discussion of the results, a discussion of the educational implications and a discussion of some methodological issues.
The first contribution delineates the role that peer assessment can play in raising the consequential validity of an assessment system. First, it clarifies the type of effects that assessment in general can have on learning, and formulates the design principles for increasing the consequential validity of an assessment system. Then, it is shown that peer assessment helps to meet the identified design principles that enhance consequential validity of an ‘assessment system’. More specifically, this dissertation shows that peer assessment can make it more feasible to include challenging and authentic tasks in one’s assessment system; it can help making the assessment demands more clear to the students; it can provide a supplement or a substitute for formative staff assessment; and finally, it can support the response to teacher feedback.
The second contribution goes beyond the impact of peer assessment on the consequential validity, and addresses the problem that the output of peer assessment is evaluated against a variety of quality criteria in the literature, resulting in a cluttered picture. The different conceptualisations of quality that appear in the literature are analysed. It is shown that discussions about the most appropriate quality criteria for the output of peer assessment should be brought back to the underlying differences in goals. The most obvious goal is its use as an assessment tool. The learning goal of peer assessment has also been well-established. Investigating the literature more closely yields three additional goals: installation of social control in the learning environment; preparation of students for self-monitoring and self-regulation in lifelong learning; and active participation of students in the classroom. Each of these goals results in different quality criteria. It is argued that only the criteria that are congruent with the goal that one is trying to achieve should be considered when evaluating the quality of peer assessment.
The third contribution starts from the observation that, together with the expansion of peer assessment research in the last decade also the diversity of peer assessment practices has increased exponentially. This diversity poses difficulties for practitioners as well as researchers. An inventory of peer assessment diversity is developed that may be of interest to practitioners, as a checklist of important decisions to take or an overview of possible alternatives to a specific practice, and to researchers, as a guideline of which information to provide on the particularities of their peer assessment design.
The fourth contribution compares the impact of peer feedback and teacher feedback on student learning, addressing the question whether peer feedback can serve as a substitute for expert feedback. A pretest posttest control group design examines the long term learning effects of individual peer feedback and collective teacher feedback on writing assignments in secondary education ( N =85). Moreover, it examines the added-value of two measures to support the response of the assessee to peer feedback: an a priori question form and an a posteriori reply form. The study shows no significant difference in students’ progress on essay marks between the condition with plain substitutional peer feedback and the control condition with teacher feedback. However, both groups (plain peer feedback ànd teacher feedback) appear to make significantly less progress then the groups in the ‘extended’ feedback conditions with the question or the reply form.
The fifth contribution examines a group of 68 first year students in secondary education who experienced formative peer assessment for three successive writing assignments. They are divided in two experimental conditions (similar to the ‘extended’ feedback conditions in the previous contribution) and a control condition with plain peer feedback. Students’ progress in writing performance is examined against the constructiveness of the peer feedback they gave and received, and against the condition in which they participated. The effect of the constructiveness of feedback is studied from two directions: from the point of view of the receiver of the peer feedback (‘assessment for learning’) and from the point of view of the assessor who gave peer feedback (‘assessing for learning’). The results of a repeated measures analysis show a significant positive effect of the composition of the received peer feedback on student performance. The constructiveness of feedback that students provided themselves is not found to improve their learning. However, the overall level of constructiveness of the feedback is low. Possible barriers preventing students from providing good feedback, and solutions to these, are discussed in the paper. Finally, the study cannot replicate the effect of condition that was found in the fourth contribution.
The sixth contribution compares strengths and weaknesses of peer feedback and staff feedback, from the student’s perspective. The study is situated in a university course with 192 first year students in educational sciences. Generic, collective staff feedback on the draft versions of a series of cumulative assignments is complemented with a formative peer assessment system. Starting from a hypothetical forced choice, a further in-depth study addresses the perceived characteristics of both sources of feedback and their perceived contribution to a learning environment that attends the learner’s needs. These perspectives are complemented with reasons reported by students to prefer one of both sources of feedback. Closed-ended questionnaire items are triangulated with qualitative data from open-ended questions. Results show that approximately half of the students were willing to trade in the credibility of staff feedback for the specificity of peer feedback if they have to choose. However, both sources of feedback show to have their own strengths and weaknesses from the student’s perspective. They are complementary and they even provide the conditions under which the complementary source becomes better. Doctor in de Pedagogische Wetenschappen Centrum voor Opleidingsdidactiek Onderzoekseenheid Pedagogische Wet. Faculteit Psychologie en pedagogische wetenschappen Doctoral thesis Doctoraatsthesis
Peer assessment is understood to be an arrangement with students assessing the quality of their fellow students’ writings and giving feedback to each other. This multiple‐case study of seven designs of peer assessment focuses on the contribution of peer assessment to the acquisition of undergraduates’ writing skills. Its aim is to arrive at an optimal design of peer assessment. Factors included in this study are: the quality of peer assessment activities, the interaction between students in oral peer feedback, students’ learning outcomes, and their evaluation of peer assessment. Most students took assessing the work of their fellow students seriously, and included the peer feedback in the revision of their work. In most conversations, students provided feedback in an evaluative manner. In others, the interaction was more exploratory. For peer assessment, we recommend a combination of written and oral peer feedback.
The nature of written and oral peer feedback will be described as it occurred in seven writing courses, each with a different design of peer assessment. In all courses, peer feedback was focused on evaluation, which is one of the four feedback functions. Feedback on structure was hardly provided. Relating feedback to design features, we suggest that feedback is adequate when (1) peer assessment has a summative (on the basis of a writing product) as well as a formative character (during the writing process); (2) the assessment is performed in small feedback groups; (3) the written feedback is orally explained and discussed with the receiver.
In this case study our aim was to gain more insight in the possibilities of qualitative formative peer assessment in a computer supported collaborative learning (CSCL) environment. An approach was chosen in which peer assessment was operationalized in assessment assignments and assessment tools that were embedded in the course material. The course concerned a higher education case-based virtual seminar, in which students were asked to conduct research and write a report in small multidisciplinary teams. The assessment assignments contained the discussion of assessment criteria, the assessment of a group report of a fellow group and writing an assessment report. A list of feedback rules was one of the assessment tools. A qualitative oriented study was conducted, focusing on the attitude of students towards peer assessment and practical use of peer assessment assignments and tools. Results showed that students' attitude towards peer assessment was positive and that assessment assignments had added value. However, not all students fulfilled all assessment assignments. Recommendations for implementation of peer assessment in CSCL environments as well as suggestions for future research are discussed.
DOI: 10.1080/135625100114885 Overcoming the potential dilemma of awarding the same grade to a group of students for group work assignments, regardless of the contribution made by each group member, is a problem facing teachers who ask their students to work collaboratively together on assessed group tasks. In this paper, we report on the procedures to factor in the contributions of individual group members engaged in an integrated group project using peer assessment procedures. Our findings demonstrate that the method we used resulted in a substantially wider spread of marks being given to individual students. Almost every student was awarded a numerical score which was higher or lower than a simple group project mark would have been. When these numerical scores were converted into the final letter grades, approximately one third of the students received a grade for the group project that was different from the grade that they would have received if the same grade had been awarded to all group members. Based on these preliminary findings we conclude that peer assessment can be usefully and meaningfully employed to factor individual contributions into the grades awarded to students engaged in collaborative group work. Author name used in this publication: Winnie Cheng
A General Practitioner (GP) is no longer a loner, but a team player in either a group practice or a care centre. This change has led to a concomitant growth in curricular interest in skills essential for successful collaboration and for enhancing critical reflection towards colleagues' performance. Giving and receiving constructive feedback are examples of these skills. The aim of this study was to gain insight in the style and quality of feedback reports on consultation skills written by GPs-in-Training (GPiTs) and by their GP-trainers. Furthermore, the preferences of the GPiTs concerning feedback style were examined. Results show significant differences between GP-trainers and GPiTs in feedback style and quality. A ranking task indicated that GPiTs have a preference for reports characterised by a large number of reflective remarks. Questionnaire results indicate the added value of the use of peer feedback. Implications for the integration of peer feedback activities in the curriculum of GPiTs are discussed.
Education is criticized for producing inert knowledge and for paying too little attention to skills such as cooperating and problem solving. Powerful learning environments have the potential to overcome these educational shortcomings. The goal of this research was to find out ways in which a ‘learning enterprise’ can best be supported (coached) in order to constitute a powerful learning environment aimed at teaching certain cooperative skills in a business context. This ‘learning enterprise’ constitutes an entrepreneurial context in which students in secondary or higher education are working together to conceptualize and eventually commercialize a product. In this research, the impact of different ways of supporting a learning enterprise will be compared. These ways are based on existing guiding principles for the design of powerful learning environments and on a further elaboration of these principles in what is conceptualized as an ‘equilibrium model’. In this model, the balances that are needed between motivating students, activating them towards self‐regulated learning, coaching, structuring and steering the learning processes have been elaborated. Based on this model a differentiation between a ‘student‐controlled’, a ‘teacher‐controlled and a ‘coached approach’, as an equilibrated way between the various approaches to coaching a learning enterprise, has been worked out. We hypothesize that the coached approach will give the best learning results in relation to cooperative skills. A combination of self, peer and teacher assessment of these skills, and an adequate feedback‐strategy based on these assessments, should be an important part of approaches used. These approaches were put into practice in a design experiment, and the impact was compared by means of a pre‐test/post‐test design. Results confirmed the postulated hypotheses that there will need to be a balance between, on the one hand giving students enough freedom for self‐discovery and self‐regulation, and on the other hand steering the students in such a way that certain problems can be avoided and that every student can get optimal learning chances. An adequate assessment‐strategy is needed to search for this balance. Further, a systematic action research of the design experiences resulted in more information on how best to coach a learning enterprise. This information has been summarized in the form of general guidelines.
Assessment procedures have a profound impact on the attitudes students take towards their work, their learning strategies and their commitment to learning. Most assessment of student learning is, however, undertaken with little or no consultation with students. Given the importance to students of developing the capacity for self‐assessment and evaluation, the unilateral control of assessment assumed by many academic staff can only be viewed as pedagogically unsound. It is important that students develop a clear understanding of tutor expectations regarding any learning task and this requires engagement in dialogue in the classroom setting. To become effective, independent, autonomous learners, students must be enabled to understand assessment criteria and expected levels of attainment. They should be empowered through meaningful feedback on their learning which relates to assessment criteria and which is given from a perspective of improvement on current attainment. This paper presents the case for academics to work in partnership with students, particularly in the context of assessment, for the purpose of enhancing learning and developing autonomous, independent and reflective learners.
One of the indicators of quality in undergraduate education is the extent to which self‐learning ability is nurtured. The authors have experimented with several approaches, some of which are reported in this paper. One part of the process of improving self‐learning ability lies in learning how to judge the quality of the work of others objectively and hence, eventually, one's own; another lies in finding and collating materials on a given topic in order to present it to others in note form, as an essay or as a brief ‘lecture’. Results are presented showing high correlations between students’ and lecturers’ assessments of individual essays and presentations. The paper addresses the need for these approaches to be made in absorbable and achievable steps, to build student confidence from experience and to give feedback to the students at each stage. Feedback concerning student opinion on the methods will be reported. The effect of students working in groups to give presentations or to make self‐ and peer assessments is also discussed. These approaches claim to maintain quality with increases in cohort sizes as education resource providers try to obtain improved productivity.
This article reports findings on the reliabilities of peer and teacher summative assessments of engineering students' oral presentation skills in a fourth year communications subject. The context of the study is unusual, in that each oral presentation was subject to multiple ratings by teams of students and teams of academic staff. Analysis of variance procedures were used to provide separate estimates of inter-rater reliability of assessments by peers and teachers for classes in four succeeding years. Teacher ratings were found to have substantially higher levels of inter-rater agreement than peer ratings. Generalising over the four years, it would require the averaging of between two and four peer ratings to match the reliability of single teacher assessments. However, the estimates of individual rater reliability for teachers, based on the intra-class correlation coefficient, were moderately low (0.40 to 0.53). It is concluded that the reliability of summative assessments of oral presentations can be improved by combining teacher marks with the averaged marks obtained from multiple peer ratings.
Various arguments have been made on affective grounds to justify peer feedback in teaching composition in English as a first language (L1). Those arguments have had considerable influence on the teaching of English as a second language (ESL) writing. Based upon current assumptions about the affective values of teacher-, peer-, and self-directed feedback, hypotheses were formulated concerning the relative appeal of the three types of feedback in the ESL writing process. Eighty-one academically oriented ESL learners who had experienced the three types of feedback responded to a questionnaire, and their preferences were statistically analyzed. The results show that claims made about the affective advantage of peer feedback in L1 writing do not apply to ESL writing. ESL students overwhelmingly prefer teacher feedback. The findings are discussed in conjunction with the larger issue of the appropriateness of L1 writing theories as guidelines for ESL writing research and instruction.
The debate surrounding assessment in education and the tensions it has caused is based upon conflicting ideologies in society. Enterprise in Higher Education and other initiatives have raised the awareness of staff to these issues and caused many of us to rethink the way in which we educate students. Changing the structure of assessment has enabled us to start adopting methods which are more student centred and appropriate to the demands of employers in particular and society as a whole more generally. There is a strong case for adopting a, more collaborative approach which breaks away from traditional norms and, while assessing students’ abilities in a summative sense, helps them to develop an awareness of their strengths and weaknesses in a formative way.
Forty-eight quantitative peer assessment studies comparing peer and teacher marks were subjected to meta-analysis. Peer assessments were found to resemble more closely teacher assessments when global judgements based on well understood criteria are used rather than when marking involves assessing several individual dimensions. Similarly, peer assessments better resemble faculty assessments when academic products and processes, rather than professional practice, are being rated. Studies with high design quality appear to be associated with more valid peer assessments than those which have poor experimental design. Hypotheses concerning the greater validity of peer assessments in advanced rather than beginner courses and in science and engineering rather than in other discipline areas were not supported. In addition, multiple ratings were not found to be better than ratings by singletons. The study pointed to differences between self and peer assessments, which are explored briefly. Results are discussed and fruitful areas for further research in peer assessment are suggested.
A definition and typology of peer assessment between students in higher education is proposed, and the theoretical underpinnings of the method are discussed. A review of the developing literature follows, including both process and outcome studies. This indicates that peer assessment is of adequate reliability and validity in a wide variety of applications. Peer assessment of writing and peer assessment using marks, grades, and tests have shown positive formative effects on student achievement and attitudes. These effects are as good as or better than the effects of teacher assessment. Evidence for such effects from other types of peer assessment (of presentation skills, group work or projects, and professional skills) is, as yet, more limited. Computer-assisted peer assessment is an emerging growth area. Important factors in successful implementation are summarized, and recommendations for future research and practice are made.
The value of peer and self-assessments is commonly diminished by scoring range restriction by the raters. This investigation studied the effect of different levels of scoring criteria specificity and written feedback requests on the distribution of scores, the correlation between peer and self-assessments and the quantity and nature of written feedback. Increasing the number of criteria decreased the mean scores and increased the standard deviations of the peer and self-assessments, providing a wider range of scores and increasing the sensitivity of the instrument. Correlation between peer and self-assessment was improved with more specific criteria, depending on the statistic used. However, analysis revealed that the more specific written feedback requests elicited more peer feedback. Educators should consider the effects of criteria specificity and written feedback solicitation on rater behaviour when designing these instruments.
To an increasing extent, teachers and students are leaving the well-known, traditional learning and assessment avenues and moving towards new learning environments and modes of assessment. However, the road from theory to practice seems to be a rocky one. This article presents the quantitative and qualitative data gathered from a research project which focused on different quality aspects of two new assessment forms in problem-based learning. A written examination was used to assess the extent to which students are able to define, analyse and solve novel, authentic problems. Peer assessment was introduced for students to report on collaborative work during the tutorial meeting, and during the study period that follows these weekly meetings. Students' perceptions are used as a tool for explaining the strengths and weaknesses of both instruments and the learning environment in which they are embedded. The article suggests that both assessment forms have acceptable qualities. The written examination seems to have an acceptable curricular, instructional and criterion validity. For the peer assessment, generalisability coefficients for both groups were over 0.75. Peer assessment seems to be accurate. Peer marks seem to correlate well with tutor marks and final examination scores. However, the students' learning outcomes, as measured by the examination, are lower than expected. Staff members experience the students' learning activities as disappointing. In the Louvain case, there are indications that the students cannot accurately reflect on their own functioning. In order to highlight plausible explanations, students' perceptions of the learning-assessment environment are analysed. In both cases, students perceive a gap between their working in the tutorial groups and the assessment. These results offer a valuable input for teachers to formulate concrete recommendations for optimising their educational and assessment practices.
Reciprocal paired qualitative formative peer assessment of academic writing was undertaken by 12 postgraduate students of educational psychology. Overall, staff and peer assessments showed a very similar balance between positive and negative statements, but this varied according to assessment criterion. However, only half of all detailed formative assessment statements made showed some degree of correspondence between staff and peers. Nevertheless, there was very little evidence of conflict between the views of staff and peers-rather, they focused on different details. Subjective feedback from students indicated that most found the process time consuming, intellectually challenging and socially uncomfortable, but effective in improving the quality of their own subsequent written work and developing other transferable skills. The reliability and validity of this type of peer assessment thus appeared adequate, and the partiality of overlap in detail between staff and peer assessments suggested that the triangulation peer assessment offers is likely to add value. However, caution is indicated regarding the generalisation of this finding. Implications for action are outlined.
Lack of research on the characteristics of peer assessment in EFL writing may inhibit teachers from appreciating the utility of this innovative assessment. This study addressed the following research questions: (1) How similar are peer, self- and teacher ratings of EFL writing?; (2) Do students favour peer ratings?; and (3) Does peer feedback influence students’ attitudes about peer rating? Forty-seven college students studying English writing in a Japanese college were assigned to write two essays. Each essay was commented on and rated by two teachers, three peers and the writers themselves. Students also completed a five-item questionnaire about their attitudes regarding peer rating. Peer and teacher ratings were found to correlate significantly. The results of the questionnaire indicated that students had favourable attitudes towards peer rating. A regression analysis suggested that peer feedback did not influence students’ favourable attitudes about the feedback.
In order to manage increased enrolment in a third‐year course in microclimatology, group work was introduced with seminar presentations and term papers. A peer‐ and self‐assessment strategy was developed that provided individual group members the opportunity to appraise their own and partners’ performance. The course instructor worked with an adviser from the university's educational development unit throughout the term to develop and critique the technique for this class. The techniques used, collaboration with the educational development unit, and student responses are described and explored within the context of an action‐based research project. Students indicated that the group work was a valuable and enjoyable learning experience which helped them to develop skills in independent research, collaboration and communication. The ‘zero‐sum’ assessment technique used in the course did help the instructor make a more accurate assessment of student performance in groups. Most students appreciated the use of self and peer assessment and believed it was used appropriately. The primary recommendation is to make the process as inclusive and participatory as possible. It is concluded that a ‘zero‐sum’ approach to self and peer assessment of group work is effective, and that a collaborative approach to educational development can be a positive experience that ultimately benefits students.
Papers in a large second-year science class were assessed using an anonymous peer review system modelled on that used for professional journals. Three students and one paid marker (outside reviewer) reviewed each paper—and each student received four reviews. A paid marker served as 'editor' and determined marks based on the four reviews, with reference to the paper as necessary. Students were asked to rank the four reviews for helpfulness and for completeness and accuracy. Consistency of reviews was analysed. On average, peer reviewers gave higher marks than paid markers, and on average students found peer reviewers to be more helpful but marginally less complete and accurate than paid markers. The differences among paid markers, however, were larger than the difference between the average peer reviewer and the average paid marker. The consistency among the four sets of marks given was not impressive. Students responded to the range of reviews they received. It can be shown statistically that the expected range for four reviews is much greater than that expected for two reviews—thus the multiplicity of reviews received exacerbated a widespread perception that marks were arbitrary. The net outcome was a moral dilemma. Giving the same paper to multiple assessors reveals the extent to which assessment rests on arbitrary factors. This may be good preparation for the real world; however, it is not an exercise to be taken lightly, and not recommended without prior preparation of context.
The study reports on an educational experience where peer assessment was introduced into the last four-month unit of the two and a half year preclinical problem-based learning (PBL) curriculum in medicine at the University of Sherbrooke. The assessment at mid-term was formative; the assessment at the end of the unit was summative. The study had two goals, to report on the feasibility of introducing a specific assessment instrument, and to assess students' perception of peer assessment while acting as both reviewers and reviewees. Students were involved in developing the peer assessment grid which includes communication skills and scientific reasoning. Peer assessment had been used for five years, before it was evaluated by the student body in preparation for an external accreditation visit. The response rate of this evaluation was 66%. Descriptive feedback of the assessment, in terms of its strengths and weaknesses, was given by 86% of students. They preferred the format being not anonymous but formative. A majority of students recognized that they had changed their behaviour after this peer assessment. The introduction of this form of assessment was feasible even in a preclinical PBL curriculum. Social acceptance was high, as the assessment was perceivend to be rewarding and worthwhile. Its formative character was appreciated much more than the summative one. Results seemed to indicate a development of positive student attitudes towards better health care.
In summary, this study showed promising results concerning the involved of students in assessment procedures. Both the teacher and the students experienced the importance of developing criteria. The criteria of reflection papers are often not transparent and differ by teacher. That teachers should be capable of writing such papers and that teachers at teacher training colleges should contribute to the development of this skill is accepted (Boud, 1989; Korthagen, 2000; Reilly-Freese, 1999; Kremer-Hayon, & Tillema, 1999), but teachers still have multiple perspectives on the concept of reflection and the effective use of reflective activities. Implementing activities like a peer assessment training helps in making these perspectives explicit.
This paper reports a study which implemented a method of student self and peer assessment involving student constructed marking criteria in the presence of exemplars. Pairs of first-year undergraduate biology students were asked to complete a poster assignment. The study was designed to allow the evaluation of (1) student self and peer marking and tutor marking for individual marking criteria following the use of exemplars; (2) the role of exemplars in providing a focus for formative feedback about subject standards. The present study shows that: (a) the use of exemplars can help students demonstrate greater understanding of both marking criteria and subject standards; (b) the use of exemplars can help students learning so that higher quality outcomes are produced; (c) the use of exemplars forms a focus for meaningful formative feedback; (d) students may make more objective judgements as a result of peer assessment compared to self-assessment.
This paper reports a study which implemented and evaluated a method of student self and peer assessment involving student constructed marking criteria. The theme of the importance of marking criteria (Orsmond et al ., 1996, 1997) is developed. Pairs of first-year undergraduate biology students were asked to complete a poster assignment. The study was designed to allow the comparison and evaluation of (1) student self and tutor marking for individual marking criteria; (2) student self and peer marking for individual marking criteria; and (3) student and tutor marking for student constructed and tutor provided individual marking criteria. The present study shows that: (a) students may be less able to discriminate between individual marking criteria which they have constructed compared to marking criteria that have been provided; (b) asking students to construct their own marking criteria in discussion with tutors or fellow students does not enhance agreement between student/tutor or student/student marking; and (c) allowing students to construct their own marking criteria may lead to different learning outcomes compared to providing students with a marking criteria.
This paper reports a study which implemented and evaluated a method of peer assessment, as a formative and summative assessment procedure. Pairs of first‐year undergraduate biology students were asked to complete a poster assignment on a specific aspect of nerve physiology. This paper contains details of a method which allows student peer and tutor marking of work against the individual marking criteria to be evaluated. The results show that a comparison between the tutor and the student peer mark may be misleading as a guide to the validity of peer assessment. The importance of considering the individual sections of the marking criteria is illustrated. It was found that when the individual criteria were analysed the number of students marking the same as the tutor ranged from 31% to 62%. It also became clear that specific areas of the marking criteria were prone to over and under‐marking. Analysis of student feedback forms showed that students not only liked carrying out peer assessment, but felt the benefits in terms of developing facets of their learning process and heightening their awareness of their work. These results are discussed in the light of other studies.
This paper describes the experience of introducing student assessment of contribution to a final year pharmaceutics formulation group assignment, using a combined self- and peer-assessment approach. The Contribution Weighting Factors (CWFs) calculated from the student assessments were very tightly distributed, and the resulting adjustment to the overall individual mark received by the majority of the students was small. Very similar contributing factors would have also been obtained if only student peer-assessment had been utilised.
This study canvasses reliability of students' self and peer evaluation, a method of assessment of university students that has recently gained renewed pedagogical interest and broad recognition. Two experiments, imbedded in classroom curriculum, examined the effects of the instrument of evaluation (with criteria vs. no criteria for evaluation provided), the format of evaluation procedure (anonymous vs. non-anonymous), and motivation of students (strong vs. weak) on the reliability of students' self and peer ratings. The results of the experiments revealed both a considerable unreliability of peer ratings in some cases as well as a notable consistency of peer evaluations in others. The instrument of evaluation with criteria provided had significant positive effect on the reliability of peer evaluations. Students strongly motivated to apply criteria for evaluation also produced more reliable peer evaluations compared to their peers that were weakly motivated to apply them. The results of the impact of the condition of anonymity were mixed. The study found that students exhibit significant “self-enhancement” bias by rating their academic performance above that of their peers. The positive self-bias does not disappear when students are offered criteria for evaluation of their work.
Undergraduate students, and their class teachers, assessed the performance of their peers in three oral and written tasks as part of a group project. The two sets of marks awarded by peers and teachers were subsequently compared to find out whether the students were competent to assess their peers alongside their class teachers and whether this competence, or lack of it, was partly determined by the nature of the task being assessed. A number of statistical tests were run to establish the levels of agreement, the ranges, differences and relationship between peer and teacher assessments. The results have led us to conclude that the peer assessments are not sufficiently reliable to be used to supplement teacher assessments. Students’ competencies in peer assessment do not appear to be dependent on the nature of the task being assessed, but there is some evidence that practical experience of assessing a particular task type can lead to an improvement in students’ assessment skills when they assess a similar task. The paper also discusses possible improvements in peer assessment procedures based on the experiences gained.
Peer assessment is a method of motivating students, involving students marking and providing feedback on other students' work. This paper reports on the design and implementation of a novel web-based peer assessment system for computer programming courses, and discusses its deployment on a large programming module. The results indicate that this peer assessment system has successfully helped students to develop their understanding of computer programming.
This study analyzes the interaction during peer response as it occurs in an authentic writing class. Transcripts of 27 response groups are analyzed using the constant comparative method (Glaser & Strauss, 1967) to identify reader stances and determine the characteristics of these stances. We identified four categories of reader stances—authoritative, interpretive, probing, and collaborative. We then analyzed the language functions and topics discussed during the response sessions. This analysis reveals interesting differences across the four stances in five function categories (Summarize Essay, Express Intention, Give Suggestion, Give Opinion, and Give Information) and in three content categories (Writing Process, Ideas, and Audience and Purpose). We conclude that interactive peer response offers benefits to the students. However, in the probing and collaborative stances, the writer is encouraged to articulate the intended meaning of the text, thereby clarifying, expanding, and shaping meaning. These two stances therefore engage students in a fuller understanding of the writing process.
The importance of formative assessment instudent learning is generally acknowledged, butit is not well understood across higher education.The identification of some key features offormative assessment opens the way for adiscussion of theory. It is argued that thereis a need for further theoretical developmentin respect of formative assessment, which needsto take account of disciplinary epistemology,theories of intellectual and moral development,students' stages of intellectual development,and the psychology of giving and receivingfeedback. A sketch is offered of the directionthat this development might take. It is notedthat formative assessment may be eitherconstructive or inhibitory towards learning. Suggestions are made regarding research intoformative assessment, and how research mightcontribute to the development of pedagogicpractice.
Peer assessment can be a valuable learning tool in teacher education because it supports student teachers to acquire skills that are essential in their professional working life. This article presents a conceptual framework in which the training of peer assessment skills by means of peer assessment tasks is integrated in teacher education courses. Theories about constructive alignment, student involvement, instructional design, and performance assessment underlie the framework. Furthermore, two recently published empirical studies will be briefly described to provide empirical support for the value of the framework. Results of these studies show that the framework offers powerful guidelines for the design and integration of peer assessment activities in teacher training courses. In general, the peer assessment tasks that were embedded in the courses led to a general improvement in students' peer assessment skills as well as their task performance in the domain of the course. Implications for course and curriculum design are discussed.
Incl. bibl., abstract. This paper examines the process of implementing peer assessment in higher education. In order to discover the aspects by which the method could be utilised to its full potential, it has been tried out among diverse types of students using an action research approach. The paper depicts the way the method was introduced and portrays students' reactions. It demonstrates the contribution of previous exposure to peer assessment, of a good preparation process and of student involvement in criteria setting to building up student confidence in using the method. It further shows how student involvement in establishing the assessment criteria contributes to their learning process. Based on the results of the action research, the paper recommends introducing peer assessment into the curriculum gradually and in a consistent way while involving students in the process of criteria setting and making the method relevant to student learning and future careers.
Peer assessment has been used successfully in higher education, with important benefits reported in terms of student learning. However, most of the literature has focused on its use with small groups of students taught by staff who are committed to the peer assessment process. This paper reports the development of peer assessment procedures for use in large classes, using a cyclical process of action, reflection and refined action. The project was carried out in three phases and after each phase changes were made to the procedures in response to student and staff feedback. The development of procedures is discussed in relation to assessment tasks, assessment criteria, anonymity, procedural guidelines, distribution systems, marking procedures and tutor remarking. Although there are specific difficulties associated with the use of peer assessment in large classes, this study suggests that these are outweighed by the learning benefits for students. Based on the findings of this study, recommendations are made for ways in which peer assessment might be successfully applied in large classes.