ArticlePDF Available

Formative and Summative Interpretations of Assessment Information

Authors:
Who says formative assessment matters 1
Formative and Summative Interpretations of Assessment Information
John Hattie
School of Education
The University of Auckland
Auckland, New Zealand
January 2003
Who says formative assessment matters 2
FORMATIVE AND SUMMATIVE INTERPRETATIONS OF ASSESSMENT
INFORMATION
There have been many articles claiming that formative rather than summative
assessment needs to be the priority of teachers. The typical claim is that formative
assessment is about discovering what the learner knows, understands or can do.
Summative assessment, in contrast is about whether the learner knows, understand or can
do a predetermined thing (cf., Torrance & Pryor, 1998, p. 153). Hence, the conclusion
that teachers need more formative assessment and less summative assessment. The aim of
this paper is to dispel this claim, especially as the repetition of this claim has major
negative implications for the quality of teaching and learning.
We first set the scene by claiming that assessment is about feedback information (and
not about “tests” although tests can be one mechanism to provide feedback information)
Setting the Scene: Assessment Is Feedback Information
Assessment is primarily concerned with providing teachers and/or students’ feedback
information, which they then need to interpret when answering the three feedback
questions: Where am I going?, How am I going? and Where to next?. Specifically,
feedback is actions or information provided by an agent (e.g., teacher, peer, book, parent,
experience) that provides information regarding aspects of one’s performance or
understanding. Feedback is information about the task that fills a gap between what is
understood (How am I going?) and what is aimed to be understood (Where am I going?).
It can lead to increased effort, motivation or engagement to reduce the discrepancy
between current status and the goal, it can lead to alternative strategies to understand the
material, it confirm to the student that they are correct or incorrect, it can indicate that
Who says formative assessment matters 3
more information is available or needed, it can point to directions that the students could
pursue, and it can lead to restructuring understandings (Where to next?).
Clearly assessment needs to relate to providing answers to the three feedback
questions. Thus, teachers and students need to devise assessments and seek information
about the task, about the processes or strategies to understand the tasks, about the
regulation, engagement and confidence to become more committed and learn, and/or
about the discrepancy between current status and the goal. As Shephard (2000) noted, the
successful teacher is “able to ask the right questions at the right time, anticipate
conceptual pitfalls, and have at the ready a repertoire of tasks that will help students take
the next steps requires deep knowledge of subject matter” (p. 12).
Crooks (1988), and more recently, Black and Wiliam (1998) have demonstrated there
is little evidence that classroom assessment has assisted in the learning process. Black
and Wiliam, for example, reviewed 578 publications relating to the role of assessment in
learning and concluded that classroom assessment “typically encourages superficial and
rote learning, concentrating on recall of isolated details, usually items of knowledge
which pupils soon forget, ... teachers do not generally review the assessment questions
that they use and do not discuss them critically with peers, so there is little reflection on
what is being assessed” (p. 17). Black and Wiliam highlighted the potential of assessment
to provide feedback, and the importance of the social context of learning. We have
argued elsewhere (Hattie & Jaeger, 1998) that Black and Wiliam needed to go further and
should have argued that effective learning and assessment occurs not by being concerned
about the testing itself, but by the manner in which the assessment is integrated into the
cycle of challenging goals—student receptivity to the challenges—and feedback.
Who says formative assessment matters 4
This cycle requires excellent teachers who are thinking, reflective, enthusiastic,
passionate, and knowledgeable about the content. These teachers need to care to make the
difference, preferably caring about their subject matter, and caring about all of their
students’ achievements. Excellent teachers encourage their students to share in seeking
answers to our three feedback questions.
There are many ways in which teachers can deliver feedback to students, and for
students to receive feedback from teachers, peers and other sources. Tests can perform
this function but too often they are devoid of much feedback to the students and thus can
be an inefficient method of providing feedback.
Assessment and Formative and Summative Interpretations
The terms formative and summative where introduced by Michael Scriven 30+ years
ago (Scriven, 1967), and it is worth tracing the history of these terms and their various
uses and abuses (Scriven, 1967). As Scriven outlined, the distinction between normative
and summative is more related to interpretations and time—as illustrated by Bob Stake’s
maxim: When the cook tastes the soup it is formative, when the guests taste the soup it is
formative. Thus, a key issue is timing, and it is possible that the same stimulus (e.g.,
tasting the soup) can be interpreted and used for both forms of assessment. Hence, it is
NOT the instrument (tasting) that is formative and summative it is the timing of the
interpretation and the purpose to which the information is used.
Similarly, it is possible to take what many consider a “summative test”, the PATs or
asTTle and use the feedback information from them to make decisions to enhance the
program (improve the soup) while in the making, or to make statements about students
when they have completed a program of study. Hence, it is the interpretations and NOT
Who says formative assessment matters 5
the test/form of assessment that is formative or summative. The same interpretations
could be used both in a formative and summative manner. As Scriven has noted it is a
fallacy to assume that formative and summative represent two types of assessment.
Instead, they refer to interpretations of information at two differing times—either to make
interpretations that can then lead to changing a program of learning, or that can lead to a
statement about the learning at the end of the program.
In the same way that the goal of the cook is to make the best soup possible for the
guests, it is imperative that teachers have excellence summative evaluation in place in
their classes (and Principals in their school) as that can be among the more powerful
evidence that there is likely to be excellent formative evaluation in place. Poor soup for
the guests is pretty powerful evidence of poor cooking. If a school has poor summative
assessment in place then it is unlikely they will have the ability, purpose or wherewithal
to be concerned with formative assessment. Serving poor soup to the guests is probably
the best indicator that the cook was lousy at tasting it during the preparations. Too much
reliance on tasting the soup, may lead to inattention to the goals—such as making soup
cold for when the guests arrive. The goals or learning intentions are critical aspects of this
process, and can have major effects on the usefulness of the formative and/or summative
evaluation.
It is worth rehearsing some of the further claims Scriven has made about formative
and summative assessment. First, sometimes we do not know when administering an
assessment whether it is formative or summative. For example, a teacher may consider
evaluating the course of teaching by using an end-of-course test, but then find that
remedial work is necessary making the test interpretations more formative than intended.
Who says formative assessment matters 6
Too often formative interpretations are used to make teachers believe that they have
made progress, and thus time to move on to other topics etc., hence formative
interpretations can be turned into summative interpretations. Too often, however, an
emphasis on “improvement” as if “any” evidence of improvement is necessarily a sign of
progress is a weak yardstick indeed. When over 500,000 studies assessing the influence
of many aspects to enhance achievement (e.g., policy, student prior learning, home,
school, curricula, teacher), over 95% of all effects indicate students have “improved”.
Almost everything works. So saying that there is evidence of “improvement”/ progress is
trivial—such evidence is rarely and barely enough. It is the level and rate of progress that
is critical, and it must be related to some defensible goal of progress (in effect-size terms
at least .40). Too often in education, the goal posts are set so low (at anything greater than
zero), too often satisfaction that there is progress (> 0) is the goal, and we do not question
vigorously enough whether sufficient progress has been made, or whether the progress
goal was high enough.
That the distinction between formative and summative relates more to the timing of
the interpretations, than to the form of assessment has major implications for the quality
of the interpretations from both. Undertaking formative evaluation any less rigorously
than summative evaluation undermines the accuracy of the mid-course corrections, which
is all to likely to send the mission in the wrong direction. Contrary to popular utterance, it
is the formative assessments that need to be more rigorous, as too often mid-course
corrections, evaluations about progress are based on very weak evidence, and very low
goal posts. Too often, we underestimate the importance of accuracy and dependability in
formative assessment—diagnosis and corrections need to be excellent.
Who says formative assessment matters 7
Conclusions
Assessment is primarily concerned with providing teachers and/or students’ feedback
information, and thus needs to address three questions: What are the learning intentions?
How am I going? And Where to next?
The distinction between normative and summative is more related to interpretations
and time as illustrated by Bob Stake’s maxim: When the cook tastes the soup it is
formative, when the guests taste the soup it is formative. It is NOT the instrument that is
formative and summative it is the timing of the feedback. Sometimes we do NOT know
when we administered an assessment whether it is formative or summative. Summative
evaluations can be powerful in education and Formative evaluations can be ineffectual in
education (and vice versa).
Too often formative is seen as less rigorous or more informal than summative, when
it needs to be the opposite. Doing formative evaluation any less rigorously than
summative evaluation undermines the accuracy of the mid-course corrections, which is
all to likely to send the mission in the wrong direction.
Schools that create a climate that all are responsible for the progress of the students,
schools that deprivatises the information and evidence, and schools that collaborate to
improve learning are great schools—it is simple. When did you meet with colleagues and
talk about the evidence of progress of your and their students, talk about how to improve
your teaching, how to change your teaching, and how to do this in light of the evidence
that what you are currently doing is just not good enough to have the effect on progress
of students? Do you feel psychological safe to discuss how to improve your teaching (not
the kids, not the curriculum, not the resources, not the class size, not the conditions) but
Who says formative assessment matters 8
YOUR teaching, with colleagues. Such schools are assessment rich schools where the
probability of great learning is occurring, and the teachers are mirroring that they ask of
their students—let’s learn together and respect each other by seeking evidence we are
doing a great job. Teachers need to share assessment, students, and teaching. Teachers
need to see assessment as saying something about what they (not the kids) are doing.
We need to use the terms formative and summative to relate to the timing of
interpretations, and not as if they pertain to a type of test. We need to talk about the
quality of the evidence and the nature of the interpretation, and the excellence of the
decision making.
Who says feedback about the quality of my teaching matters—me. When you use
asTTle, Exemplars, ARBs, NEMP or any assessment tool, I hope you do learn about your
teaching while you are seeking evidence about your students progress and about what is
not working and thence to improve the teaching and learning, and that this informs you
about the closeness and adequacy of your learning intentions, and that it does help in
deciding where to go next. The art and science of effective student learning is directly
related to the quality of decision making by the teacher—go forth and seek that evidence
in dependable formative and summative ways.
Who says formative assessment matters 9
References
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in
Education: Principles, Policy & Practice, 5(1), 7-74.
Crooks, T. J. (1988). The impact of classroom evaluation practices on students. Review of
Educational Research, 58(4), 438-481.
Hattie, J., & Jaeger, R. (1998). Assessment and classroom learning: A deductive
approach. Assessment in Education: Principles, Policy & Practice, 5(1), 111-122.
Scriven, M. (1967). The methodology of evaluation. In R. E. Stake (Ed.), Perspectives of
curriculum evaluation (Vol. 1, pp. 39-55). Chicago: Rand McNally.
Shepard, L. A. (2000). The role of assessment in a learning culture. Educational
Researcher, 29(7), 4-14.
Torrance, H., & Pryor, J. (1998). Investigating formative assessment: Teaching, learning
and assessment in the classroom. Florence, KY: Taylor & Francis.
... One more type that requires complex technology is virtual reality, which is likely to facilitate immersion (e.g., Ahlberg et al., 2007). Empirical research on the effects of technology use on learning (comparing the use of computers in the classroom with no technology) provides some supportive evidence of small to moderate positive effects of technology use on learning and achievement (Hattie, 2003;Tamim et al., 2011). Some evidence of no particular effects of technology use comes from medical education comparing high-and low-fidelity simulations for learning (e.g., Ahad et al., 2013). ...
... There is only a very small number of instructional methods for which these relations hold true. These include feedback and formative assessment (see Hattie, 2003;Hattie & Timperley, 2007), which already point to possible interpretations of effects that large. One of the issues in higher education is the lack of feedback in the context of complex authentic activities. ...
Article
Full-text available
Simulation-based learning offers a wide range of opportunities to practice complex skills in higher education and to implement different types of scaffolding to facilitate effective learning. This meta-analysis includes 145 empirical studies and investigates the effectiveness of different scaffolding types and technology in simulation-based learning environments to facilitate complex skills. The simulations had a large positive overall effect: g = 0.85, SE = 0.08; CIs [0.69, 1.02]. Technology use and scaffolding had positive effects on learning. Learners with high prior knowledge benefited more from reflection phases; learners with low prior knowledge learned better when supported by examples. Findings were robust across different higher education domains (e.g., medical and teacher education, management). We conclude that (1) simulations are among the most effective means to facilitate learning of complex skills across domains and (2) different scaffolding types can facilitate simulation-based learning during different phases of the development of knowledge and skills.
... Formative assessment and summative assessment are commonly used in educational settings to give information regarding student achievement (Hattie, 2003). ...
... Teacher education has emphasized teachers' assessments of their students' knowledge and skills. This research has focused on the accuracy of teachers' judgments of students' learning prerequisites and outcomes in primary and secondary school (Klug, Bruder, Kelava, Spiel, & Schmitz, 2013;Spinath, 2005; as well as on the role of teachers' diagnoses in formative assessment (e.g., Glogger-Frey et al., 2018;Bennett, 2011;Hattie, 2003). However, diagnostic activities in teaching have not yet been systematically investigated (e.g., Herppich et al., 2018;Shulman, 2015). ...
Article
Full-text available
We propose a conceptual framework which may guide research on fostering diagnostic competences in simulations in higher education. We first review and link research perspectives on the components and the development of diagnostic competences, taken from medical and teacher education. Applying conceptual knowledge in diagnostic activities is considered necessary for developing diagnostic competences in both fields. Simulations are considered promising in providing opportunities for knowledge application when real experience is overwhelming or not feasible for ethical, organizational or economic reasons. To help learners benefit from simulations, we then propose a systematic investigation of different types of instructional support in such simulations. We particularly focus on different forms of scaffolding during problem-solving and on the possibly complementary roles of the direct presentation of information in these kinds of environments. Two sets of possibly moderating factors, individual learning prerequisites (such as executive functions) or epistemic emotions and contextual factors (such as the nature of the diagnostic situation or the domain) are viewed as groups of potential moderators of the instructional effects. Finally, we outline an interdisciplinary research agenda concerning the instructional design of simulations for advancing diagnostic competences in medical and teacher education.
... Both work as a learning evaluation for the students, and their relative weights should be decided by the nature of the subject [1]. The difference between formative and summative assessments is more related to time and interpretations; in other words, the timing of the feedback [2]. With formative assessment, feedback provides the gap between desired and actual levels of performance [3]. ...
... Coming back to the question of what formative assessment constitutes, the particular importance of eliciting evidence and providing feedback has been emphasized by many authors (e.g., Black & Wiliam, 1998aHattie, 2003;Kingston & Nash, 2011;Sadler, 1998;Stiggins, 2006). According to McMillan et al. (2013) it is apparent that in most studies in the field emphasis has been on gathering data and providing feedback, and little emphasis has been on instructional correctives. ...
Article
Full-text available
Although formative assessment is regarded as a promising way to improve teaching and learning, there is considerable need for research on precisely how it influences student learning. In this study we developed and implemented a formative assessment intervention for mathematics instruction and investigated whether it had effects on students' interest and achievement directly and via students' perception of the usefulness of the feedback and their self-efficacy. We conducted a cluster randomized field trial with pretest and posttest. The 26 participating classes were randomly assigned to a control group or the intervention group. Results of path analyses indicate that feedback was perceived as more useful in the formative assessment condition, self-efficacy was greater, and interest tended to increase; learning progress did not differ between the groups. The assumed indirect effects were partly confirmed: formative assessment showed an indirect effect on interest via its perceived usefulness.
Article
Full-text available
Det tværprofessionelle element undersøger samarbejdet mellem lærere og pædagoger. Her foreligger der: 1) feltstudie samt analyse af samarbejdet mellem lærer og pædagog, 2) pædagog- og lærervejledning til et selvskabt læremiddel i sammenhæng med det tværprofessionelle samarbejde og 3) teoretisk redegørelse for det digitale læremiddel. Denne undersøgelse er fremstillet i sammenhæng med at opfylde kravene for TPE-forløbet (2. og 3. del) på læreruddannelsen.
Article
Full-text available
In teaching, one must consider the research on language acquisition, attending to the deliberate practice of each language skill. Accordingly, Paul Nation emphasizes that language fluency is advanced by concentrating on “each of the four skills of listening, speaking, reading and writing”. This lesson plan here is thus designed to facilitate language acquisition and communication, developing the four skills. On this note, the language learning pedagogy encompasses the language skills, while the strands framework instigates a thorough pedagogic and didactic process of assembling and evaluating this language course.
Chapter
Das den Lernprozess kontinuierlich begleitende Diagnostizieren und Rückmelden von Schüler*innenleistungen (sog. formatives Assessment) gilt als einer der erklärungsmächtigsten Faktoren für die Qualität von Unterricht und schulisches Lernen. Weitestgehend unklar ist bislang jedoch, wie genau die Implementation so verstandenen formativen Assessments (bereits) in den alltäglichen Mathematikunterricht der Grundschule gelingen kann. Der vorliegende Beitrag greift dieses Desideratum auf und berichtet über eine entsprechende Umsetzung formativen Assessments im Mathematikunterricht eines zweiten Schuljahrs. Im Rahmen einer sechswöchigen Unterrichtseinheit zur Orientierung im Hunderterraum (Zahlraumerweiterung) wurde hierzu von der Erstautorin ein Instrument zur Umsetzung individueller Diagnose und lernprozessbegleitender Rückmeldung entwickelt und erprobt. Empirische Ergebnisse zeigen: Schüler*innen nehmen bewusst wahr, dass mit Hilfe dieses Instruments ihr individueller Kenntnisstand durch die Lehrkraft treffend diagnostiziert wird, dass nächste Lernschritte verständlich aufgezeigt und dass adaptive Übungsaufgaben angeboten werden. Ausgehend von der bekannten empirischen Erkenntnis, dass formatives Assessment, wenn es Lernprozesse wirksam unterstützen soll, von den Lernenden auch als unterstützend und hilfreich wahrgenommen werden muss, belegen diese Ergebnisse die Qualität des Instruments. Der Beitrag zeigt somit (unter Rückgriff auf konkrete Materialien), dass die Umsetzung individueller Diagnose und lernprozessbegleitender Förderung als zentrale Momente formativen Assessments bereits im Mathematikunterricht einer zweiten Klasse gelingen kann. Es wird jedoch ebenso deutlich: Die Umsetzung des vorgestellten Konzepts stellt teils einen erheblichen zeitlichen Mehraufwand und damit eine zusätzliche Belastung für Lehrkräfte dar.
Article
Full-text available
This paper outlines and discusses most of the literatures on formative and summative assessment. It considers some of the key features of both formative and summative assessment as a means of evaluating teaching and learning processes. This study suggests that both types of assessment should be proportionally designed and aligned with learning goals. More importantly, this study proposes three key aspects of assessment that should be considered in designing effective assessment policy: (a) assessment forms an integral part of planning, assessing, and reporting, (b) the use of assessment as a means to monitor the progress of student learning and achievement, (c) assessment as a tool to determine the effectiveness of teaching. The three aspects of assessment policy can help educational leaders to design a proportional formative and summative assessment for an effective assessment policy. Abstrak Artikel ini memaparkan dan mendiskusikan literatur-literatur tentang penilaian formatif dan sumatif. Artikel ini juga menyuguhkan gambaran penting penilaian formatif dan sumatif sebagai alat evaluasi pengajaran dan proses pembelajaran. Kajian ini menemukan bahwa kedua jenis penilaian tersebut harus didesain secara proporsional dan harus sejalan dengan tujuan pembelajaran. Lebih penting lagi, kajian ini menawarkan tiga aspek penting yang harus dipertimbangkan dalam merancang kebijakan penilaian yang efektif.: (a) penilaian harus membentuk sebuah bagian integral dari perencanaan, penilaian dan pelaporan, (b) penggunaan penilaian sebagai alat untuk memonitor progress pembelajaran siswa dan pecapaiannya, (c) penilaian sebagai alat untuk menentukan efektivitas mengajar. Tiga aspek kebijakan penilaian tersebut dapat membantu pemimpin pendidikan untuk merancang penilaian formatif dan sumatif yang proporsional untuk sebuah kebijakan penilaian yang efektif.
Article
Full-text available
Zusammenfassung Formatives Assessment umfasst die Gewinnung von diagnostischen Informationen zum Lernen und seinen Ergebnissen sowie deren Nutzung zur Optimierung von Lehr- und Lernprozessen. Das Ziel des Forschungsprojekts bestand darin, zu untersuchen, mit welcher Häufigkeit, mit welcher Dauer und in welcher Qualität Lehrpersonen formatives Assessment informell im alltäglichen Unterricht durchführen. Um diese Forschungsfragen zu klären, wurde in der vierten Primarschulstufe von 52 Lehrpersonen und ihren Schulklassen im Fach Mathematik je eine Doppelstunde zur Einführung in das halbschriftliche Dividieren videografiert. Im Fokus standen fünf Strategien des formativen Assessments: (1) Lernziel klären, (2) Lernstand erfassen (Eliciting-Evidence), (3) Self-Assessment, (4) Peer-Assessment und (5) Feedback-Interaktion. Hinsichtlich der Häufigkeit ergaben die Videoanalysen, dass der überwiegende Teil der teilnehmenden Lehrpersonen alle fünf Strategien nutzt, die Einsatzdauer mit Ausnahme der Feedback-Interaktionen in der Regel jedoch kurz ist. Die Ergebnisse der Qualitätseinschätzung deuten insgesamt auf eine höchstens mittlere, meist aber geringe Qualität der Umsetzung der Strategien hin. Es bestehen Zusammenhänge zwischen der Anwendungsdauer und der Qualitätsausprägung der Strategien. Je länger die Strategien eingesetzt wurden, desto besser wurde ihre Qualität im Rating eingeschätzt. Diese Ergebnisse erweitern den bisherigen Forschungsstand insofern, als erstmals ökologisch valide Aussagen für den deutschen Sprachraum zur Häufigkeit, zur Dauer und zur Qualität von Strategien des formativen Assessments im alltäglichen Mathematikunterricht vorgelegt werden. Aus den Befunden werden Schlussfolgerungen für die Aus- und Weiterbildung von Lehrpersonen gezogen.
Article
Full-text available
In most educational programs, a substantial proportion of teacher and student time is devoted to activities which involve (or lead directly to) evaluation by the teacher of student products or behavior. This review summarizes results from 14 specific fields of research that cast light on the relationships between classroom evaluation practices and student outcomes. Particular attention is given to outcomes involving learning strategies, motivation, and achievement. Where possible, mechanisms are suggested that could account for the reported effects. The conclusions derived from the individual fields are then merged to produce an integrated summary with clear implications for effective educational practice. The primary conclusion is that classroom evaluation has powerful direct and indirect impacts, which may be positive or negative, and thus deserves very thoughtful planning and implementation.
Article
Based on empirical work with classes of 4- to 7-year-olds in English primary schools, this book provides an analysis of how classroom assessment occurs within routine teacher-student interaction. The chapters are: (1) "Introduction"; (2) "Defining and Investigating Formative Assessment"; (3) "Teachers' Perceptions of 'Teacher Assessment'"; (4) "Classroom Assessment and the Language of Teaching"; (5) "The Power of Assessment: Appropriating Children's Responses for Learning, or Social Control?"; (6) "Formative Assessment and Learning: Where Psychological Theory Meets Educational Practice"; (7) "Ask a Genuine Question, Get a Genuine Answer"; (8) "Constructing and Integrating Assessment and Learning"; and (9) "Formative Classroom Assessment: Prospects for Improvement." An appendix contains the transcript conventions for classroom interaction. (Contains 19 tables and 140 references.) (SLD)