Content uploaded by Dylan Wiliam
Author content
All content in this area was uploaded by Dylan Wiliam
Content may be subject to copyright.
Wor k i ng
Inside the Black Box:
Assessment for Learning in the Classroom
In their widely read article “Inside the Black Box,” Mr. Black
and Mr. Wiliam demonstrated that improving formative assessment
raises student achievement. Now they and their colleagues report on
a follow-up project that has helped teachers change their practice
and students change their behavior so that everyone shares
responsibility for the students’ learning.
BY PAUL BLACK, CHRISTINE HARRISON, CLARE LEE, BETHAN MARSHALL, AND DYLAN WILIAM
SEPTEMBER 2004 9
PAUL BLACK is a professor emeritus, Department of Education and Professional Studies, King’s College London, where CHRISTINE
HARRISON is a lecturer in science education. CLARE LEE is a teacher advisor for the Wa r w i ckshire County Council. BETHAN MAR-
SHALL is a lecturer in English education, Department of Education and Professional Studies, School of Education, King’s College
London. DYLAN WILIAM is director of the Learning and Te a ching Research Center, Educational Testing Service, Princeton, N.J. Th e y
wish to thank the Medway and Oxfordshire local education authorities and the schools directly involved in the project described
here, especially the teachers who shouldered the greatest burdens. The work on wh i ch this article is based was supported by gra n t s
from the Nuffield Foundation and from the U.S. National Science Foundation, as part of the Stanford University Classroom Assess-
ment Project to Improve Te a ching and Learning (CA P I TAL). But the opinions expressed are those of the authors. © 2004, Dylan Wi liam.
IN 1998 “Inside the Black Box,” the predecessor
of this article, appeared in this journal.1Since then
we have learned a great deal about the practical
steps needed to meet the purpose expressed in the
a r t i c l e ’s subtitle: “raising standards through class-
room assessment.”
In the first part of “Inside the Black Box,” we
set out to answer three questions. The first was, Is
there evidence that improving formative assess-
ment raises standards? The answer was an un-
equivocal yes, a conclusion based on a review of
evidence published in over 250 articles by researchers
from seve ral countries.2Few initiatives in education have
had such a strong body of evidence to support a claim to
raise standards.
This positive answer led naturally to the second ques-
tion: Is there evidence that there is room for improve m e n t ?
Here again, the available evidence gave a clear and pos-
itive answer, presenting a detailed picture that identified
three main problems: 1) the assessment methods that
t e a chers use are not effective in promoting good learning,
2) grading practices tend to emphasize competition ra t h e r
than personal improvement, and 3) assessment feedback
often has a negative impact, particularly on low-achiev-
ing students, who are led to believe that they lack “abili-
ty” and so are not able to learn.
Phi Delta Kappan, Vol. 86 #1, pp. 8-21, 2004
10 PHI DELTA KAPPAN
H ow e ve r , for the third question — Is there evidence about
h ow to improve formative assessment? — the answer wa s
less clear. While the evidence provided many ideas for im-
p r ovement, it lacked the detail that would enable teach e r s
to implement those ideas in their classrooms. We argued
that teachers needed “a variety of living examples of im-
plementation.”
T H E J O U R N E Y:
L E A R N I N G W I T H T E AC H E R S
Since 1998, we have planned and implemented sever-
al programs in which groups of teachers in England have
been supported in developing innova t ive practices in their
classrooms, drawing on the ideas in the original article.
While this effort has amply confirmed the original propos-
als, it has also added a wealth of new findings that are both
practical and authentic. Thus we are now confident that
we can set out sound advice for the improvement of class-
room assessment.
THE KMOFAP PROJECT
To carry out the exploratory work that was called for,
we needed to collaborate with a group of teachers will-
ing to take on the risks and extra work involved, and we
needed to secure support from their schools and districts.
Funding for the project was provided through the gener-
osity of the Nuffield Foundation, and we were fortunate
to find two school districts —Oxfordshire and Medway,
both in southern England — whose supervisory staff mem-
bers understood the issues and were willing to work with
us. Each district selected three secondary schools: Oxford-
shire chose three coeducational schools, and Medway ch o s e
one coeducational school, one boys’ school, and one girls’
s chool. Each school selected two science teachers and two
mathematics teachers. We discussed the plans with the
principal of each school, and then we called the first meet-
ing of the 24 teachers. So in January 1999, the King’s - M e d-
way-Oxfordshire Fo r m a t ive Assessment Project (KMOFA P )
was born.
Full details of the project can be found in our book, A s-
sessment for Learning: Putting It into Practice.3For the
present purpose, it is the outcomes that are important. The
findings presented here are based on the observations and
records of visits to classrooms by the King’s College team,
records of meetings of the whole group of teachers, inter-
views with and writing by the teachers themselves, and a
few discussions with student groups. Initially, we worked
with science and mathematics teachers, but the work has
been extended more recently to involve teachers of Eng-
lish in the same schools and teachers of other subjects in
other schools.
SPREADING THE WORD
Throughout the development of the project, we have
responded to numerous invitations to talk to other groups
of teachers and advisers. Indeed, over five years we have
made more than 400 such contributions. These have ra n g e d
across all subjects and across both primary and second-
ary phases. In addition, there has been sustained work with
some primary schools. All of this gives us confidence that
our general findings will be of value to all, although some
important details may differ for different age groups and
subjects. Furthermore, a group at Stanford University ob-
tained funding from the National Science Foundation to
set up a similar development project, in collaboration with
King’s, in schools in California. Extension of our own work
has been made possible by this funding. And we also ac-
knowledge support from individuals in several govern-
ment agencies who sat on the project’s steering group, of-
fered advice and guidance, and helped ensure that assess-
ment for learning (see “Assessment for Learning,” below)
is a central theme in education policy in England and Scot-
land.
THE LEARNING GAINS
From our review of the international research litera t u r e ,
ASSESSMENT FOR LEARNING
sAssessment for learning is any assessment for
wh i ch the first priority in its design and pra c t i c e
is to serve the purpose of promoting students’
learning. It thus differs from assessment de-
signed primarily to serve the purposes of ac-
c o u n t a b i l i t y, or of ranking, or of certifying com-
petence. An assessment activity can help learn-
ing if it provides information that teachers and
their students can use as feedback in assessing
t h e m s e l ves and one another and in modifying
the teaching and learning activities in wh i ch
they are engaged. Such assessment becomes
“ f o r m a t ive assessment” when the evidence is
actually used to adapt the teaching work to meet
learning needs.
SEPTEMBER 2004 11
we were convinced that enhanced formative assessment
would produce gains in student achievement, even when
measured in such narrow terms as scores on state-man-
dated tests. At the outset we were clear that it was impor-
tant to have some indication of the kinds of gains that could
be ach i e ved in real classrooms and over an extended peri-
od of time. Since each teacher in the project was free to
choose the class that would work on these ideas, we dis-
cussed with each teacher what data were available with-
in the school, and we set up a “mini-experiment” for each
teacher.
E a ch teacher decided what was to be the “output” meas-
ure for his or her class. For grade-10 classes, this was gen-
erally the grade achieved on the national school-leaving
examination taken when students are 16 (the General Cer-
tificate of Secondary Education or GCSE). For grade-8 class-
es, it was generally the score or level achieved on the na-
tional tests administered to all 14-ye a r-olds. For other classes,
a variety of measures were used, including end-of-module-
test scores and marks on the school’s end-of-year exami-
nations.
For each project class, the teacher identified a compar-
ison class. In some cases this was a parallel class taught
by the same teacher in previous years (and in one case in
the same year). In other cases, we used a parallel class
taught by a different teacher or, failing that, a nonparallel
class taught by the same or a different teach e r. When the
project and the control classes were not strictly parallel,
we controlled for possible differences in prior ach i e ve m e n t
by the use of “input” measures, such as school test scores
from the previous year or other measures of aptitude.
This approach meant that the size of the improve m e n t
was measured differently for each teacher. For example,
a grade-10 project class might outperform the compari-
son class by half a GCSE grade, but another teach e r ’s gra d e -
8 project class might outscore its control class by 7% on
an end-of-year exam. To enable us to aggregate the results,
we adopted the common measuring stick of the “standard-
ized effect size,” calculated by taking the difference be-
tween the scores of the experimental and control groups
and then dividing this number by the standard deviation
(a measure of the spread in the scores of the groups).
For the 19 teachers on whom we had complete data,
the ave rage effect size was around 0.3 standard deviations.
S u ch improvements, produced across a school, would ra i s e
a school in the lower quartile of the national performance
tables to well above ave rage. Thus it is clear that, far from
h aving to choose between teaching well and getting good
test scores, teachers can actually improve their students’
results by working with the ideas we present here.
HOW CHANGE CAN HAPPEN
We set out our main findings about classroom work un-
der four headings: questioning, feedback through gra d i n g ,
peer- and self-assessment, and the formative use of sum-
mative tests. Most of the quotations in the following pages
are taken directly from pieces written by the teachers. Th e
names of the teachers and of the schools are pseudony m s ,
in keeping with our policy of guaranteeing anonymity.
QUESTIONING
Many teachers do not plan and conduct classroom di-
alogue in ways that might help students to learn. Research
has shown that, after asking a question, many teachers wa i t
less than one second and then, if no answer is forthcom-
ing, ask another question or answer the question them-
selves.4A consequence of such short “wait time” is that
the only questions that “work” are those that can be an-
swered quick l y, without thought — that is, questions call-
ing for memorized facts. Consequently, the dialogue is at
a superficial level. As one teacher put it:
I ’d become dissatisfied with the closed Q & A style
that my unthinking teaching had fallen into, and I
would frequently be lazy in my acceptance of right
answers and sometimes even tacit complicity with
a class to make sure none of us had to work too
hard. . . . They and I knew that if the Q & A wasn’t
going smoothly, I’d change the question, answer it
myself, or only seek answers from the “brighter stu-
d e n t s .” There must have been times (still are?) wh e r e
an outside observer would see my lessons as a small
discussion group surrounded by many sleepy on-
lookers. — James, Two Bishops School
The key to changing such a situation is to allow longer
wait time. But many teachers find it hard to do this, for it
requires them to break their established habits. Once they
change, the expectations of their students are ch a l l e n g e d :
Increasing waiting time after asking questions prove d
difficult to start with due to my habitual desire to
“add” something almost immediately after asking the
Many teachers do not plan
and conduct classroom
dialogue in ways that might
help students to learn.
12 PHI DELTA KAPPAN
original question. The pause after asking the ques-
tion was sometimes “painful.” It felt unnatural to
have such a seemingly “dead” period, but I perse-
vered. Given more thinking time, students seemed
to realize that a more thoughtful answer was re-
quired. Now, after many months of changing my
style of questioning, I have noticed that most stu-
dents will give an answer and an explanation (wh e r e
necessary) without additional prompting. — D e r e k,
Century Island School
One teacher summarized the ove rall effects of her efforts
to improve the use of question-and-answer dialogue in the
classroom as follows:
Questioning
•My whole teaching style has become more in-
teractive. Instead of showing how to find solutions,
a question is asked and pupils are given time to ex-
plore answers together. My year 8 [grade 7] target
class is now well-used to this way of working. I find
myself using this method more and more with other
groups.
No hands
• Unless specifically asked, pupils know not to
put their hands up if they know the answer to a ques-
tion. All pupils are expected to be able to answer at
any time even if it is an “I don’t know.”
Supportive climate
• Pupils are comfortable with giving a wrong an-
swer. They know that these can be as useful as cor-
rect ones. They are happy for other pupils to help ex-
plore their wrong answers further. — N a n cy, Rive r-
side School
Increasing the wait time can help more students be-
come involved in discussions and increase the length of
their replies. Another way to broaden participation is to ask
students to brainstorm ideas, perhaps in pairs, for two to
three minutes before the teacher asks for contributions.
Overall, a consequence of such changes is that teachers
learn more about the students’ prior knowledge and about
any gaps and misconceptions in that knowledge, so that
teachers’ next moves can better address the learners’ real
needs.
To exploit such changes means moving away from the
routine of limited factual questions and refocusing atten-
tion on the quality and the different functions of classroom
questions. Consider, for example, the use of a “big ques-
tion”: an open question or a problem-solving task that can set
the scene for a lesson and evoke broad discussion or prompt
focused small-group discussions. How e ve r, if this stra t e g y
is to be productive, both the responses that the task might
g e n e rate and the ways of following up on these responses
h ave to be anticipated. Collaboration between teachers to
exchange ideas and experiences about good questions is
very valuable. The questions themselves then become a more
significant part of teaching, with attention focused on how
they can be constructed and used to explore and then de-
velop students’ learning. Here’s one teach e r ’s thinking on
the matter:
I chose a ye a r-8, middle-band group and really start-
ed to think about the type of questions I was asking
— were they just instant one-word answers — wh a t
were they testing — knowledge or understanding
— was I giving the class enough time to answer the
question, was I quickly accepting the correct answ e r,
was I asking the girl to explain her answ e r , how wa s
I dealing with a wrong answer? When I really stopped
to think, I realized that I could make a very large dif-
ference to the girls’ learning by using all their an-
swers to govern the pace and content of the lesson.
— Gwen, Waterford School
Effective questioning is also an important aspect of the
impromptu interventions teachers conduct once the stu-
dents are engaged in an activ i t y. Asking simple questions,
s u ch as “Why do you think that?” or “How might you ex-
press that?” can become part of the interactive dynamic
of the classroom and can provide an invaluable opportu-
nity to extend students’ thinking through immediate feed-
back on their work.
O ve rall, the main suggestions for action that have emerged
from the teachers’ experience are:
•More effort has to be spent in framing questions that
SEPTEMBER 2004 13
are worth asking, that is, questions that explore issues that are
critical to the development of students’ understanding.
• Wait time has to be increased to several seconds in
order to give students time to think, and everyone should
be expected to have an answer and to contribute to the dis-
cussion. Then all answers, right or wrong, can be used to
d e velop understanding. The aim is thoughtful improve m e n t
rather than getting it right the first time.
• Follow-up activities have to be rich, in that they cre-
ate opportunities to extend students’ understanding.
Put simply, the only point of asking questions is to ra i s e
issues about wh i ch a teacher needs information or about
wh i c h the students need to think. When such changes have
been made, experience demonstrates that students become
more active participants and come to realize that learning
may depend less on their capacity to spot the right answer
and more on their readiness to express and discuss their
own understanding. The teachers also shift in their role,
from presenters of content to leaders of an exploration and
d e velopment of ideas in wh i c h all students are invo l ve d .
FEEDBACK THROUGH GRADING
When giving students feedback on both oral and writ-
ten work, it is the nature, rather than the amount, of com-
mentary that is critical. Research experiments have estab-
lished that, while student learning can be advanced by feed-
b a ck through comments, the giving of numerical scores or
grades has a negative effect, in that students ignore com-
ments when marks are also give n . 5These results often sur-
prise teachers, but those who have abandoned the giv i n g
of marks discover that their experience confirms the find-
ings: students do engage more productively in improving
their work.
Many teachers will be concerned about the effect of
returning students’ work with comments but no scores or
grades. There may be conflicts with school policy:
My marking has developed from comments with tar-
gets and grades, which is the school policy, to com-
ments and targets only. Pupils do work on targets and
corrections more productively if no grades are give n .
Clare [Lee] observed on seve r al occasions how little
time pupils spent reading my comments if there were
g rades given as well. My routine is now, in my target
class, i) to not give grades, only comments; ii) to give
comments that highlight what has been done well
and what needs further work; and iii) to give the min-
imum f o l l ow-up work expected to be completed next
time I mark the books. — Nancy, Riverside School
Initial fears about how students might react turned out
to be unjustified, and neither parents nor school inspectors
h ave reacted adve r s e l y. Indeed, the provision of comments
to students helps parents to focus on the learning issues
rather than on trying to interpret a score or grade. We now
believe that the effort that many teachers devote to grad-
ing homework may be misdirected. A numerical score or
a grade does not tell students how to improve their wo r k ,
so an opportunity to enhance their learning is lost.
A commitment to improve comments requires more
work initially, as teachers have to attend to the quality of
the comments that they write on students’ work. Collab-
o ration between teachers in sharing examples of effective
comments can be very helpful, and experience will lead
to more fluency. There is, how e ve r, more invo l ved because
comments become useful feedback only if students use them
to guide further work, so new procedures are needed.
After the first INSET [inservice training meeting] I
was keen to try out a different way of marking books
to give pupils more constructive feedback. I was
keen to try and have a more easy method of moni-
toring pupils’ response to my comments without hav-
ing to trawl through their books each time to find
out if they’d addressed my comments. I implemented
a comment sheet at the back of my year-8 class’
books. It is A4 [letter] in size, and the left-hand side
is for my comments, and the right-hand side is for
the pupils to demonstrate by a reference to the page
in their books where I can find the evidence to say
whether they have done the work. . . . The comments
h ave become more meaningful as the time has gone
on, and the books still take me only one hour to mark.
— Sian, Cornbury Estate School
We have encountered a variety of ways of accommo-
dating the new emphasis on comments. Some teachers have
ceased assigning scores or grades at all, some teachers enter
scores in their own record books but do not write them in
the students’ books, others give a score or grade only after
a student has responded to the teach e r ’s comments. Some
t e a chers spend more time on certain pieces of work to en-
sure that they obtain good feedback and, to make time for
this, either do not mark some pieces, or look at only a third
of their students’ books each week, or invo l ve the students
in checking the straightforward tasks.
A particularly valuable method is to devote some les-
son time to rewriting selected pieces of work, so that em-
phasis can be put on feedback for improvement within a
s u p p o r t ive environment. This practice can change students’
expectations about the purposes of class work and home-
work.
As they tried to create useful feedback comments, many
of the project teachers realized that they needed to reassess
14 PHI DELTA KAPPAN
the work that they had asked students to undertake. Th e y
found that some tasks were useful in revealing students’
understandings and misunderstandings, while others fo-
cused mainly on conveying information. So some activ i t i e s
were eliminated, others modified, and new and better tasks
actively sought.
Overall the main ideas for improvement of feedback
can be summarized as follows:
•Written tasks, alongside oral questioning, should en-
courage students to develop and show understanding of
the key features of what they have learned.
•Comments should identify what has been done well
and what still needs improvement and give guidance on
how to make that improvement.
•Opportunities for students to respond to comments
should be planned as part of the overall learning process.
The central point here is that, to be effective, feedback
should cause thinking to take place. The implementation of
such reforms can change both teachers’ and students’ atti-
tudes toward written work: the assessment of students’ wo r k
will be seen less as a competitive and summative judgment
and more as a distinctive step in the process of learning.
PEER ASSESSMENT AND SELF-ASSESSMENT
Students can achieve a learning goal only if they un-
derstand that goal and can assess what they need to do to
reach it. So self-assessment is essential to learning.6Many
t e a chers who have tried to develop their students’ self-as-
sessment skills have found that the first and most difficult
task is to get students to think of their work in terms of a
set of goals. Insofar as they do so, they begin to develop
an overview of that work that allows them to manage and
control it for themselves. In other words, students are de-
veloping the capacity to work at a metacognitive level.
In practice, peer assessment turns out to be an important
complement to self-assessment. Peer assessment is unique-
ly valuable because students may accept criticisms of their
work from one another that they would not take seriously
if the remarks were offered by a teach e r. Peer work is also
valuable because the interchange will be in language that
students themselves naturally use and because students
learn by taking the roles of teachers and examiners of oth-
e r s .7One teacher shared her positive views of peer assess-
ment:
As well as assessing and marking (through discus-
sion and clear guidance) their own work, they also
assess and mark the work of others. This they do in
a very mature and sensible way, and this has prove d
to be a very worthwhile experiment. The students
k n ow that homework will be ch e cked by themselve s
or another girl in the class at the start of the next les-
son. This has led to a well-established routine and
only on extremely rare occasions have students failed
to complete the work set. They take pride in clear
and well-presented work that one of their peers may
be asked to mark. Any disagreement about the an-
swer is thoroughly and openly discussed until agree-
ment is reached. — Alice, Waterford School
The last sentence of this teach e r ’s comments brings out
an important point: when students do not understand an ex-
planation, they are likely to interrupt a fellow student wh e n
they would not interrupt a teacher. In addition to this ad-
vantage, peer assessment is also valuable in placing the
work in the hands of the students. The teacher can be free
to observe and reflect on what is happening and to fra m e
helpful interventions:
We regularly do peer marking — I find this very help-
ful indeed. A lot of misconceptions come to the fore,
and we then discuss these as we are going over the
h o m e work. I then go over the peer marking and talk
to pupils individually as I go round the room. —
Rose, Brownfields School
However, self-assessment will happen only if teachers
help their students, particularly the low achievers, to de-
velop the skill. This can take time and practice:
The kids are not skilled in what I am trying to get
them to do. I think the process is more effective long
term. If you invest time in it, it will pay off big div i-
dends, this process of getting the students to be more
independent in the way that they learn and to take the
responsibility themselves. —Tom, Riverside School
One simple and effective idea is for students to use “tra f-
fic light” icons, labeling their work green, ye l l ow, or red ac-
cording to whether they think they have good, partial, or
little understanding. These labels serve as a simple means
of communicating students’ self-assessments. Students may
then be asked to justify their judgments in a peer group,
thus linking peer assessment and self-assessment. This link-
age can help them develop the skills and the detachment
needed for effective self-assessment.
Another approach is to ask students first to use their
“traffic light” icons on a piece of work and then to indi-
cate by hands-up whether they put a green, ye l l ow, or red
icon on it. The teacher can then pair up the greens and the
ye l l o ws to help one another deal with their problems, wh i l e
the red students meet with the teacher as a group to deal
with their deeper problems. For such peer-group work to
SEPTEMBER 2004 15
succeed, many students will need guidance about how to
b e h ave in groups, including such skills as listening to one
another and taking turns.
In some subjects, taking time to help students under-
stand scoring rubrics is also very helpful. Students can be
g iven simplified versions of the rubrics teachers use, or they
can be encouraged to rewrite them or even to create their
own. Again, peer assessment and self-assessment are in-
timately linked. Observers in several language arts class-
rooms saw children apply to their own work lessons they
had learned in peer assessment. A frequently heard com-
ment was “I didn’t do that either” or “I need to do that
too.”
Students’ reflection about their understanding can also
be used to inform future teaching, and their feedback can
indicate in wh i ch areas a teacher needs to spend more time.
A useful guide is to ask students to “traffic light” an end-
of-unit test at the beginning of the unit: the ye l l ow and red
items can be used to adjust priorities within the teach i n g
plan. Our experience leads us to offer the following rec-
ommendations for improving classroom practice:
• The criteria for evaluating any learning ach i e ve m e n t s
must be made transparent to students to enable them to
have a clear overview both of the aims of their work and
of what it means to complete it successfully. Such criteria
m ay well be abstract, but concrete examples should be used
in modeling exercises to develop understanding.
• Students should be taught the habits and skills of col-
laboration in peer assessment, both because these are of
intrinsic value and because peer assessment can help de-
velop the objectivity required for effective self-assessment.
• Students should be encouraged to keep in mind the
aims of their work and to assess their own progress towa r d
meeting these aims as they proceed. Then they will be able
to guide their own work and so become independent
learners.
The main point here is that peer assessment and self-as-
sessment make distinct contributions to the development
of students’ learning. Indeed, they secure aims that cannot
be achieved in any other way.
THE FORMATIVE USE OF SUMMATIVE TESTS
The practices of self-assessment and peer assessment
can be applied to help students prepare for tests, as in tack-
ling the following problem:
[The students] did not mention any of the review-
ing strategies we had discussed in class. When ques-
tioned more closely, it was clear that many spent
their time using very passive revision [reviewing] tech-
niques. They would read over their work doing ve r y
little in the way of active revision or reviewing of their
work. They were not transferring the active learning
strategies we were using in class to work they did
at home. — Tom, Riverside School
To remedy this situation, students can be asked to “tra f-
fic light” a list of key words or the topics on wh i ch the test
will be set. The point of this exercise is to stimulate the stu-
dents to reflect on where they feel their learning is secure,
which they mark green, and where they need to concen-
t rate their efforts, in ye l l o w and red. These traffic lights then
form the basis of a review plan. Students can be asked to
identify questions on past tests that probe their “red” areas.
Then they can work with textbooks and in peer groups to
ensure that they can successfully answer those questions.
The aftermath of tests can also be an occasion for forma-
t ive work. Peer marking of test papers can be helpful, as with
normal written work, and it is particularly useful if students
are required first to formulate a scoring rubric — an exer-
cise that focuses attention on the criteria of quality rele-
vant to their productions. After peer marking, teachers can
r e s e r ve their time for discussion of the questions that give
widespread difficulty, while peer tutoring can tackle those
problems encountered by only a minority of students.
One other finding that has emerged from research studies
is that students trained to prepare for examinations by gen-
erating and then answering their own questions outper-
formed comparable groups who prepared in conve n t i o n a l
way s .8Preparing test questions helps students develop an
overview of the topic:
Pupils have had to think about what makes a good
question for a test and in doing so need to have a
clear understanding of the subject material. As a de-
velopment of this, the best questions have been used
for class tests. In this way, the pupils can see that their
work is valued, and I can make an assessment of the
progress made in these areas. When going over the
test, good use can be made of group work and dis-
cussions between students concentrating on specific
areas of concern. — A n g e l a, Cornbury Estate Sch o o l
D e velopments such as these challenge common expec-
tations. Some have argued that formative and summative
assessments are so different in their purpose that they have
to be kept apart, and such arguments are strengthened wh e n
one experiences the harmful influence that narrow, high-
stakes summative tests can have on teaching. How e ve r, it is
unrealistic to expect teachers and students to practice such
separation, so the challenge is to achieve a more positive
relationship between the two. All of the ways we have de-
16 PHI DELTA KAPPAN
scribed for doing so can be used for tests in wh i ch teach e r s
h ave control over the setting and the marking. But their ap-
plication m ay be more limited for tests in wh i ch the teach e r
has little or no control.
O ve rall, the main possibilities for improving classroom
practice by using summative tests for formative purposes
are as follows:
•Students can be engaged in a reflective review of the
work they have done to enable them to plan their revision
effectively.
• Students can be encouraged to set questions and mark
a n swers so as to gain an understanding of the assessment
process and further refine their efforts for improvement.
•Students should be encouraged through peer assess-
ment and self-assessment to apply criteria to help them un-
derstand how their work might be improved. This may in-
clude providing opportunities for students to rework exam-
ination answers in class.
The ove rall message is that summative tests should be-
come a positive part of the learning process. Through active
involvement in the testing process, students can see that
they can be the beneficiaries rather than the victims of test-
ing, because tests can help them improve their learning.
REFLECTIONS:
SOME UNDERLYING ISSUES
The changes that are entailed by improved assessment
for learning have provoked us and the teachers invo l ved to
reflect on deeper issues about learning and teaching.
LEARNING THEORY
One of the most surprising things that happened dur-
ing the early INSET sessions was that the participating teach-
ers asked us to run a session on the psychology of learning.
In retrospect, perhaps we should not have been so surprised
at this request. After all, we had stressed that feedback func-
tioned formatively only if the information fed back to the
learner was used by the learner in improving performance.
But while one can work out after the fact whether or not
a ny feedback has had the desired effect, what the teach e r s
needed was a way to give their students feedback that they
knew in advance was going to be useful. To do that they
needed to build up models of how students learn.
So the teachers came to take greater care in selecting
tasks, questions, and other prompts to ensure that students’
responses actually helped the teaching process. Such re-
sponses can “put on the table” the ideas that students bring
to a learning task. The key to effective learning is then to
find ways to help students restructure their knowledge to
build in new and more powerful ideas. In the KMOFAP
classrooms, as the teachers came to listen more attentive l y
to the students’ responses, they began to appreciate more
fully that learning was not a process of passive reception of
knowledge, but one in which the learners were active in
creating their own understandings. Put simply, it became
clear that, no matter what the pressure to ach i e ve good test
scores, learning must be done by the student.
Students came to understand what counted as good wo r k
through exemplification. Sometimes this was done through
focused whole-class discussion around a particular exam-
ple; at other times it was ach i e v ed through the use of sets
of criteria to assess the work of peers.
Engaging in peer assessment and self-assessment is much
more than just ch e cking for errors or weaknesses. It invo l ve s
making explicit what is normally implicit, and thus it re-
quires students to be active in their learning. As one stu-
dent wrote:
After a pupil marking my investigation, I can now
a ck n owledge my mistakes easier. I hope that it is not
just me who learned from the investigation but the
pupil who marked it did also. Next time I will have
to make my explanations clearer, as they said “It is
hard to understand.”. . . I will now explain my equa-
tion again so it is clear.
The students also became much more aware of when
they were learning and when they were not. One class,
wh i ch was subsequently taught by a teacher not empha-
sizing assessment for learning, surprised that teacher by
complaining: “Look, we’ve told you we don’t understand
this. Why are you going on to the next topic?” While stu-
dents who are in tune with their learning can create diffi-
culties for teachers, we believe that these are exactly the
kinds of problems we should want to have.
SUBJECT DIFFERENCES
From hearing about research and discussing ideas with
other colleagues, the teachers built up a repertoire of gener-
ic skills. They planned their questions, allowed appropriate
wait time, and gave feedback that was designed to cause
thinking. They ensured that students were given enough
time during lessons to evaluate their own work and that of
others.
H ow e ve r , after a while it became clear that these gener-
ic strategies could go only so far. Choosing a good ques-
tion requires a detailed knowledge of the subject, but not
necessarily the knowledge that is gained from advanced study
SEPTEMBER 2004 17
in a subject. A high level of qualification in a subject is less
important than a thorough understanding of its fundamental
principles, an understanding of the kinds of difficulties that
students might have, and the creativity to be able to think
up questions that stimulate productive thinking.9F u r t h e r-
more, such pedagogical content knowledge is essential in
interpreting responses. That is, what students say will con-
tain clues to aspects of their thinking that may require atten-
tion, but picking up on these clues requires a thorough know l -
edge of common difficulties in learning the subject. Th u s ,
while the general principles of formative assessment apply
across all subjects, the ways in wh i c h they manifest them-
s e l ves in different subjects may differ. We have encountered
s u ch differences in making comparisons between teach e r s
of mathematics, science, and language arts.
In mathematics, students have to learn to use valid pro-
cedures and to understand the concepts that underpin them.
Difficulties can arise when students learn strategies that
apply only in limited contexts and do not realize that they
are inadequate elsewhere. Questioning must then be de-
signed to bring out these strategies for discussion and to
explore problems in understanding the concepts so that stu-
dents can grasp the need to change their thinking. In such
learning, there is usually a well-defined correct outcome.
In more open-ended exercises, as in investigations of the
application of mathematical thinking to eve r y d ay problems,
there may be a variety of good solutions. Then an under-
standing of the criteria of quality is harder to achieve and
m ay require joint discussion of examples and of the abstra c t
criteria that they exemplify.
In science, the situation is very similar. There are many
features of the natural world for wh i ch science provides a
“correct” model or explanation. However, outside school,
many students acquire different ideas. For example, some
students come to believe that animals are living because
they move but that trees and flowers are not because they
d o n ’t. Or students may believe that astronauts seem almost
weightless on the moon because there is no air present.
M a ny of these “alternative conceptions” can be anticipated,
for they have been well documented. What has also been
documented is that the mere presentation of the “correct”
view has been shown to be ineffective. The task in such cases
is to open up discussion of such ideas and then provide
feedback that challenges them by introducing new pieces
of evidence and argument that support the scientific model.
There are other aspects for which an acceptable out-
come is less well defined. As in mathematics, open-end-
ed investigations call for different approaches to formative
assessment. Even more open are issues about social or eth-
ical implications of scientific ach i e vements, for there is no
“ a n sw e r .” Thus such work has to be “open” in a more fun-
damental way. Then the priority in giving feedback is to ch a l-
lenge students to tease out their assumptions and to help
them to be critical about the quality of any arguments.
Peer assessment and self-assessment have a long history
in language arts. Both the nature of the subject and the
open outcome of many of the tasks ch a racteristically make
s u ch practices central to one of the ove r all aims of the dis-
cipline, which is to enhance the critical judgment of the
students.
A second important function of peer assessment and
self-assessment was introduced by Royce Sadler, who ar-
gued that criteria alone are unhelpful in judging the quali-
ty of a piece of work or in guiding progression, because
there will always be too many variables.10 The key lies in
k n owing how to interpret the criteria in any particular case,
which involves “guild knowledge.”Teachers acquire this
knowledge through assessing student work, and it is this
process that allows them to differentiate between grades
and to gain a sense of how progress is ach i e ved. Peer assess-
ment and self-assessment provide similar opportunities for
students to be apprenticed into the guild, provided the cri-
teria of quality are clearly communicated.
In language arts, as in science and mathematics, atten-
tion needs to be paid to the central activities. Those that are
the most successful are those rich tasks that provide students
with an opportunity either to extend their understanding
of a concept within the text or to “scaffold” their ideas be-
fore writing. Chara c t e r i s t i c a l l y, these include small-group
and pair work, with the results often being fed back into a
whole-class discussion. Again, this type of work is not un-
common in language arts, the skill being to make the task
sufficiently structured to scaffold learning but not so tight-
ly defined as to limit thinking. Such activities not only pro-
vide students with a chance to develop their understand-
ing through talk, but they also provide the teacher with the
opportunity to give feedback during the course of a lesson
through further questioning and guidance. The better the
quality of the task, the better the quality of the interve n t i o n s .
The priority in giving
feedback is to challenge
students to tease out their
assumptions and to help
them be critical about the
quality of arguments.
18 PHI DELTA KAPPAN
Differences between learning tasks can be understood
in terms of a spectrum. At one end are “closed” tasks with
a single well-defined outcome; at the other are “open” tasks
with a wide range of acceptable outcomes. Tasks in language
a r t s — for example, the writing of a poem — are mainly at
the open end. But there are closed components even for
s u ch tasks — for example, the observance of gra m m a t i c a l
or genre conventions. Tasks in, say, mathematics are more
often closed, but applications of mathematics to eve r y d ay
problems can require open-ended evaluations. Thus, in va r y-
ing measure, the guidance needed for these two types of
learning work will be needed in all subjects.
Despite these differences, experience has shown that
the generic skills that have been developed do apply across
subjects. One of the project’s science teachers gave a talk
to the whole staff about his experiences using some of the
generic skills that we’ve been discussing and subsequently
found how such practices distributed themselves through-
out the disciplines:
Art and drama teachers do it all the time, so do tech-
nology teachers (something to do with open-ended
activities, long project times, and perhaps a less
c ramped curriculum?). But an English teacher came
up to me today and said, “Ye s t e r d ay afternoon wa s
fantastic. I tried it today with my year 8s, and it wo r k s .
No hands up, and giving them time to think. I had
fantastic responses from kids who have barely spoken
in class all year.They all wanted to say something,
and the quality of answers was brilliant. This is the
first time for ages that I’ve learnt something new
that’s going to make a real difference to my teach-
ing.” — James, Two Bishops School
MOTIVATION AND SELF-ESTEEM
Learning is not just a cognitive exercise: it invo l ves the
whole person. The need to motivate students is evident, but
it is often assumed that offering such extrinsic rewards as
g rades, gold stars, and prizes is the best way to do it. How-
e ve r , there is ample evidence to challenge this assumption.
Students will invest effort in a task only if they believe
that they can achieve something. If a learning exercise is
seen as a competition, then everyone is aware that there
will be losers as well as winners, and those who have a
t ra ck record as losers will see little point in trying. Thus the
problem is to motivate everyone, even though some are
bound to ach i e ve less than others. In tackling this problem,
the type of feedback given is very important. Many research
studies support this assertion. Here are a few examples:
•Students who are told that feedback “will help you
to learn” learn more than those who are told that “how
you do tells us how smart you are and what grades yo u ’ l l
get.” The difference is greatest for low achievers.11
• Students given feedback as marks are likely to see it
as a way to compare themselves with others (ego invo l ve-
ment); those given only comments see it as helping them
to improve (task invo l vement). The latter group outperforms
the former.12
• In a competitive system, low ach i e vers attribute their
performance to lack of “ability”; high ach i e vers, to their ef-
fort. In a task-oriented system, all attribute performance to
effort, and learning is improved, particularly among low
achievers.13
• A comprehensive review of research studies of feed-
back found that feedback improved performance in 60%
of the studies. In the cases where feedback was not help-
ful, the feedback turned out to be merely a judgment or
grade with no indication of how to improve.14
In general, feedback given as rewards or grades enhances
ego involvement rather than task involvement. It can fo-
cus students’ attention on their “ability” rather than on the
importance of effort, thus damaging the self-esteem of low
achievers and leading to problems of “learned helpless-
ness.”15 Feedback that focuses on what needs to be done
can encourage all to believe that they can improve. Such
feedback can enhance learning, both directly through the
effort that can ensue and indirectly by supporting the mo-
tivation to invest such effort.16
THE BIG IDEA:
FOCUS ON LEARNING
Our experiences in the project all point to the need to
rethink a teacher’s core aim: enhancing student learning.
To achieve this goal calls for a willingness to rethink the
planning of lessons, together with a readiness to change
Students given marks are
likely to see it as a way to
compare themselves with
others; those given only
comments see it as helping
them to improve. The latter
group outperforms the
former.
SEPTEMBER 2004 19
the roles that both teacher and students play in supporting
the learning process.
AL E A R N I N G E N V I R O N M E N T :
P R I N C I P L E S A N D P L A N S
I m p r ovement in classroom learning requires careful fore-
t h o u g h t .
Actually thinking about teaching has meant that I
have been able to come up with ideas and strate-
gies to cope with wh a t e ver has arisen and has con-
tributed greatly to my professional development. I
n ow think more about the content of the lesson. Th e
influence has shifted from “What am I going to teach
and what are the pupils going to do?” toward “How
am I going to teach this and what are the pupils go-
ing to learn?” — Susan, Waterford School
One purpose of a teach e r ’s forethought is to plan to im-
p r ove teaching actions. So, for example, the planning of ques-
tions and activities has to be in terms of their learning func-
tion.
I certainly did not spend sufficient time deve l o p i n g
questions prior to commencing my formative train-
ing. . . . Not until you analyze your own question-
ing do you realize how poor it can be. I found my-
self using questions to fill time and asking questions
wh i c h required little thought from the students. W h e n
talking to students, particularly those who are ex-
periencing difficulties, it is important to ask ques-
tions which get them thinking about the topic and
will allow them to make the next step in the learn-
ing process. — Derek, Century Island School
Of equal importance is concern for the quality of the re-
sponses that teachers make, whether in dialogue or in feed-
b a ck on written assignments. Effective feedback should make
more explicit to students what is invo l v ed in a high-quality
piece of work and what steps they need to take to improve .
At the same time, feedback can enhance students’ skills
and strategies for effective learning.
There is also a deeper issue here. A learning environ-
ment has to be “engineered” to involve students more ac-
tively in the learning tasks. The emphasis has to be on stu-
dents’ thinking and making that thinking public. As one
teacher put it:
There was a definite transition at some point, from
focusing on what I was putting into the process, to
what the students were contributing. It became ob-
vious that one way to make a significant sustainable
change was to get the students doing more of the
thinking. I then began to search for ways to make
the learning process more transparent to the stu-
dents. Indeed, I now spend my time looking for way s
to get students to take responsibility for their learn-
ing and at the same time making the learning more
collaborative. — Tom, Riverside School
Collaboration between teachers and students and be-
tween students and their peers can produce a supportive
e nvironment in wh i ch students can explore their own ideas,
hear alternative ideas in the language of their peers, and
evaluate them.
One technique has been to put the students into small
groups and give each student a small part of the unit
to explain to [his or her] colleagues. They are given
a few minutes’ preparation time, a few hints, and use
of their exercise books. Then each student explains
[his or her] chosen subject to the rest of the group.
Students are quick to point out such things as, “I
thought that the examples you chose were very good
as they were not ones in our books. I don’t think I
would have thought of those.” Or “I expected you
to mention particles more when you were explain-
ing the difference between liquids and gases.” Th e s e
sessions have proven invaluable — not only to me,
in being able to discover the level of understanding
of some students, but to the students too. — P h i l i p,
Century Island School
An additional advantage of such an environment is that
a teacher can work intensively with one group, challeng-
ing the ideas and assumptions of its members, knowing that
the rest of the class members are also working hard.
So the main actions to be taken to engineer an effec-
tive learning environment are:
• Plan classroom activities to give students the oppor-
tunity to express their thinking so that feedback can help
develop it;
• formulate feedback so that it guides improvement in
learning;
• use activities that demand collaboration so that eve r y-
one is included and challenged and train students to listen
to and respect one another’s ideas; and
• be sure that students are active participants in the les-
sons and emphasize that learning may depend less on their
capacity to spot the right answer and more on their readi-
ness to express and discuss their own understanding.
AL E A R N I N G E N V I R O N M E N T:
R O L E S A N D E X P E C TAT I O N S
It is one thing to plan new types of classroom activity
and quite another to put them into practice in ways that
are faithful to the aims they were developed to serve. Here
20 PHI DELTA KAPPAN
there are no recipes to follow in a uniform way. Inside the
Black Box was clear in stating that the effective develop-
ment of formative assessment would come about only if
“each teacher finds his or her own ways of incorporating
the lessons and ideas that are set out above into her or his
own patterns of classroom work.”
A second principle is that the learning environment en-
visaged requires a classroom culture that may well be un-
familiar and disconcerting for both teachers and students.
The effect of the innovations implemented by our teach-
ers was to change the “classroom contract” between the
teacher and the student —the rules that govern the be-
haviors that are expected and seen as legitimate by teach-
ers and students.
The students have to change from behaving as passive
recipients of the knowledge offered by the teacher to be-
coming active learners who can take responsibility for and
manage their own learning.
For the teachers, courage is necessary. One of the strik-
ing features of the project was that, in the early stages, many
participants described the new approach as “scary” be-
cause they felt they were going to lose control of their class-
es. Toward the end of the project, they spoke not of losing
control but of sharing responsibility for the students’ learn-
ing with the class —exactly the same process but viewed
from two very different perspectives. In one perspective,
the teachers and students are in a delivery/recipient relation-
ship; in the other, they are partners in pursuit of a shared
goal:
What formative assessment has done for me is made
me focus less on myself but more on the children.
I have had the confidence to empower the students
to take it forward. — Robert, Two Bishops School
What has been happening here is that eve r y b o dy ’s ex-
p e c t a t i o n s — that is, what teachers and students think that
being a teacher or being a student requires you to do —
have been altered. While it can seem daunting to under-
take such changes, they do not have to happen suddenly.
Changes with the KMOFAP teachers came slowly and stead-
ily, as experience developed and confidence grew in the
use of the various strategies for enriching feedback and in-
teraction. For example, many teachers started by using
questions to encourage thinking. Then they improved their
o ral and written feedback so that it brought thinking for-
ward and went on to develop peer and self-assessment.
To summarize, expectations and classroom culture can
be changed:
• by changing the “classroom contract” so that all ex-
pect that teacher and students work together for the same
end: the improvement of everyone’s learning;
• by empowering students to become active learners,
thus taking responsibility for their own learning;
• by incorporating the changes in the teach e r ’s role one
step at a time, as they seem appropriate; and
• by sustained attention to and reflection on ways in wh i ch
assessment can support learning.
WHAT YOU CAN DO
To incorporate some of the ideas about formative as-
sessment into your own practice, the first step is to reflect
on what you are now doing. Discussion with colleagues
and observation of one another’s lessons can help spark
such reflection.
A next step must be to try out changes. Wholesale ch a n g e
can be too risky and demanding, so it is often best to think
of one thing you feel confident to try — be it “traffic lights,”
peer assessment, improved questioning, whatever — and
simply try it. If you are a teacher in a middle school or high
s chool, try it with just one group. Or if you are an elemen-
tary teach e r, try it in just one subject area. We found that,
as teachers gained confidence in the power of allowing stu-
dents to say what they know and what they need to know,
the teachers decided that they should extend assessment
for learning to the whole of their teaching.
Taking on further strategies will then lead to further prog-
ress. When seve ral colleagues are collaborating, each starts
with a different strategy and then shares findings. This process
should lead to the explicit formulation of an “action plan,”
comprising a range of strategies to be used, in combina-
tion, preferably starting with a class at the beginning of the
s chool ye a r. The first reason to start at the beginning of the
year is so that there can be time to accustom both teach e r
and students to a new way of working. The second is that
it can be very difficult to change the established habits and
routines in the middle of a ye a r. The experience of a ye a r ’s
sustained work, with only a few classes, preferably along-
side similar efforts by colleagues, can provide a firm basis
for subsequent adoption of new practices on a wider scale.
Collaboration with a group trying out similar innova-
tions is almost essential. Mutual observation and the shar-
ing of ideas and experiences about the progress of action
plans can provide the necessary support both with the tac-
tics and at a strategic level. Support for colleagues is par-
ticularly important in overcoming those initial uncertain-
ties when engaging in the risky business of changing the
culture and expectations in the classroom.
As for any innovation, support from administrators is es-
sential. One way administrators can support change of this
SEPTEMBER 2004 21
kind is to help peer groups of teachers find time to meet
on a regular basis. Opportunities should also be found for
teachers to report to faculty and staff meetings.
The work of any group experimenting with innova t i o n s
is an investment for the whole school, so that support should
not be treated as indulgence for idiosyncratic practices. In-
deed, such work should be integrated into a school im-
p r ovement plan, with the expectation that the dissemina-
tion of fruitful practices will follow from the evaluation of
a group’s experiences.
At the same time, there may be a need to review cur-
rent school policies because such policies can actually con-
s t rain the use of formative assessment. A notable example
would be a policy that, by demanding that a score or gra d e
be given on every piece of homework, prevents the serious
use of comments. Five of the six schools in the KMOFA P
project have, following the experience of their science and
mathematics teachers, modified their policies to allow “com-
ment only” marking; for two of these, the modification wa s
that no scores or grades be given on homework through-
out the school. In another example, a “target setting” sys-
tem that required very frequent review was inhibiting any
change in learning methods that might slow down imme-
diate “progress” in order to produce medium- to long-term
gains in learning skills. Those engaged in innovations may
need formal exemption from such policies.
Thus support, evaluation, and subsequent dissemination
of innovation in assessment for learning will be planned
in a coherent way only if the responsibility for strategic ove r-
sight of the development is assigned to a member of the
s chool leadership team. Our experience supports the view
that to realize the promise of formative assessment by leav-
ing a few keen individuals to get on with it would be unfair
to them, while to do it by imposing a policy that requires
all teachers to immediately change their personal roles and
styles would be absurd.
What is needed is a plan, extending over at least three
years, in wh i ch a few small groups are supported for a two -
year exploration. These groups then form a nucleus of ex-
perience and expertise for disseminating their ideas through-
out the school and for supporting colleagues in making sim-
ilar explorations for themselves.
1. Paul Black and Dylan Wiliam, “Inside the Black Box: Raising Stan-
dards Through Classroom A s s e s s m e n t ,” Phi Delta Kappan, October 1998,
pp. 139-48. A version of this article has been published and widely sold
in the United Kingdom. A booklet, published in 2002, has also been wide-
ly distributed in the UK. It covers the same issues as the article and bears
the same title with the same authors. Both booklets, and further booklets
in this series, are published by NFER-NELSON.
Only a few references to the literature are given here. Further infor-
mation about publications and other resources can be obtained on the
King’s College website in the pages of the King’s Formative Assessment
Group. Some of the publications can be downloaded from this site:
www.Kel.ac.uk/education/research/Kal.html.
2. Paul Black and Dylan Wiliam, “Assessment and Classroom Learning,”
Assessment in Education, March 1998, pp. 7-71.
3 . Paul Black, Christine Harrison, Clare Lee, Bethan Marshall, and Dylan
Wiliam, Assessment for Learning: Putting It into Practice (Buckingham,
U.K.: Open University Press, 2003).
4 . Mary Budd Rowe, “Wait Time and Rewards as Instructional Va r i a b l e s ,
Their Influence on Language, Logic, and Fate Control,” Journal of Research
in Science Teaching, vol. 11, 1974, pp. 81-94.
5. Ruth Butler, “Enhancing and Undermining Intrinsic Motivation; The
Effects of Task-Involving and Ego-Involving Evaluation on Interest and
Performance,” British Journal of Educational Psychology, vol. 58, 1988,
pp. 1-14.
6. Royce Sadler, “Formative Assessment and the Design of Instructional
Systems,” Instructional Science, vol. 18, 1989, pp. 119-44.
7. Royce Sadler, “Formative Assessment: Revisiting the Territory,” As-
sessment in Education, vol. 5, 1998, pp. 77-84.
8. See, for example, Paul W. Foos, Joseph J. Mora, and Sharon Tkacz,
“Student Study Techniques and the Generation Effect,”Journal of Edu -
cational Psychology, vol. 86, 1994, pp. 567-76; and Alison King, “Fa-
cilitating Elabora t ive Learning Through Guided Student-Generated Ques-
t i o n i n g , ” Educational Psychologist, vol. 27, 1992, pp. 111-26.
9. See, for example, Mike Askew et al., Effective Teachers of Numeracy:
Final Report (London: King’s College London, School of Education, 1997).
In this study, there was no correlation between the progress made by ele-
mentary school students in arithmetic and the highest level of mathemat-
ics studied by the teacher. Indeed, there was a nonsignificant negative
correlation between the two. The students who made the most progress
were taught by teachers without high levels of subject knowledge, but
who emphasized the connections between mathematics concepts.
10. Sadler, “Formative Assessment and the Design of Instructional Sys-
tems.”
11. Richard S. Newman and Mahna T. Schwager, “Students’ Help Seek-
ing During Problem Solving: Effects of Grade, Goal, and Prior Achieve-
ment,” American Educational Research Journal, vol. 32, 1995, pp. 352-
76.
12. Ruth Butler, “Task-Involving and Ego-Involving Properties of Evalu-
ation: Effects of Different Feedback Conditions on Motivational Percep-
tions, Interest, and Pe r f o r m a n c e ,” Journal of Educational Psych o l o g y, vo l .
79, 1987, pp. 474-82.
13. Rhonda G. Craven, Herbert W. Marsh, and Raymond L. Debus, “Ef-
fects of Internally Focused Fe e d b a ck on Enhancement of Academic Self-
Concept,” Journal of Educational Psychology, vol. 83, 1991, pp. 17-27.
14. Avraham N. Kluger and Angelo DeNisi, “The Effects of Feedback In-
terventions on Performance: A Historical Review, a Meta-Analysis, and
a Preliminary Fe e d b a ck Intervention Th e o r y,” P s y chological Bulletin, vo l .
119, 1996, pp. 254-84.
15. Carol S. Dweck, “Motivational Processes Affecting Learning,”Amer-
ican Psychologist (Special Issue: Psychological Science and Education),
vol. 41, 1986, pp. 1040-48.
1 6 . Carol S. Dweck, S e l f - T heories: Their Role in Motivation, Pe r s o n a l i t y,
and Development (Philadelphia: Psychology Press, 2000).
K
There may be a need to review
school policies: an example
would be a policy that, by
demanding a grade on every
piece of homework, prevents
the serious use of comments.
Copyright Notice
Dylan Wiliam holds copyright to this article and has given Phi Delta Kappa
permission to distribute it. All further use must comply with U.S. law
governing fair use. MULTIPLE copies, in print and electronic formats, may
not be made or distributed without express permission from Phi Delta
Kappa International, Inc. All rights reserved.
Note that photographs, artwork, advertising, and other elements to which
Phi Delta Kappa does not hold copyright may have been removed from
these pages.
Please fax permission requests to the attention of KAPPAN Permissions
Editor at 812/339-0018 or e-mail permission requests to
kappan@pdkintl.org.
k0409bla.pdf
Paul Black, Christine Harrison, Clare Lee, Bethan Marshall, and
Dylan Wiliam, “Working Inside the Black Box: Assessment for
Learning in the Classroom,” Phi Delta Kappan, Vol. 86, No. 1,
September 2004, pp. 9-21.
File Name and Bibliographic Information
