Content uploaded by Dylan Wiliam
Author content
All content in this area was uploaded by Dylan Wiliam on Dec 31, 2022
Content may be subject to copyright.
Pedagogy, Culture & Society
Vol. 14, No. 1, March 2006, pp. 71–81
ISSN 1468-1366 (print)/ISSN 1747-5104 (online)/06/010071–11
© 2006 Pedagogy, Culture & Society
DOI: 10.1080/14681360500487470
The half-second delay: what follows?
Dylan Wiliam*
King’s College London, United Kingdom
Taylor and Francis LtdRPCS_A_148730.sgm10.1080/14681360500487470Pedagogy, Culture and Society1468-1366 (print)/1747-5104 (online)Original Article2006Taylor & Francis141000000Spring 2006DylanWiliamdylanwiliam@mac.com
There is an increasing body of evidence that only a minuscule proportion of the sensory data
processed by the unconscious mind (capable of processing approximately 11 million bits per
second) is referred to the conscious mind (capable of processing approximately 50 bits per second).
It is also clear that conscious awareness of stimuli from the environment lags behind actual percep-
tion by approximately half a second, but that a backward referral of subjective experience results in
a individual’s perception of the stimulus and its conscious awareness as simultaneous. These find-
ings challenge the primacy and supremacy of conscious processing of information on which a
substantial proportion of educational practice and policy is based, and suggest a re-evaluation of the
nature of teacher competence and expertise.
In search of brain-based education?
The above phrase is the title of an article by John Bruer, who argues that most of what
is known about the physiology of brains is inadequate to inform teaching and learning
(Bruer, 1999). He cites the example of brain lateralization, which has spawned a
number of teaching approaches that are designed to complement ‘left-brain’ and
‘right-brain’ approaches to learning. However, as he points out, the research that
shows that some activities are lateralized in one side of the brain or the other deals
with highly specialized activities, and all educationally significant activities require
both sides of the brain. While some of the ‘prescriptions’ claiming to emerge from
research on brain lateralization may have benefit, Bruer concludes that none are
currently grounded in evidence.
Similar conclusions follow from the research on learning styles. While some studies
claim to show that changing teaching to cater for individuals’ learning styles produces
improvements in learning, other studies have found that forcing learners to learn in a
style different from their preference can increase learning (Adey et al., 1999). The
message from this research is that teaching should cater for a range of learning styles;
but the same conclusion arises from most other theoretical stances on learning too.
*Learning and Teaching Research Center, ETS, Rosedale Road (MS 04-R), Princeton, NJ 08541,
USA. Email: dylanwiliam@mac.com
72 D. Wiliam
At the risk of adding one more item to the pile of unwarranted extrapolations from
cognitive science, in this article I want to explore the consequences of some little-
known but highly significant research findings that I think have profound implications
for the way that we think about education in general, and the process of teaching in
particular.1 The remainder of this and the next section of this article draw heavily on
a very readable account of the nature of consciousness written by Tor Nørretranders
(1998).
There is an increasing body of evidence that only a minuscule proportion of the
sensory data processed by the unconscious mind is referred to the conscious mind.
Table 1 is based on data from Zimmerman (1989), and shows that the bandwidth of
consciousness processing is far less than the bandwidth of the sensory inputs gener-
ated by the body’s sensory systems.
Now of course, it could be that most of the vast amount of information received by
the body’s sensory systems is simply ignored. In other words, all but the information
in conscious perception may be discarded by the brain, or may not even arrive at the
brain; for example, it could be that sensory data that is not the subject of ‘attention’
is not even processed. However, several strands of research evidence suggest that this
is not so.
For example, there is a substantial body of evidence that subliminal (literally ‘below
the threshold’) perception (i.e., perception that is not available to consciousness) can
be used in conscious processing. It is now well known that stimuli that are perceived
for too short a period to be consciously perceived (subliminal stimuli) can, neverthe-
less, ‘prime’ respondents in the way that they answer subsequent questions (Kihl-
strom, 1987). Further evidence for the important of sensations that are not perceived
consciously comes from descriptions of cases of patients with damage to certain parts
of the brain. Many of these patients are, to all intents and purposes, blind, in that they
are not aware of being able to see anything. Yet, in tests of visual perception, they
consistently score well above chance levels. This ‘blindsight’ can only be explained by
a level of visual processing that is not available to consciousness (Sacks, 1995).
It is also clear that this filtering process is largely, if not completely, determined by
previous experience. In the well-known optical illusion shown in Figure 1, generally
known after the German psychologist Franz Carl Müller-Lyer as the Müller-Lyer illu-
sion, the two vertical line segments are exactly the same length. However, the left-hand
vertical appears to be much shorter than the one on the right.
Table 1. Information flow in sensory systems and conscious perception
Sensory system Total bandwidth (in bits/second) Conscious bandwidth (in bits/second)
Eyes 10,000,000 40
Ears 100,000 30
Skin 1,000,000 5
Taste 1,000 1
Smell 100,000 1
The half-second delay 73
The most convincing explanation of this has been provided by Richard Gregory,
who suggests that the illusion is caused by the visual processing system’s interpreta-
tion of the left-hand figure as an external corner of an object, while that on the right
is seen as an internal corner (Gregory, 1966). Since the visual perception system
‘knows’ that objects that are further away are smaller, it compensates for this by creat-
ing a perception of the internal corner as larger, because it must be further away. As
Hundert (1995, p. 211) observes:
It is tempting to call this phenomenon of the accommodation of our plastic visual input
analyzers to the realities of depth perspective and size constancy in our world a ‘natural
phenomenon’—except for the simple fact that such rectilinear lines do not exist in nature!
(emphasis in original)
Indeed, it has been found that people raised in cultures that do not build rectilinear
structures do not experience the Müller-Lyer illusion (Deregowsky, 1974). Even
something as basic as what we see, therefore, is not ‘transparent’, but rather is filtered
by the unconscious processing that takes place prior to referral to consciousness.
Evidence that this filtering takes place after unconscious perception is provided by
a study relating to another well-known optical illusion (known as a size-contrast illu-
sion). The two central circles in Figure 2 are actually the same size, but the circle on
the left appears to be bigger, because of the presence of the smaller surrounding
circles. However, Aglioti et al. (1995) found that when the visual illusion was recre-
ated with three-dimensional plastic discs, while participants continued to see the
circle surrounded by small discs as smaller, a videotape of the participants reaching
to pick up the central disc showed exactly the same finger separation for the two situ-
ations. In the words of the title of Aglioti et al.’s article, these size-contrast illusions
‘deceive the eye but not the hand’, or, in terms of the present analysis, visual illusions
deceive the conscious, but not the unconscious mind.
Another interesting effect is observed if one holds both one’s hands in front of one’s
face, one hand being held twice as far away from the eyes as the other. The laws of
physics tell us that the nearer hand should be seen as twice as large (and as having
four times the area) of the further hand, but for most people, this is not what is observed.
Figure 1. The Müller-Lyer illusion
74 D. Wiliam
A final example of the role of unconscious processing in perception is provided by
the effect of ‘colour constancy’, whereby objects do not appear to change colour as
the lighting conditions vary, or as the object moves to a region where the ambient
lighting conditions are different. Again, the laws of physics tell us that the actual
wavelength of the light falling on the retina changes, but this is not perceived as a
change in colour. The unconscious mind’s theoretical model—that the object has not
changed colour—overrides the raw sense data and creates in the conscious mind the
phenomenon of colour constancy.
While our perception of the real world therefore appears as a transparent process,
the ‘physical’ model of sense data being referred to consciousness in a direct and
unproblematic way simply does not hold up:
We do not experience the world as raw data. When our consciousness experiences the world,
the unconscious discarding of sensory information has long since interpreted things for us.
What we experience has acquired meaning before we become conscious of it. (Nørretranders, 1998,
p. 187, original emphasis)
It is also important to note that the meaning that is attached to experience includes
affective content. Traditional models of affect posit that data are processed
consciously, and then referred for an affective reaction. In other words, first we decide
what we think, and then we decide how we feel about it. However, the evidence
provided by Damasio (1994) indicates that the real order of things is likely to be the
reverse of this. The unconscious processing that occurs prior to referral to conscious-
ness invests the sense data with affect before referral to consciousness. In other words,
Figure 2. An optical illusion that ‘deceives the eye but not the hand’
The half-second delay 75
what we feel about something tells us what we think. Further elaboration of these
ideas is beyond the scope of this article, but the interested reader is referred to Dama-
sio’s work, which provides a very readable introduction to recent research on affect
and reasoning.
Now, all this processing and filtering of sense data prior to referral to consciousness
must take some time, but when experimenters attempted to work out exactly how long
this took, the answer was so surprising that many people have found it hard to accept.
The half-second delay
In the 1960s and 1970s, two German neurophysiologists attempted to estimate
exactly how long it took for a decision to initiate a movement (for example, moving a
finger) to result in action. They found that any action, such as flexing a finger, is
presaged by an increase in electrical activity (what they termed a readiness potential),
which appears to correspond to the brain preparing to carry out an action by deter-
mining how it is to be done. What was surprising was that, on average, the increase
in readiness potential began 0.8 seconds before the finger was flexed. The question
that then arises is: when is the conscious decision to act actually made? The classical
answer would be that the conscious decision to act must occur at the same time as the
onset of the increase in readiness potential, but this would suggest that it took over
three-quarters of a second for this decision to result in action, which does not accord
with either our experience or our ‘common sense’. However, the alternative is even
less plausible, and that is that the increase in readiness potential precedes the decision
to act.
The matter was settled by a series of very elegant experiments reported by
Benjamin Libet in 1979 (Libet et al., 1979), in which participants were asked to flex
a finger whenever they felt like doing so. In order to answer the question as to the
sequence of events, it was necessary to get data on three events: when the increase in
readiness potential began, when the conscious decision was made, and when the
finger moved. The first and last of these were easy to determine. In order to get reli-
able data on when the conscious decision was made, the participants were asked to
look at a clock face with a single hand which made one revolution every 2.56 seconds.
Asking the participants to note the position of the clock hand when they decided to
flex the finger provided a reliable measure of when the decision was taken.
The results were highly consistent, and showed that the increase in readiness poten-
tial began 0.55 seconds before the act, but also, much more surprisingly, that the
conscious decision occurred 0.2 seconds before the act. In other words, the conscious
decision to act occurs 0.35 seconds after the body has started the mechanism to act!
Although this result is astonishing, and provoked much controversy (see Nørre-
tranders, 1998, pp. 220–250 for a discussion), Libet’s interpretation of these events
did actually explain experimental and other phenomena that had been observed. For
example, many people have had the sensation of looking at the second-hand of a clock
for more than a second, and yet the hand does not appear to move. The delay between
experience and conscious awareness seems the most likely explanation.
76 D. Wiliam
Libet’s explanation also explained some experimental results that Arthur Jensen
had collected in the 1960s (Libet, 1981). In collecting the reaction times of individ-
uals, which averaged 0.25 seconds, he suspected that some of the participants were
deliberately slowing down their reactions (because they distrusted what he would do
with the data). Jensen then asked the participants to increase their reaction times
gradually, but none of the participants could do this. As soon as they tried to slow
down their reactions at all, the reaction times increased to over half a second
(reported in Libet, 1981, pp. 185–186).
These and many similar experiments show that conscious awareness of stimuli
from the environment lags behind actual perception by up to half a second. Informa-
tion from different parts of the body about simultaneous events arrives at the brain at
different times, but the brain assembles these data so that an individual perceives the
stimulus and its conscious awareness as simultaneous. Michael Gazzaniga sums it up
thus:
Major events associated with mental processing go on, measurably so, in our brain before
we are aware of them. At the same time, these done deals do not leave us feeling we are
watching only a movie of our life. Because of temporal referral mechanisms, we believe we
are engaged in effecting these deals. (Gazzaniga, 1999, pp. 73–74)
Educational implications
Much (if not most) of our current practice in education is based on a simplistic model
of brain function: one in which data is received in a way very similar to that in which
it is transmitted and passed directly to consciousness, where more or less sense is
made of it. This criticism applies just as much to so-called ‘constructivist’ views of
learning as to more traditional models. Recently, there has been recognition that
implicit learning (learning in which you don’t know that you’re learning and you
don’t know what you’ve learnt) is an important element of learning, but much of this
research is relatively atheoretical, and rarely ventures beyond description.
The central feature of the argument I want to advance here is that traditional theo-
ries of learning will not serve, either as descriptions of what does happen, or of
prescriptions of what should happen. Acceptance of the significance of the neuro-
physiological evidence outlined above requires a radical reconceptualisation of both
learning and teaching.
Some important work has been done in this regard. Hare brain, tortoise mind by Guy
Claxton (1997) provides compelling evidence about the centrality of unconscious
processing in learning. In the remainder of this article I want to explore the relevance
of these experimental findings to two aspects of the activities of teachers: teachers’
cognitive activities in day-to-day teaching, and assessment.
Teachers’ cognitive activities
The failure of psychology to impact on the improvement of teaching competence in
general, and on teacher education specifically, has been lamented for many years. In
The half-second delay 77
his influential book The reflective practitioner, Donald Schön (1983) attributes this to
the pursuit of a project of technical rationality which he regards as epistemologically
inappropriate, in that technical rationality seeks to separate theory (based on explicit
knowledge—knowing that) from practice (often based on implicit knowledge—know-
ing how). Schön argues that competence is achieved not by reflection on action but by
reflection in action. However, as Tomlinson (no date) points out, technical rationality
may also fail because it is inadequate to illuminate practice. As Tomlinson says:
I think it would be fair to say that when education consulted the psychology cupboard some
decades ago, it didn’t find too much of direct relevance to the promoting of relatively
complex human learning, in pupils of varying ages, in typically complex social settings. (p. 3)
Compared with other areas of research in education, the cognitive activity of teachers
has received relatively little attention. Researchers acknowledge that a lesson deliv-
ered by an experienced teacher consists of a number of ‘scripts’ (Schank & Abelson,
1977), ‘frames’ (Minsky, 1975) or ‘images’ that are linked together. However, it is
common to regard the assembly of these scripts as an ‘automatic’ process:
Much of their instruction is accomplished though previously developed routines that mini-
mize the need for conscious decision making and that tend to be played out once they are
begun. Using these images and routines, teachers proceed on ‘automatic pilot’ so long as
events develop as expected. Their attention is focused on keeping the activity moving at a
good pace, managing the group, and monitoring the involvement of individuals. This is
accomplished mostly automatically using routines and heuristics so long as pupil response
remains within the expected and acceptable levels of tolerance. (Bromme & Brophy, 1986,
p. 116)
While this analysis portrays, I think accurately, the lack of conscious decision-making
in much of teachers’ day-to-day classroom activities, I want to suggest that regarding
the teacher’s ‘normal’ activity as one of being on ‘automatic pilot’ is unhelpful, and
underplays the significance of the unconscious cognitive processing that takes place
during this activity. To illustrate this, let us first consider an aspect of the nature of
expertise in playing chess.
Chase and Simon (1973) showed chessboards, on which were placed a number of
pieces, to an expert, an intermediate and a novice chess player for 5 seconds, after which
they were asked to reconstruct what they had seen (this process was repeated several
times, with different arrangements of pieces). The arrangements on the chessboards
were produced in two ways. Half of the chessboards were positions from actual games,
either in the middle of a game (with 24 to 26 pieces), or at the end of a game (12 to
15 pieces). The other positions were generated by taking the pieces from one of the
actual games and rearranging them randomly. The results are shown in Table 2. As
can be seen, experts were slightly worse than intermediates and novices when the pieces
were arranged randomly, but were four times better when the pieces were arranged
in a pattern from a real game. In their discussion of this result, Chase and Simon suggest
that the much better performance of experts with ‘real’ chessboards stems from an
ability to ‘read’ a chessboard in terms of a series of standard configurations of pieces
that they have learnt through their experience as chess players.
78 D. Wiliam
This is analogous to a much more prosaic task, in which we ask someone to memo-
rize a 7-letter string. The string ‘p-a-n-c-a-k-e’ is easier to memorize than ‘a-m-o-e-
b-a-e’, which in turn is easier to memorize than ‘j-k-q-x-z-q-g’, because the first two
can be processed as a whole (that is, as a ‘gestalt’), although the second probably also
requires some familiarity with conventions of Latin spelling. The third string can be
memorized by the conscious construction of a mnemonic, but that is likely to take
some time.
From this perspective, expertise is not the result of being able to deploy a small
number of highly general and abstract strategies but rather the ability to ‘read’ a situ-
ation in terms of a very large number of highly specific configurations. Simon (1980)
estimated that a chess grandmaster might have a repertoire of as many as 50,000
‘chunks’ of information to represent the configuration of pieces on a chessboard.
Another example of the context-sensitivity of expertise is provided by the mathe-
matical activity used in ‘Countdown’, a British television programme. The contestants
are given a set of numbers, and, by using the four arithmetic operations of addition,
subtraction, multiplication and division, are required to generate an arithmetical
expression to the value of a given target number. For example, the given numbers
might be 20, 11, 2, 5 and 3, with a target number of 105. There are many relatively
general strategies that can be developed for solving problems such as these, but the
most interesting feature of such problems for the purposes of this article is that expert
performance does not appear to use such general strategies. In fact most people use
surprisingly specialized strategies that only work with the specific constraints and
affordances (Gibson, 1979) of this particular problem. A typical approach relies on
working forwards from the givens and backwards from the goal at the same time. It is
important to note that this does not appear to be an iterative process, but rather, for
most people with a reasonable repertoire of number facts, the fact that 5 × 20 = 100,
which is close to the target number, seems to leap unbidden into consciousness.
Returning to the case of teaching, the research undertaken by Bennett et al. (1984)
found that teachers interpreted the information with which they were presented in
classrooms in the light of their past experiences, and these ‘filters’ proved relatively
difficult to modify. In terms of the arguments presented here, we can interpret this as
the result of unconscious processing of sense data prior to their referral to conscious
perception. The relative stability of the teachers’ filters even in the face of a teacher’s
conscious desire to focus on a particular aspect of behaviour which he or she wishes
to modify is not surprising, because the filter has done its work in advance of the refer-
ral of sense data to consciousness. Even with the best will in the world, if consciousness
Table 2. Performance in reconstructing actual and random chessboards (number of pieces placed
correctly)
Position Novices Intermediates Experts
Random 4 3.5 3
Actual 4 8 16
The half-second delay 79
does not actually get the data, it is impossible for it to act on them. In other words,
inexpert teachers were simply unable to ‘read’ their classrooms in the same way as an
expert might. Support for this interpretation comes from the work of David Berliner
and his colleagues (see, for example, Berliner et al., 1989), who found that expert
teachers had much richer interpretations of slides and videotapes of teaching situa-
tions, even though they could not say why they believed what they did. Perhaps most
interestingly, expert teachers found situations in which actors played the roles of
students much more disorienting than novice teachers, although they were unable to
articulate why. They appeared to rely on cues and signals that are processed uncon-
sciously, but which actors playing the role of students did not reproduce.
From this it appears that there are no ‘shortcuts’ to teaching expertise. Expertise in
teaching, as in most other areas, requires the development of the facility for uncon-
scious processing of richly detailed data, from which the relevant details are
abstracted and referred to consciousness. Furthermore, the pace of classroom activity
means that there is literally no time to think (at least consciously). If we are to effect
real and lasting improvements in teachers’ competence, we need a much more
detailed understanding of how different teachers, exposed to similar experiences,
develop different filters in their unconscious processing.
Assessment
In assessment, there is a considerable debate over the use of analytic versus holistic
marking schemes (Wiliam, 1996, 2000; Mabry, 1999). Concerns for inter-rater reli-
ability have driven educational policy-makers to implement assessment systems that
attempt to define quality in terms of explicit criteria for success. This certainly has the
effect of increasing the inter-rater reliability of the assessments (although relatively
little work has been done on estimating the effects on other sorts of reliability).
However, there is widespread agreement that in many domains, particularly those
that require creativity, and in the assessment of authentic performance, the imposi-
tion of such mark schemes compromises validity. Where there is more than one
correct answer, and where students can produce solutions not envisaged by the
authors of the mark scheme, high-quality responses can frequently be given low
scores (Wiliam, 1998).
In terms of modern validity theory, this would be regarded as a kind of construct
under-representation (Messick, 1989). Only some aspects of quality are represented
in the mark scheme, and other aspects are ignored. In terms of the arguments of this
article, however, we can interpret the mismatch between analytic and holistic scoring
as a distinction between conscious and unconscious processing.
It is fairly clear that teachers acquire notions of ‘standards’ much more effectively
when presented with actual samples of students’ work that exemplify the standards
being promulgated than when given criterion-based descriptions of the standards. In
the light of the research evidence presented above, I would like to suggest that the
explanation of this phenomenon is that the grading of pieces of work with respect to
a set of internal standards involves a far greater use of unconscious processing than
80 D. Wiliam
has previously been acknowledged. Models that rely on the use of ‘grade descriptions’
within consciousness have insufficient bandwidth to capture the complexity of the
processes involved. Ultimately, the perception of quality can never be an entirely
conscious process. As Robert Pirsig has argued:
Quality doesn’t have to be defined. You understand it without definition. Quality is a direct
experience independent of and prior to intellectual abstractions. (Pirsig, 1991, p. 64)
Again, a recognition and an understanding of the role of unconscious processing will
be necessary before we can begin to develop teachers’ ability to assess students’ work
reliably and fairly.
Conclusion
The education systems of most developed countries are predicated on the assumption
that the most valuable kind of knowledge is abstract, generalized and transcendent,
and this assumption has been supported by the findings of mainstream psychology.
Hyman Witkin, widely credited as the first to distinguish between field-dependence
and field-independence (Witkin, 1950), has gone so far as to assert that field-
independence—the ability to ignore contextual effects—is intelligence (Abelson,
1995, p. 182n). This has led to a search for broad general principles of expertise that
can be used to ‘short-cut’ the learning process.
Of course, general intellectual capability does clearly play a role in expertise of all
kinds, but such capability will not lead to expertise if the right kinds of data are not
available to conscious processing. The availability of these data requires a thorough
grounding in experience, which it appears is extraordinarily difficult to acquire
without prolonged immersion in the relevant settings.
In the context of the research findings presented here the idea of intuition remains
mysterious, but can be viewed as an exquisite and largely unconscious sensitivity to
very small details. In the words of Andy Clark (1997), expertise can only be developed
by ‘being there’.
Note
1. An earlier version of this article was presented at the annual conference of the European Educa-
tional Research Association, Lahti, Finland, September 1999.
References
Abelson, R. P. (1995) Statistics as principled argument (Hillsdale, NJ, Lawrence Erlbaum).
Adey, P. S., Fairbrother, R. W., Wiliam, D., Johnson, B. & Jones, C. (1999) A review of research
related to learning styles and strategies (London, King’s College London Centre for the Advance-
ment of Thinking).
Aglioti, S. A., DeSouza, J. & Goodale, M. (1995) Size contrast illusions deceived the eye but not
the hand, Current Biology, 5, 679–685.
Bennett, N., Desforges, C., Cockburn, A. & Wilkinson, B. (1984) The quality of pupil learning expe-
riences (London, Lawrence Erlbaum).
The half-second delay 81
Berliner, D. C., Stein, P., Saberrs, D., Brown Claridge, P., Cushing, K. & Pinnegar, S. (1989) Impli-
cations of research on pedagogical expertise and experience for mathematics teaching, in: D.
A. Grouws, T. J. Cooney, & D. Jones (Eds) Perspectives on research on effective mathematics teach-
ing (Reston, VA, National Council of Teachers of Mathematics/Lawrence Erlbaum), 67–95.
Bromme, R. & Brophy, J. (1986) Teachers’ cognitive activities, in: B. Christiansen, A. G. Howson,
& M. Otte (Eds) Perspectives on mathematics education (Dordrecht, Netherlands, Kluwer), 99–139.
Bruer, J. T. (1999) In search of … brain-based education, Phi Delta Kappan, 80(9), 648–657.
Chase, W. G. & Simon, H. A. (1973) Perception in chess, Cognitive Psychology, 4, 55–81.
Clark, A. (1997) Being there: putting brain, body and world together again (Cambridge, MA: Massa-
chusetts Institute of Technology Press).
Claxton, G. L. (1997) Hare brain, tortoise mind: why intelligence increases when you think less
(London, Fourth Estate).
Damasio, A. R. (1994) Descartes’ error: emotion, reason and the human brain (London, Picador).
Deregowsky, J. B. (1974) Illusion and culture, in: R. Gregory & E. H. Gombrich (Eds) Illusion in
nature and art (New York, Scribner).
Gazzaniga, M. S. (1999) The mind’s past (Los Angeles, CA, University of California Press).
Gibson, J. J. (1979) The ecological approach to visual perception (London, Houghton Mifflin).
Gregory, R. (1966) Eye and brain: the psychology of seeing (London, Weidenfeld and Nicholson).
Hundert, E. M. (1995) Lessons from an optical illusion (Cambridge, MA, Harvard University Press).
Kihlstrom, J. F. (1987) The cognitive unconscious, Science, 237, 1445–1452.
Libet, B., Wright Jr, E. W. & Pearl, D. K. (1979) Subjective referral of the timing for a conscious
sensory experience, Brain, 102, 193–224.
Libet, B. (1981) The experimental evidence for a subjective referral of a sensory experience back-
wards in time, Philosophy of Science, 48, 182–197.
Mabry, L. (1999) Writing to the rubric: lingering effects of traditional standardized testing on
direct writing assessment, Phi Delta Kappan, 80(9), 673–679.
Messick, S. (1989) Validity, in: R. L. Linn (Ed.) Educational measurement (Washington, DC:
American Council on Education/Macmillan), 13–103.
Minsky, M. (1975) A framework for representing knowledge, in: P. H. Winston (Ed.) The psychology
of computer vision (New York, McGraw-Hill), 211–277.
Nørretranders, T. (1998) The user illusion: cutting consciousness down to size (J. Sydenham, Trans.)
(London, Allen Lane/Penguin).
Pirsig, R. M. (1991) Lila: an inquiry into morals (New York, Bantam).
Sacks, O. (1995) An anthropologist on Mars (London, Picador).
Schank, R. C. & Abelson, R. P. (1977) Scripts, plans, goals and understanding: an enquiry into human
knowledge structures (Hillsdale, NJ, Lawrence Erlbaum).
Schön, D. (1983) The reflective practitioner: how professionals think in practice (New York, Basic Books).
Simon, H. A. (1980) Problem solving and education, in: D. T. Tuma & F. Reif (Eds) Problem
solving and education: issues in teaching and learning (Hillsdale, NJ, Lawrence Erlbaum), 81–96.
Tomlinson, P. (no date) Introductory paper, paper presented at British Psychological Society Confer-
ence on the Role of Psychology in Initial Teacher Training, Leeds, UK, University of Leeds School
of Education.
Wiliam, D. (1996) Standards in examinations: a matter of trust? Curriculum Journal, 7(3), 293–306.
Wiliam, D. (1998) What makes an investigation difficult? Journal of Mathematical Behaviour,
17(3), 329–353.
Wiliam, D. (2000) Standards: what are they, what do they do and where do they live?, in: R.
Moon, S. Brown & M. Ben-Peretz (Eds) Routledge international companion to education
(London, Routledge), 351–363.
Witkin, H. A. (1950) Individual differences in the ease of perception of embedded figures, Journal
of Personality, 19(1), 1–19.
Zimmerman, M. (1989) The nervous system in the context of information theory, in: R. F.
Schmidt & G. Thews (Eds) Human physiology (Berlin, Springer), 166–173.
