Exploring Learning Through Visual, Actional and Linguistic Communication: The multimodal environment of a science classroom

Abstract

This paper draws on ideas and data from the ESRC funded research 'Rhetorics of the science classroom: a multimodal approach'. It suggests that learning is realised through the interaction between visual, actional and linguistic communication (i.e. learning is multimodal) and involves the transformation of information across different communicative systems ('modes'), e.g. from speech to image. It demonstrates that learning is a process of selection, adaptation and transformation motivated by the interests of pupils and the context of learning. Specifically, it analyses the different ways four Year 7 science pupils transformed their teacher's descriptions of cells in their production of texts on onion cells (and the choices they engaged with in producing them), focusing in particular on their use of analogy.

Full text

Educational Review, Vol. 53, No. 1, 2001
Exploring Learning Through Visual,
Actional and Linguistic Communication:
the multimodal environment of a science
classroom
CAREY JEWITT, GUNTHER KRESS, JON OGBORN & CHARALAMPOS
TSATSARELIS, Institute of Education , London University, London, UK
ABSTRACT This paper draws on ideas and data from the ESRC funded research
‘Rhetorics of the science classroom: a multimodal approach’. It suggests that
learning is realised through the interaction between visual, actiona l and linguistic
communication (i.e. learning is multimodal) and involves the transformation of
information across different communicative systems (‘modes’), e.g. from speech to
image. It demonstrates that learning is a process of selection, adaptation and
transformation motivated by the interests of pupils and the context of learning.
Speci cally, it analyses the different ways four Year 7 science pupils transformed
their teacher’s descriptions of cells in their production of texts on onion cells (and
the choices they engaged with in producing them), focusing in particular on their use
of analogy.
Introduction
Previous educational research has tended to focus on classroom talk and written
texts, re ecting the dominant view of learning as a primarily linguistic accomplish-
ment. In contrast, our research has explored the full repertoire of meaning-making
resources which pupils and teachers brought to the classroom (actional, visual and
linguistic resources) and how these were ‘organised’ to make meaning, i.e. we took
a ‘multimodal’ approach to classroom interaction (Kress et al., 2001) [1]. Extending
communication to refer to all meaning-making systems, what we term ‘mode’
(organised, regular, socially speci c means of representation), including language as
speech or as writing, highlights the need to consider how teachers and learners use
a range of modes to express meanings in the classroom. A multimodal perspective,
as this paper will demonstrate, brings visual expressions of learning into focus and
enables research to ‘recognise’ the central role of action in the learning of science.
This paper tentatively concludes that a multimodal perspective offers some new
messages for the practice of science teaching.
ISSN 0013–1911 print; 1465–3397 online/01/ 010005-1 4 Ó2001 Educational Review
DOI: 10.1080/0013191012003360 0

6C. Jewitt et al.
Learning as a Process of ‘Sign Making’
We view learning as a process in which pupils are involved in actively ‘remaking’
the information and messages (or complexes of ‘signs’) which teachers communicate
in the classroom. In this way learning can be seen as the pupils’ ‘reshaping’ of
meaning (‘signs’) to create new meanings (‘signs’). This conception of learning as
a dynamic process of sign making is informed by social semiotics (Halliday, 1985).
The  eld of social semiotics is primarily concerned with communication as an
inherently social phenomenon (Hodge & Kress, 1988). The conceptual framework of
social semiotics accounts for how modes of communication (speech, writing, image,
gesture, three-dimensional models, etc.) are used. This account is based on the
assumption that communicative systems have evolved to satisfy societal needs and
that modes such as writing or gesture are organised to function with respect to these
needs, e.g. language, in Michael Halliday’s phrase, is as it is because of its social
uses.
The organising concept of a social semiotic approach is that meaning arises as a
consequence of choice and that meaning is multiple. So, for instance, if I dress
formally rather than informally when meeting someone, I make a choice and that
choice expresses a meaning I wish to make about that meeting. When we make
meaning we have and exercise choices in simultaneously constructing a presentation
of something, orienting it to others and in doing so we create an organised structure
of related elements (Lemke, 1998). In formal ‘grammars’ these three types of
meaning functions are modelled as sets of features which represent the work which
is required (the choices available) when encoding or decoding a text. In the
innocuous greeting ‘lovely day we’re having’ I am presenting information in a
fashion relevant for my neighbour, in an utterance that  ts unobtrusively into its
particular environment.
Recently, this approach has been elaborated to take account of modes and systems
of making meaning other than language, including music/sound (van Leeuwen,
1999), action (Martinec, 1996), visual communication (Kress & van Leeuwen, 1996;
O’Toole, 1994). Most recently, work in Hallidayan functional linguistics (Martin &
Veel, 1998) and in social semiotics has increasingly viewed the social production of
systems of signs and meanings as a ‘multimodal’ phenomenon, i.e. there is now an
increasing understandin g that occasions of communication always draw on a multi-
plicity of modes of communication at the same time. When we speak we also make
facial expressions, we gesture, stand at a certain distance and so on, all of which
make meaning together. This ensemble of modes we regard as the normal condition
of communication and we refer to that as multimodal communication or as multimo-
dality.
Within this theoretical perspective we can look newly at what happens in
classrooms. Importantly, pupils’ remaking of a teacher’s message to create new signs
can be seen as the process of learning. We regard what the teacher says or does as
a complex of ‘signs’: a meaningful complex of form and content seen as a motivated
whole. This ensemble of communicative actions constitutes many of the resources
(and imposes some of the constraints) involved in pupils’ production of texts in the
classroom. These representational resources are, to some extent, also constrained by
‘how the world is’, i.e. phenomena, objects and events in the world, as well as the
teacher, play an important role in mediating the meanings that learners produce. That
is, from the complex of signs, made in a number of modes, the pupils make their

7A Multimodal Environment
selections, re ecting their ‘interest’ in the complex message produced by the teacher.
Their selection is integrated into existing ‘internal complexes’ and forms the basis
for their making of new texts. In this way, pupils’ texts can be read as ‘signs’ which
mediate their responses to the teacher’s message: one kind of evidence of what
thinking may have been like, i.e. we see learning as a motivated activity in which the
interest of the pupil is expressed through their making of apt new signs in a given
context (Kress, 1997). We analyse pupils’ responses in terms of the new signs (new
meanings) they are making out of the meanings made by teachers’ texts utilised in
the classroom, i.e. we treat teacher and pupils on a par, both producing new signs in
response to the signs of the other, re ecting their different interests.
Approaching sign making as inherently involved in the dynamic process of
learning, as a complex process in which teachers shape ideas to be learned through
a plurality of communicative means (linguistic, visual, through the materiality of
three-dimensional models, and through actional modes) in order to make these ideas
appropriate and convincin g to pupils, ‘re-opens’ the question of why pupils’ texts
vary so much from each other. Today, viewing the learning process as the trans-
mission of knowledge from teacher to pupil is less dominant amongst educational
practitioners and researchers. Yet despite this, the common response to differences
between pupils’ texts is taken as an indication of pupils’ failure to correctly read (or
reproduce) the stable messages encoded in teacher’s communications and is under-
pinned by this conception of learning. In contrast, we suggest that an alternative way
of understanding the variation between pupils’ texts is to read ‘variation’ as the
pupils’ shaping of meaning in what is for them the most apt and plausible way,
re ecting their interests, with the resources available to them in a speci c context.
The focus of this paper is how pupils transform the structural and content aspects
of a teacher’s communication through their use of a range of modes, their selection
and adaptation of elements presented and their introduction of new elements. In other
words, the focus is on how pupils use the resources made available to them in the
classroom, from the teacher and from other sources (e.g. other lessons, from the
television, their experiences and interests outside school) to construct meanings.
The analysis presented in this paper is based on: (1) observing the pupils’
production of the texts; (2) interviews with the pupils about their experience of
conducting the investigation and producing the texts; (3) the texts themselves. In
order to ‘get at’ the cognitive processes of pupils’ learning we focus on the texts as
an ‘outcome’ of this process (in the form of ‘complexes of signs’). We suggest that
pupils’ texts can be viewed as ‘one kind of evidence’ of the cognitive processes that
they have engaged in: the effect of the teacher’s communication at that particular
moment on individual pupils. We analyse pupils’ texts in terms of ‘what is there’.
This requires a rigorous and serious look at what pupils have actually produced: as
writing, looking at the genres apparent in the writing; as image, the use of colour,
the kinds of realism deployed in the images; in the combination of all these together.
We take ‘what is there’ to be ‘there’ as the consequence of choices made by the pupil
from among the resources made available by the teacher in preceding teaching, but
also resources taken from talk with friends in class, from other experiences in school
and from experiences and ‘knowledge’ out of school, aspects explored in the
interviews with the pupils. ‘What is there’ is therefore a selection, a recon guration,
a reshaping, the result of an active, complex process of transformation. In addition,
we pay attention to the consequences, possibilities and limitations of the representa-

8C. Jewitt et al.
tional ‘design’ of the new text, to its rhetorical arrangement of mode and elements,
as an expression of the intentions of the pupil as rhetor.
The purpose of analysing pupils’ texts as the product of the action of ‘making a
new meaning’ is not to declare them to be correct or legitimate, incorrect or
illegitimate, rather it is to explore the signi cance in the characteristics of pupils’
transformative actions.
Multimodal Learning
The lesson discussed in this paper centred on looking at the cells of an onion under
the microscope. The pupils’ produced a text which recorded visually ‘what they saw’
and in written form ‘what they did’. The resources made available to the pupils to
do this included: the teacher’s talk, in particular his use of two analogie s for cells ‘a
building block’ and ‘a honeycomb’ which served as a ‘template’ for the pupils’
transformations; worksheets to help them in their task, which also served as models
of scienti c texts; the materials and equipment for the experiment (microscope,
slides and onions); the pupils’ talk with one another around the microscope as they
engaged with the task.
At the beginning of the lesson the teacher stood in front of the pupils and exhibited
the scalpel, slide, iodine and microscope required to transform an ordinary everyday
thing, an onion, into an extraordinary unknown thing, a collection of cells. He
described through pantomime action, manipulation of the microscope and his speech
the process of the experiment. In summary, he performed the placing of a section of
the onion skin on a slide, he pantomimed adding iodine to stain the cells, he acted
out covering the slide with another slide cover, he pantomimed placing it under the
microscope, he mimed adjusting the microscope and again through mime he per-
formed ‘looking through the lenses’ of the microscope. His careful handling of the
microscope and his naming of its parts, together with his comments on the great cost
of the equipment, served to provide it with authority and to separate it from the
microscopes they might have at home (‘this is not from Toys R Us’). The teacher’s
demonstration of the process of the experiment also gave authoritative meaning to
the process.
The demeanour of the teacher’s body, of what Bourdieu termed habitus (Bourdieu,
1991), when using the microscope embodied scienti c historical traditions and
knowledge, indicate d respect for the equipment, showed an understanding of the
effect of light on the mirrors and embodied ‘observation’: doing ‘being a scientist’.
The pupils made their sense of the science teacher’s habitus through imitation and
transformation.
The experiment involved the pupils and teacher in interacting with both everyday
and scienti c objects. Experiment realised the rhetorical function of constructing
science as exploration and constructing the pupils as ‘scientists’ through handling of
the equipment (e.g. microscope): the habitus of science. Experiment also rhetorically
transformed the everyday into the realm of the scienti c (e.g. the transformation of
‘onion’ into ‘collection of cells’). Experiment was used to construct the entity ‘cell’
through empirical evidence and the rhetorical stance ‘see/do it for yourself’. In this
way experiment realised and construed a realist approach to the nature of science.
During the lesson the teacher gave direct instructions on three areas of making the
text: composition and labelling; standard ways of drawing in science; ways of
reporting an experiment. He requested the pupils to divide the page in their exercise

9A Multimodal Environment
FIG. 1. The overhead projector image used by the teacher.
books into two halves and to draw the image of the onion cells in the bottom half
of the page and later to write ‘what I did’ in the top half of the page. The relationship
of the image to the writing as set out in the teacher’s instructions suggests that he
intended the image to be read as the actualisation of the writing: the result of the
experiment. The switch in mode from image to writing served to further mark a clear
boundary between the events of ‘seeing’ and ‘doing’, i.e. the compositional structure
encoded process and result as two discrete categories. When the teacher’s instruc-
tions on composition were followed the chronologica l order of the pupils’ creation
of the texts was reversed (i.e.  rst they drew and then they wrote) but the temporal
order of action in the lesson was re ected.
The teacher introduced the pupils to the generic convention s of producin g
scienti c drawings, namely line pencil drawings without the use of colour and
drawing from direct observation. The teacher showed the pupils a photocopy of a
generalised image of the pattern of onion cells magni ed 400 3(Fig. 1) on the
overhead projector to con rm what the pupils should look for and reproduce. The
darkness of the photocopy obliterated much of the detail and reduced the detail of the
representation of the pattern made by the interconnecting cell walls. This image
functioned to highlight what the teacher wanted the pupils to look at: the pattern of
the cells.
The four pupils who produced Texts 1–4 (Figs 2–5) participated in the same
lesson: they had the same materials to work with. Yet, as a quick look at the texts
shows, they produced four distinctly different texts. The setting up of a slide
containing onion cells is a skill that needs to be developed and it is doubtful that year
7 pupils doing it for the  rst time will be easily able to produce one without air
bubbles or multilayers of cells, it has, therefore been suggested that these differences
are due to the fact that what they are looking at may be markedly different. It has
also been suggested that these differences are the the result of individual readings of
the events in the lesson, a consequence of the pupils’ lack of attention or their poor
writing and drawing skills. Whilst these explanations may go some way to explain
some of the differences between the texts we suggest that they do not provide a

10 C. Jewitt et al.
convincing explanation of the pupils’ production of such distinctly different texts
(particularly as Texts 1 and 2 were produced by two pupils looking at the same slide
through the same microscope). An alternative explanation for the differences be-
tween the texts, we suggest, is to see them as expression s of the different interests
of the pupils and their transformation of the resources made available to them via the
teacher. Transformations, for example, of layout and composition, representation (i.e.
standard ways of drawing in science) and genres of reporting (expressions of
‘scienti cness’).
In the following section we analyse the ways four of the pupils in the lesson
transformed their teacher’s descriptions of cells in their texts on onion cells (and the
choices they engaged with in producing them). Speci cally, we focus on the pupils’
transformation of the teacher’s analogies for cells in order to explore the role of
multimodality in the process of learning, show the transduction of information across
modes within learning and suggest that this multimodal transformation maybe
motivated by the interests of pupils. The act of analysis within our approach is itself
a reading of signs (even when supported by interviews with the makers of those
signs) and we acknowledge that our interests and experiences are present in our
analysis. We agree with the comments of Hodge and Kress in Social Semiotics:
The interpretation of texts is always a matter of guesses, not facts. But
some guesses are richer and/or more plausible than others. A transforma-
tional reading of a text is often hypothetica l to some extent, but this is by
no means a reason why semioticians should avoid attempting it. (Hodge &
Kress, 1988, p. 168)
Pupils’ Texts as Transformations
The teacher introduced the notion of cells using the verbal metaphor ‘building
blocks’. The metaphor of cells as ‘building blocks’ enabled the pupils to begin to
visualise the physical characteristics and properties of something usually invisible to
them, building blocks. Building blocks, and therefore perhaps cells, are rectangular
and have clear boundaries; probably they combine with each other to make bigger
parts, to make a whole. The teacher used the analogy ‘a honeycomb arrangement’ to
guide the pupils’ experience of looking through the microscope at the onion cells, to
help them see ‘what you are actually looking for’. The teacher’s analogy and later
use of images (see Fig. 1) provided the pupils with an ‘image’ of what to look for
before they saw the cells through the microscope, a pattern to act as a  lter between
their eyes and the slide with the onion epidermis.
This  lter served to let the pupils know what it was that the teacher wanted them
to record. Potentially, this raised a dif culty for pupils; what if what they saw did not
look like what the teacher wanted them to see? What should they draw? The tension
between the rhetorical positioning of pupils in relation to the experiment as revealing
‘bedrock reality’ and involving personal experience created a gap between teacher
expectation and pupils’ experience. It is in these gaps that variation grows.
Below we focus on two of the analogies used by the pupils, ‘a brick wall’ and ‘a
wavy weave’, and how these analogies constructed the entity ‘cell’ in ways which
transformed the teacher’s meaning and extended their learning.

11A Multimodal Environment
It Looks Like a Brick Wall
As Pupil A looked through the microscope in the lesson she said: ‘it looks like a
brick wall’. This analogy is also apparent in her drawing of cells (see Fig. 2).
She applied the analogy of a brick wall which was directly suggested in the
worksheet and implied in the teacher’s verbal analogy with building blocks and
reproduced it by analogy in a different mode (image), i.e. Pupil A selected the
rhetorical device used by the teacher and transformed it from a verbal to a visual
analogy. Her visual analogy focused on the positive elements of regularity and
uniformity of cells and visually embodied the relationship of the part (the cell or
brick) to the whole (the onion or the brick wall). A brick wall is a familiar thing in
an urban environment and the familiarity implied by the pupil’s analogy comments
on its everydayness: cells are everywhere.
Pupil A and Pupil B worked together in the lesson and they are friends. They
looked at the same piece of onion through the same microscope. However, the
comparison of their drawings (see Figs 2 and 3) shows that they are markedly
different.
The strict rectangular shape of bricks is treated as a negative element of the
analogy and ignored by pupil B in her image, while pupil A treated it as a positive
element and included it (including the horizontal orientation of bricks) in the
construction of her image. Pupils A and B incorporate the notion of irregularity in
their texts, for instance Pupil B says: ‘This is different it’s got like cracks in it’.
Cement and cracks in a brick wall are irregularities in an otherwise set pattern and
the pupils’ elaboration of the brick wall analogy in this way enables the analogy to
account for the air bubbles in their images and in doing so marks them as
irregularities. The possibility of irregularity enables Pupil B to share the brick wall
analogy as she can now use it to explain the irregularity she observed. The analogy
of the brick wall was selected and adapted to dry stone wall by Pupil C. Again, we
suggest that the use of analogy by Pupil C is apparent in her image (see Fig. 4).
Here Pupil C transforms the context of the analogy from London to the country-
side. She substitutes a commonplace wall in urban London for a rare type of wall in
the countryside. If, as we suggested earlier, all signs are motivated, in other words
the pupil chose to represent and describe cells as ‘like a dry stone wall’ as opposed
to a brick wall or a wavy weave, what are we to make of her selection? We suggest ,
that the change in context and speci cation by Pupil C provides a more elite and
distanced reference to cells (to those living in an urban environment). We speculate
that her use of analogy suggests cells are well out of most pupils’ everyday
experience: you must have a specialist knowledge to be able to see them, you have
to travel (metaphorically) to view it, it is outside the common urban experience: it
is exotic. The context of the dry stone wall analogy, the countryside and the
substitution of manufactured bricks with naturally formed stones suggests a link
between cells and nature and introduced a sense of dynamism and potential for
change through decay.
In summary, Pupil A’s original analogy for a cell of a brick wall suggest s the
qualities of regularity, uniformity and structure, which are positive properties of the
verbal analogy introduced by the teacher and transformed into a visual analogy.
Pupils A and B incorporated the possibility of irregularity and the analogy was
adapted further by Pupil C to introduce the notion of nature and specialism. The
extension of the analogy happened through exploration of the neutral elements of

12 C. Jewitt et al.
FIG. 2. Pupil A’s text.

13A Multimodal Environment
FIG. 3. Pupil B’s text.
analogy, which were  nally treated as positive features and facilitated a better  t
between the analogy brick walls and cells.
More Like a Wavy Weave
Another analogy introduced and used by pupils was that of a ‘wavy weave’. This
verbal analogy used by Pupil D is present in her visual realisation of the cells (see
Fig. 5).
The analogy of a cell with a wavy weave again draws on the qualities of regularity
and uniformity and presents cells as a part of something bigger, something visible.
However, the scale of the analogy introduces something else; it presents cells as
something less immediately visible than a brick; cells are not an obvious part of
something. Pupil D’s wavy weave analogy (and the other analogies she generated
during our interview with her, ‘a cotton weave like a sheet’, ‘like skin’) conceives
of cells as organic and dynamic in comparison with Pupil A’s static analogy of cells

14 C. Jewitt et al.
FIG. 4. Pupil C’s text.
as bricks. This difference is highlighted by comparison of the lines of the drawing
in Fig. 5, which are noticeably thinner and more  uid, with those in Fig. 2. We
suggest that the construction of the wavy weave analogy is that it is related to the
image of cells shown in the overhead projector by the teacher (see Fig. 1). The
overhead projector image could be read by some pupils as an image of interconnect-
edness. Weave can be thought of as a sheet of interconnectedness. The weave
analogy focuses on interconnectedness whereas the analogy of ‘wall’ shows a
balance between the notion of unit and interconnectedness. Pupil D we suggest has
read the pattern created by the cell walls on the overhead projector as an example of
interconnectedness, rather than the boundaries of isolated units.
It is interesting to note that Pupil D’s use of analogy in her writing is reproduced
from the teacher’s speech. She writes of searching for ‘a pattern like a honeycomb’,

15A Multimodal Environment
FIG. 5. Pupil D’s text.

16 C. Jewitt et al.
however, her image bears no relation to a honeycomb pattern and is based on her
visual analogy of a wavy weave.
Conclusion
The pupils’ construction of the entity ‘cell’ was a multimodal event, a process
involving speech, action (in the form of experimentation) and image. The process of
constructing the entity involved collating, selecting and adapting information from a
range of modes. It also involved transforming/transducing information across modes
(e.g. verbal analogy to visual analogy, experimentation into written report), each
mode enabling information to be expressed and extended in speci c ways (to say
‘brick wall’ and to draw ‘brick wall’ require different work), depending on the
interests of the pupils.
In the pupils’ use of analogy the teacher’s conception of the cell was maintained
and elaborated to incorporate the pupils’ experiences and interests. The teacher’s use
of the ‘building block’ metaphor and a honeycomb arrangement as an analogy for the
pattern of the cells is transformed by the pupils to a more familiar thing: a brick wall,
a piece of cloth or the human skin. The teacher’s interest in the aspects of regularity,
uniformity and the organic within his analogy are maintained in the pupils’ anal-
ogies. The attribution between positive and negative elements of an analogy is not
a static classi cation, but a dynamic process of negotiation.
The visual entity ‘cells’ was not constructed in the lesson as such, but negotiated
between the different worlds of scienti c knowledge and the everyday. Where the
scienti c world consists of the pupils’ knowledge of science and the envisage d world
of the known, books and rules, the everyday world is made up of the pupils’
perceptions, what they can see, say and imagine (e.g. through analogy). An example
of pupils’ negotiation between what is considered irrelevant in science and the
teacher’s rhetoric of ‘seeing is believing’ is the pupils’ decisions to represent or to
not represent an air bubble in their drawings. Where pupils’ knowledge of what to
ignore was limited, the negotiation was not possible and the ‘seeing is believing’
ideology elevated air bubbles to the status of cells: we are looking at cells, what we
see is a cell. The pupils’ conceptualisation of onion cells was more or less
problematical depending on their understanding of air bubbles . So some pupils left
them out, others represented them as ‘irregularity’, while others represented them as
a part of the cell itself. These two worlds of reality worked together to produce a
notion of scienti c realism; a form of hyper-realism.
Focusing on the pupils’ texts and their transformation of the teacher’s communica-
tive work (across modes, in terms of composition, genre and analogy) serves to
emphasise the dynamic nature of the process of learning and the ways in which
different pupils’ interests in uence this process. It also highlights how the visual and
the written elements of a text can be used to attend to different aspects of meaning;
to realise different functions.
We suggest that bringing a multimodal perspective to educational research has a
number of advantages over perspectives which focus primarily on talk and writing.
First, it enables research to focus on a rich range of resources, including talk, speech,
image and action. This appears particularly important in the case of science edu-
cation where action and image are often foregrounded in the classroom and highly
valued within the curriculum. Second, a multimodal approach expands notions of
‘work’ (text and data) beyond speech and writing. We suggest that research which

17A Multimodal Environment
focuses on learning and teaching as purely linguistic processes, fails to capture the
full range of learning and teaching within the classroom and, in doing so, it negates
the ‘non-linguistic’ work of teachers and pupils. In contrast, the interpretative work
of making sense of diagrams, for instance, or, as discussed in this paper, the
experience of looking at an entity through a microscope is brought into the arena of
research by a multimodal perspective. Third, it highlights the active role of the
teacher, not only in what he or she says, but in their actions, posture and movement.
A multimodal approach to the classroom also suggests new messages for the
practice of science teaching. First, it highlights the need to attend consciousl y to all
modes of communication, both in terms of the resources available for teachers and
those made available to pupils (i.e. in the setting up of tasks). Second, it points to
the need to develop an ‘awareness’ of how these modes are used in the science
classroom (in the same way as educational research on language has made us more
conscious of the use of language, pace and voice, as a resource for teaching). For
example, asking what modes are used: are they used to attend to different aspects of
a phenomena (e.g. using the visual as a  lter for what to see and speech as
instructions on ‘how to see’)? Third, a multimodal perspective also highlights the
bene ts of thinking about what mode may be ‘best suited’ to a task and the different
cognitive and representational demands different modes place on pupils. Fourth,
looking at learning and teaching as multimodal processes also requires thinkin g
about the relations between modes in the construction of explanations, etc. For
example, is the use of image, action and speech in a lesson producin g con icting
messages or are they reinforcing each other? Much of this is implicit in the teaching
of science but remains inaccessible at the level of ‘intuition’. Our work suggests that
there is a need for the ‘work’ of image, action and other modes to become more
explicitly articulated, to be made available for re ection, re nement and improve-
ment as teaching and learning resources.
We are not suggesting that the linguistic realisation of meanings is no longer
important, we are, however, suggesting that the visual realisation of meaning is
important. Learning can no longer usefully be considered a purely linguistic accom-
plishment. Within this multimodal environment, social semiotics with its emphasis
on the many modes we use for representing and communicating and its insistence
that signs come about as the re ection of the interest of their makers provides a
framework for rethinking learning, for looking in detail at what pupils ‘do’. It
enables us to approach pupils’ texts as one kind of evidence of their interests. This
way of thinking about learning enables us to acknowledge the agency, interests and
transformative work of pupils in learning. It also enables teachers to re ect produc-
tively on the effects of their teaching and on their responses to the pupils’ interests.
Correspondence: Carey Jewitt, Culture, Communication and Societies, Institute of
Education, London University, 20 Bedford Way, London WC1H 0AL, UK.
NOTE
[1] This paper draws on the  ndings from an ESRC funded research project entitled ‘The rhetoric of the
science classroom: a multimoda l approach’, C. Jewitt, G. Kress, J. O gborn, and C. Tsatsarelis,
Institute of Education, University of London.

18 C. Jewitt et al.
REFERENCES
BOURDIEU, P. (1991) In: J.B. THOMPSON (E d.) Language and Symbolic Power [trans. G. RAYMOND & M.
ADAMSON] (Cambridge, Polity Press).
HALLIDAY, M. (1985) An Introduction to Functional G rammar (London, Edward Arnold).
HODGE, R. & KRESS, G. (1988) Social Semiotics (Cambridge , Polity Press).
KRESS, G. (1997) Before W riting (London, Routledge).
KRESS, G. & VAN LEE UWEN, T. (1996) R eading Images: the grammar of visual design (London,
Routledge).
KRESS, G., JEWITT , C., OGBORN, J. & TSATSAR ELIS, C. (2001) M ultimodal Teaching and Learning:
rhetorics of the science classroom (L ondon, Continuum).
LEMKE, J. (1998) Multiplying meaning: visual and verbal semiotics in scienti c text, in: J.R. MARTIN &
R. VEEL (Eds) Reading Science: functional perspectives on discourses of science (London, Routledge).
MARTIN, J.R. & VEEL, R. (Eds) (1998) Reading Science: functional perspectives on discourse s of science
(London, Routledge).
MARTINEC, R. (1996) Rhythm in Multimodal T exts (London, The London Institute).
O’TOOLE, M (1994) The Language of Displayed Art (Leicester, Leicester University Press).
VAN LEEUWEN, T. (1999) Speech , Music, Sound (London, Macmillan).