Content uploaded by George Kaliampos
Author content
All content in this area was uploaded by George Kaliampos on Apr 28, 2022
Content may be subject to copyright.
International Journal of Evaluation and Research in Education (IJERE)
Vol. 11, No. 2, June 2022, pp. 617~627
ISSN: 2252-8822, DOI: 10.11591/ijere.v11i2.22074 617
Journal homepage: http://ijere.iaescore.com
Narration and multimodality: The role of the human body and
material objects in science teaching
Panagiotis Pantidos1, George Kaliampos2, Konstantinos Ravanis3
1Department of Early Childhood Education, National and Kapodistrian University of Athens, Athens, Greece
2Department of Education, School of Education, University of Nicosia, Nicosia, Cyprus
3Department of Educational Sciences and Early Childhood Education, University of Patras, Patras, Greece
Article Info
ABSTRACT
Article history:
Received Aug 6, 2021
Revised Feb 25, 2022
Accepted Mar 28, 2022
This article seeks to shed light on the semiotic approach to science teaching
and learning. Essentially, the mental representations of learners are also
affected by the sign vehicles employed to communicate ideas in the material
world. Thus, any learning object also appears as a material representation,
consisting of acoustic and visual forms, which affect its content. The human
body’s kinetic modalities, spatial configurations (i.e., graphs, images),
material objects, prosody, as well as the written and spoken word constitute
the perceptual data that encode the concepts. This particular paper deals with
the possibility that the more emphatic signifiers, i.e., the human body and
material objects, can create narrative spaces and produce meaning during
science teaching. It also discusses alternative uses of material objects along
with the multiple interpretations their visual images can evoke. As regards
the human body, iconic, deictic, and ergotic gestures are analyzed as forms
that produce meaning and are autonomous and dynamic when working with
the other semiotic systems. Both material objects and the human body rely
upon the ability of the learners’ imagination to transport them to narrative
worlds located outside the classroom.
Keywords:
Human body
Material objects
Multimodality
Narration
Science teaching
This is an open access article under the CC BY-SA license.
Corresponding Author:
Panagiotis Pantidos
Department of Early Childhood Education, National and Kapodistrian University of Athens
13α Navarinou, Athens, 10680, Greece
Email: ppantidos@ecd.uoa.gr
1. INTRODUCTION
Research in science education has shown that cognitive processes are the product of the continuous
interaction of signs produced by the teacher and the students [1]–[4]. The representation of a scientific
concept through the different semiotic systems (spoken word, gestures, spatial compositions) is not a
rendition of the same thing in different ways; rather it gives knowledge shape [5]. In short, it appears that any
conceptualization of scientific entities derives not only from within the sciences in which these entities are
primarily created (e.g., physics, chemistry) or from the learners who have already conceptualized them as
preconceptions, but also from the sign vehicles constructed to “transfer” them during the learning process
(i.e., rhetorical figures, gestures, storytelling, use of material objects). Thus, any learning situation is regarded
as consisting of acoustic and visual forms and modalities, which affect its content. In other words, the
production of meaning is based on a three-factors syntax consisting of the synergies of verbal, kinetic, and
spatial elements [6]. Multimodality is perhaps a key pillar of science teaching and learning, with each
semiotic system offering various modalities as meaning-carriers, while each modality frequently carries
different information [3], [7]–[10]. In general, the morphology of the teaching framework leads to the
emergence of specific modalities (iconic gestures) while learners, for their part, develop a multimodal syntax
ISSN: 2252-8822
Int J Eval & Res Educ, Vol. 11, No. 2, June 2022: 617-627
618
(speech, space, body) [11]–[13]. In this context, the human body specifically constitutes a factor for synergy
and for the conceptual “welding” of the modes of communication [1], [5], [14]. It should be noted that
contemporary approaches argue that cognition is linked to processes of movement and of the human body’s
physical interaction with the environment [15].
Embracing the contribution of semiotic resources in the meaning-making process, Givry and Roth
[16] state that mental representations of the concepts and phenomena of the natural world do not simply
constitute an individual’s internal issue; they have characteristics related to the morphology of the
environment and the actions of learners. Previous researchers argue that conceptual change is understood as:
“i) Evolution in the use of modalities (using words to describe an object, instead of designing it by deictic
gesture); ii) Evolution into the same modality (using more gestures to describe the same objects); and iii)
Evolution of the link between different modalities (the time between talk and gesture decreasing).”
This article is focused on the narrative character of science teaching, i.e., the ability of various
semiotic systems to produce signifiers inside physical space, which create narrative spaces. These spaces,
created by material signifiers (i.e., speech, body, spatial entities) refer to 3D referents, regardless of whether
they exist in the time and space in which the narration takes place. For example, the snapshots of the
oscillation of a spring of a school textbook image, tell the story of a 3D material spring, which however, does
not exist in the physical space where the narration is taking place, i.e., in the classroom during the teaching
moment [17], [18].
In this article, the discussion will center on the 3D signifiers involved in the production of meaning
and, actually, those that can change their form and/or dynamics in space, thus increasing their narrative
capacity. The discussion covers the human body as well as material objects, because the ability of the first to
signify is indisputable, while as for material objects, given their ability to move in space and collaborate with
the human body, they too possess narrative dynamics [19]–[21]. It should be noted that the current analysis
excludes the spoken word since the verbal analysis of meanings already occupies an important place in the
tradition of science education. Thus, this paper’s objective is to describe the signifiers related to the 3D
nature of science teaching and of the human body and material objects in particular. Factors, although not
related to traditional orality, do participate in a dynamic way in distributed cognition [22]. This is associated
with the integrated approach to learning, according to which significations are carried out through the
learners’ action experiences in each environment, something that in recent years has been systematically
explored in science, mathematics and information and communication technologies (ICT) education [10],
[23], [24]. Any analysis of the human body and material objects, should be preceded by a discussion on the
inherent potential of verbal, somatic, spatial, and sonic signifiers to create “3D” referents as imaginary
creations. Namely, in each case, whether this is the physical space the learners are occupying, or the
imaginary space manifested through the narration, all signifiers cause learners, as existing members of the
material world and of experience, to think in terms of materiality. Either through physical experiences or
imaginatively, learners participate in worlds containing the acts and forms of 3D signifiers and/or “3D”
referents.
2. THE NARRATIVE CHARACTER OF SCIENCE TEACHING
Science teaching in the classroom, consists of signifiers, which locate the concepts in spaces outside
it. The linguistic and non-linguistic treatment of natural phenomena contains referents that exist outside the
school and, consequently, invoking the learners’ imagination is a teaching requirement. It should be noted
that to activate their imagination, learners use abstraction and narration to transfer themselves from the
physical space they occupy (classroom) to other places [17]. Thus, this is the requirement for teaching
science, since the objective is to escape from the phenomenology of “here” to the imaginary (the abstract
world) of “there” or of “elsewhere”. This function is achieved by activating the semiotic resources, with the
human body playing a dynamic role. Bodily actions are a key means of activating the imagination, since they
assist in the construction of narrative worlds, within which learners actually live and think. Whatever
happens in the material world of the classroom that is realized through semiotic systems, invites learners to
imagine and construct concepts in an intelligible “there” [13], [25]–[27].
Pozzer-Ardenghi [28] also emphasized the search for a communication channel between “here”
(classroom) and “there” (outside the classroom). These researchers refer to the shifts in body positions of the
speaker (educator), the movements of the body in general, the prosodic features of the voice, and the use of
pronouns as factors that shape the narrative framework. All these factors contribute to learners imagining that
they are in places and engaged in activities outside the classroom environment, dealing with scientific code in
the “here”, the “now”, and the “then” [29].
In particular, for the discussion of the code of the inscriptions (graphs, maps, diagrams), Roth,
Lawless, and Tobin [30] considered that by using gestures, especially iconic ones, listeners are transported
Int J Eval & Res Educ ISSN: 2252-8822
Narration and multimodality: The role of the human body and material objects in … (Panagiotis Pantidos)
619
from the material place that contains the image (blackboard, book) to the place of narration. When discussing
a drawing, a graph, or even the arrangement of material objects, learners use gestures to move, rotate, and
reshape entities that exist in the material space or to create entities that are, at the moment, absent from the
action area. This leads to the creation of imaginary worlds, realistic or unrealistic. In such worlds we can
place objects, people and entities and co-operate with their mental images [17], [31], [32]. For example, the
learners that cope with a mechanical pulley system in physical space can use their imagination to create
mental images of it in order to be able to predict its later states. Thus, they give the opportunity to the
mechanical system, on an imaginary level, to move, rotate, change and generally obey a scenario with spatio-
temporal variables (mental animation) [33].
In the following dialogue between a teacher and a learner, the spoken word, assisted by the material
object “spring”, creates a place that consists of the entities “greengrocer”, “spring scale”, and “fruits”.
“Teacher: When you shop at the greengrocer’s … have you noticed that the greengrocer has a spring
scale (Holds a spring)? Have you seen it? Doesn’t the scale have a spring? What’s it for?
Learner: To measure force.
Teacher: To measure force. What force? The force of the weight of the fruit. Essentially, to measure
a force, what do we need to measure, the … (shows the spring) deformation” [34].
An appeal to the imagination takes place so that, based on their own experiences, learners will
imagine themselves in a greengrocer’s shop—or will be more observant the next time they go—focusing on
the scale and perhaps, if it’s visible, on its spring. This narrative space, composed mainly of the spoken word,
invites learners to work with the mental images of the entities involved, something that although possibly
more demanding, in many cases leads to desirable learning outcomes [32], [35].
It is worth noting that narrative spaces can be created not only of the spoken or written word but
also through the use of the human body, visually, and through sounds as well. Figure 1 shows the teacher
constructing the narrative space, assisted by a “pendulum”, which as a referent, is located outside the
classroom. The field of action only contains its physical representation. Figure 1 contains two snapshots
where the teacher embodies the entities “string” and “sphere”, while subsequently, he himself physically
represents the oscillation of the pendulum. When gestures and, generally, bodily expression represent entities
and refer to actions outside the physical space, they then can constitute a facilitative factor in learning. The
research of Ping and Goldin-Meadow [35] conducted with young children regarding the concept of quantity
conservation is characteristic. It demonstrated that the gestures the teacher used, invoking the children’s
imagination, to refer to material objects absent from the field of action were elements of the children’s
imaginary world and mental images that influenced their learning.
Figure 1. Physical representation of two positions of a “pendulum”
Figure 2 is a visual representation of a spring-bar system. One can see two successive snapshots of a
rotating bar, which is released by a coiled vertical spring. It should be noted that in terms of visual semiotics,
this particular inscription carries a specialized code and comprehending it requires the viewer to possess
special knowledge. In other words, everyday life does not come with a vertical spring and a rotating bar.
Consequently, in their effort to approach them mentally, learners create an imaginary space with the mental
images of the spring-bar; there they can “turn” the bar, “compress” the spring, and generally “observe” the
movement of the system using their imagination exclusively.
ISSN: 2252-8822
Int J Eval & Res Educ, Vol. 11, No. 2, June 2022: 617-627
620
Figure 2. Rotation of a bar
Spaces expressed through sounds, being perhaps more abstract, require a greater effort on the
learners’ part to journey in them. For example, the sharper sound
(https://www.youtube.com/watch?v=5I_cvSbz2l4) reflects a gradual increase in frequency. One perceives
that this change in frequency creates a sound landscape that refers to “filling a vessel with water that falls
with a certain momentum.” More specifically, when the surface of the water gradually rises, the wavelength
of the produced static wave becomes shorter. The result is a corresponding increase in frequency, making the
sound sharper. Similarly, in the video (https://www.youtube.com/watch?v=qvWxhhi0_yk) the timbre of the
sound refers to a moving car on a F1 race track. The perceived frequency of the F1 car first increases and
then decreases as the car passes an immobile observer/listener, which, from a physics point of view,
corresponds to the Doppler effect. The sonic signifiers invite learners to use their imagination to create
spaces in which they can mentally manage the various “3D” entities. Generally, signifiers can construct or
suggest places that are structurally governed by various degrees of abstraction.
Thus, Figure 3 describes a familiar space, where a bike with a rider slows down by braking rather
abruptly; the resulting inertia causes the rider to “want” to maintain his existing kinetic state and he continues
moving and is launched forward. This particular “story” encapsulates Newton’s first law, inertia and
decelerating movement. It creates conditions for learners to embody inertia in 3D physical space, giving them
the opportunity to use their imagination to search for similar or related experiences in their everyday life.
Conversely, Figure 4, which contains a speed/time graph on decelerating motion, requires learners to be
familiar with the written code of the diagram in order to understand it.
Figure 3. The rider is launched forward due to the inertia
When teaching physics, the narrated “stories” obviously need interpretations. The different quality
of the signifiers involves corresponding semantic requirements. Thus, Figures 3 and 5 constitute situations
that are related to embodied everyday experiences and narrate Newton’s first and third law respectively,
while Figure 4, as a graph with a special code, is far removed from its referents. In other words, the static,
two-dimensional, inclined line of the graph in Figure 4 refers to a moving, 3D, smoothly decelerating vehicle
found in our everyday world, such as, for example, the bike in Figure 3. Under no circumstances, however, is
the relationship between Figures 3 and 4 explicit and obvious.
Int J Eval & Res Educ ISSN: 2252-8822
Narration and multimodality: The role of the human body and material objects in … (Panagiotis Pantidos)
621
Figure 4. Decelerating motion
Figure 5. Action and (re)action
Regarding code in the context of mathematical formalism and special representations in general, it
seems necessary the creation of semantic connections between these abstract representations and the
knowledge of everyday life. After all, this is the way that science supposed to operate; that is to enable,
through knowledge construction, to move from physical world towards the world of signs. The didactic
process operates in a similar context, with the main goal being the reshape of the pre-existing mental
representations of the trainees in order to acquire, to a certain extent, compatibility with the scientific models.
Therefore, whether they are entities that appear in the classroom, or entities that narrative places outside of it,
either abstract or concerning the knowledge of everyday life, it is required through teaching process to create
semantic bridges between them. In this context, the cultivation and invoke of learners’ imagination as well as
their training in the production of synergies between the various semiotic systems (oral speech, human body,
material objects, simulations), can bring closer the codes of abstraction with the codes of physical reality.
Consequently, science teachers are called upon to carry out intertextual transitions “travelling” to narrative
worlds constructed by the various semiotic systems [36].
3. MATERIAL OBJECTS
Material objects acquire an unmistakable semiotic perspective in science teaching if they are co-
construction factors of the learners’ mental representations. Papert and Harel [37] considered them as
‘objects to think with’, carrying a semiotic potential given that learners, by handling them, create signs and
gradually forming a communication relationship. Objects can be integrated into different frameworks of
action by acquiring alternative usages and thus promoting creativity and divergent thinking [38], [39].
Science teaching requires, on the one hand, the use of specialized materials, appliances, and instruments,
which learners must familiarize themselves with and train in their use. One such example is a simple
electrical circuit, consisting of a switch, electrical source, voltmeter, and resistor. On the other hand,
everyday objects that learners are familiar with can be used, revealing hidden properties and differentiated
uses. Figure 6 shows a glass that contains water and oil. While in everyday life both materials are food, they
can also be used unconventionally, i.e., as components of an inhomogeneous mixture. Oil has a lower density
than water and consequently floats on it.
Figure 6. Glass containing oil and water
ISSN: 2252-8822
Int J Eval & Res Educ, Vol. 11, No. 2, June 2022: 617-627
622
Similarly, although a lemon can also serve as food, if a nail with a protective zinc coating and a
copper coin are inserted into it, then the lemon becomes a galvanic element Zn-Cu as shown in Figure 7.
Usually, we drink the water but a glass of it serves as a converging lens producing an inverted image if an
object is positioned at a greater distance from the focal point as seen in Figure 8. Generally, the use of objects
in a fashion different than that specified by their manufacturer is gaining momentum in science teaching.
Figure 7. Lemon acting as a battery
Figure 8. Inverted image
Moreover, in terms of visual semiotics, when objects are approached as images, as visual stimuli
that lead viewers to interpretations, then the images of material objects are open to multiple interpretations.
Thus, as a visual form, a snapshot, or even a change in which a material object participates, is open to many
different readings. An ‘image’ from the action of an object or successive ‘images’ of it, constitute one or
more moments of its history. That is, the ability of the object to capture in its form the entire interactions with
animate and non-living entities. In this context, the observer is asked to interpret these ‘images’ along with
the changes in the ‘image’ of the object by attributing denotations and connotations [40], [41].
With reference to Figure 9, we can picture “a collapsed bottle”. On a first reading, an observer
understands that mechanical pressure exerted from our hands made it crashed; at the connotation level
crashing is due to the negative pressure produced inside the bottle. Of course, the connotative interpretation is
more demanding and requires specialized knowledge and skills to understand the unobvious. More
specifically, pouring some hot boiling water into a plastic bottle, the water vapor fills the bottle and pushes
some of the gas air molecules to go out from it. At the same time some of the air that remains inside the
bottle is heated and expands pushing also gas air molecules outside the bottle. Hence the amount of the air in
the bottle reduces. Tapping the bottle, the reduced but still hot air molecules remain in the bottle. When cool
the tapped bottle by pouring cold water over it, the water vapor condenses to liquid water and the gas air
molecules contract. These lower the pressure inside the bottle. At this point of the experiment the outside
atmospheric pressure is much higher than the pressure inside the bottle and makes the bottle collapse creating
finally a balance between inside and outside pressure.
Figure 9. Α collapsed bottle
Int J Eval & Res Educ ISSN: 2252-8822
Narration and multimodality: The role of the human body and material objects in … (Panagiotis Pantidos)
623
Figure 10 constitutes a similar case of multiple readings regarding material objects on the level of
visual change. A sequential viewing of the two images, as two different forms of the same material object,
shows that some factors caused this change. At the denotation level, this particular change could be
interpreted as the product of mechanical pressure; at the connotative level, as the result of thermal expansion
caused by an increase in temperature under exceptional climatic conditions. Indeed, when heated, the
molecules of the metal rails alter the relative distances between them and lengthen. If the manufacturer had
not planned to leave gaps between two consecutive pieces of rail, then the elongated sections pressing against
each other would result in distortions as evidenced in Figure 10. The first interpretation might be more
popular, because in everyday life, mechanical results are more obvious than thermal ones, and the lack of
specialized knowledge might be the factor leading to it. Respectively, different interpretations about the two
directions of the drawn arrow in Figure 8 could be an action of turning the paper upside down or an action of
putting a glass of water in front of it. In another instance, an image of a paper clip in a glass of water and
another image of the paper clip outside the glass can be attributed to the act that someone put his/her hand in
the water and took out the clip or he/she used a magnet to remove it.
Figure 10. Thermal expansion of railroad tracks
What is important for science teaching is the didactic use of these visual changes. Teaching learners
to focus on changes related to the form of material objects and experimental devices and on the search for
multiple interpretations promotes inquiry learning and cultivates critical thought. In general, change can be
considered a tool that helps assign meaning to a sequence of situations. In that sense, students can seek to
interpret why at least two instances in a teaching procedure are linked. These can be successive incidents in
an oral narration, different sections of a diagram, different indications on a measuring instrument, and so on.
The interpretation process starts when students are invited to assign meaning to a sequence of these narrative
pictures. It should be noted that the students can provide multiple interpretations for a specific change
depending on how any previous relevant conceptions they might have relate to the existing context.
4. HUMAN BODY
Many researchers in science teaching focus on gestures, adopting the distinction between iconic,
ergotic and deictic. Iconic gestures are those that are a morphological representation of some human action or
an (in) animate entity, ergotic gestures are associated with the handling of material objects, while deictic
gestures are those that point in one direction (usually with the forefinger) [21], [42]. All these gestures,
including movement in space (proxemics), create conceptual connections with the spoken word and the
material environment [1], [43]. For example, in teaching activities, when physically active, preschool-aged
children can produce iconic, deictic, and ergotic gestures as thought acts [44]. Pantidos et al. [45] starting
from the semiotics of the theater, refer to gestural or kinetic signs, involving the entire human body, which
can produce images and point to as well as handle objects. Regarding the latter, these researchers refer to
manipulation of an actual object, manipulation of a model (e.g., of a water molecule), as well as of objects,
which, although absent from the field of action, are implied.
Iconic gestures are bodily forms that represent the shape and movements of the referent, while they
can also acquire a symbolic character. Ping and Goldin-Meadow [35] argue that they contextually link the
spoken word with the material entities in a learning state, while emphasizing that even when material objects
are absent, iconic gestures create conceptual synergies with the mental images of the objects. In related
research, the previous researchers demonstrated that, faced with conservation of quantity as the learning
objective, learners aged 5–7 developed more complete reasoning when teachers provided them with both oral
ISSN: 2252-8822
Int J Eval & Res Educ, Vol. 11, No. 2, June 2022: 617-627
624
instructions and iconic gestures for specific projects, compared to when instructions were exclusively oral.
Actually, the morphology of a learning environment influences the forms of the iconic gestures displayed by
learners. Thus, specific gestures create a conceptualization framework for each respective phenomenon or
concept. Roth also argues that such a process leads more quickly to cognitive achievements, as opposed to a
process during which iconic gestures would not be activated. Usually, such situations commence with
learners using embodied actions to express their questioning and investigation. For example, handling a globe
as an act of investigation leads to the discovery of spin as a property of the specific object, as well as to the
production of corresponding iconic gestures.
Approaching the iconic bodily expressions in teaching physics, they can be perceived as factors that
create meaning [45]. Thus, these gestures might represent a human act (e.g., pushing or pulling
someone/Newton’s third law) or an inanimate entity and its action (e.g., a photon colliding with an opaque
object). They can collaborate with the spoken word in such a way that the human body represents an
inanimate entity that experiences an action (statement: “I am kicking a ball”) or even the entity itself
acquiring the qualities of a person (statement: “I am a fast-running ball”). Generally, iconic bodily
expressions resemble to a small or large extent the form they represent, producing syntactic spoken
word/bodily expression/spatial entity structures. Additionally, when teaching activities are based on
embodied actions, bodily expression plays a central role. For example, when, during an activity on the
phenomenon of shadow formation, learners play the role of the obstacle and investigate where there is
shadow, using their body to depict the journey of light. In other cases, when trying to explain something,
learners may use iconic gestures for a short period of time.
Science in particular, as a teaching subject, contains to a large degree the use of ergotic gestures by
learners, since they relate to the handling of material objects and experimental devices [13], [21]. Interacting
with material objects through the human body leads to more complete learning outcomes and is linked to
learning even more abstract concepts [23], [46], [47]. In conditions where experimental devices are prevalent,
all kinds of gestures, including ergotic, form a common communication code, a common descriptive
language, and the spoken word is frequently unnecessary. While following an investigative thread, learners
might operate devices, observe measurements, depict, point, and simply use the spoken word to create a
voiced context [30], [48]. Young children in particular find that the use of ergotic gestures allows them to
experiment and formulate and emerge different ideas, since by interacting with materials they can run and
test scenarios, and redefine their practices, both individually and collectively [49].
Deictic along with iconic gestures are often used as interpretive filters by learners when producing
explanations for photographs, drawings, and diagrams [45], [50]. In teaching science, teachers as well as
learners, use deictic gestures to link the spoken word with elements of the space and also to denote entities
that are not materially present. Thus, deictic gestures can uphold or emphasize material entities, while
ergotic, through manipulation, can reveal their hidden properties [1], [51]. In science learning environments,
the deictic function is characteristic and very often desirable relative to the spoken word, as in, for example,
activities concerning the shape of the earth, with preschool children pointing towards the object “sphere”
without uttering a word.
Creating forms through body is not limited simply to one instrumental level but is clearly linked to
different levels of the cognitive processing of concepts. Hwang and Roth [1], extoling the human body’s
meaning-making functionality, argue that it is what harmonizes the various semiotic vehicles (written text,
graphs, utterances, material objects), uniquely contributing to the formation of an explanation framework
during the teaching of scientific concepts. Generally, research in science education has demonstrated that an
improvement in the learner’s bodily conceptualization is also an explicit demonstration of cognitive progress
[2], [22]. For example, Pantidos, Herakleioti, and Chachlioutaki [4] compared two data analyses related to
the spoken word and the gestures of preschool-aged children, examining conceptualizations regarding the
shadow formation phenomenon. In the first case, the analysis covered each child’s spoken words and deictic
gestures, while in the second case, iconic gestures were added to the analysis. It was established that the
second analysis was not only more accurate, but also showed improvements in the children’s performance
regarding certain aspects of the phenomenon, which were exclusively expressed through their iconic
gestures. Additionally, gestures as ways of thinking in learning environments gradually lead to the
conceptualization of scientific entities through the spoken word [51]. Roth, Lawless, and Tobin [30] reported
that when learners acquire their first laboratory experiences, they replace the spoken word with gestures at
this early stage. This occurs because at this stage, compared to gestures, the spoken word is incapable of
conveying aspects of the new environment. The previous researchers declared that learners find ways to use
their bodies to free themselves from this weakness of the spoken word and to imbue scientific entities with
meaning. Generally, they believe that gestures can play a significant role even for complex scientific
meanings.
Int J Eval & Res Educ ISSN: 2252-8822
Narration and multimodality: The role of the human body and material objects in … (Panagiotis Pantidos)
625
5. CONCLUSION
Alternative uses of material objects can be a good way to bridge the gap between the
phenomenology of the everyday and the scientific code. This results in exciting the learners’ interest, while
also cultivating observation in the context of everyday life, which enhances their exploratory skills.
Moreover, approaching material objects as visual images, as well as focusing on changes in their image,
cultivates the ability to detect variations and attribute causality to them. Generally, when learners search for
multiple interpretations for a given form or a change in the form of an object or arrangement of objects, they
are able to correlate between certain features of the material environment and aspects of scientific (or even
school) knowledge.
Gestures and bodily expression in general function similarly, since they contextually integrate
aspects of scientific concepts into the material environment, thus creating a hybrid communication code. For
example, in Figure 8 iconic gestures representing the path of the light rays and deictic gestures showing the
points where the rays “break” (refract), as well as combined gestures to define the outline of the image, are
necessary for communication between what is perceived and what is mentally constructed. The creation of
such communication codes could illuminate different aspects of the classic constructivist research for
teaching and learning science.
The semiotic approach to science teaching emphasizes the ability of narratives produced by material
objects and the human body to create bridges between the current “here” and the narrated “there”. Science
teachers should create the conditions that allow learners to integrate the codes of school knowledge into the
material signifiers of the material objects and the human body. Generally, such an approach might offer
additional tools for transforming scientific knowledge into school knowledge. Although the dynamics of a
semiotic perspective are recognized in science education, more systematic research is appropriate.
ACKNOWLEDGEMENTS
The authors are grateful to S. Avgoulea Linardatos School, the physicist Nektarios Protopapas for
the contribution to this study, and to Pantelis Constantinou for the drawings in Figures 1, 2, 3 and 5.
REFERENCES
[1] S. Hwang and W.-M. Roth, “The (Embodied) Performance of Physics Concepts in Lectures,” Research in Science Education,
vol. 41, no. 4, pp. 461–477, Apr. 2010, doi: 10.1007/s11165-010-9175-4.
[2] K. S. Tang, C. Delgado, and E. B. Moje, “An Integrative Framework for the Analysis of Multiple and Multimodal
Representations for Meaning-Making in Science Education,” Science Education, vol. 98, no. 2, pp. 305–326, Jan. 2014, doi:
10.1002/sce.21099.
[3] A. E. Pierson, D. B. Clark, and C. E. Brady, “Scientific modeling and translanguaging: A multilingual and multimodal approach
to support science learning and engagement,” Science Education, vol. 105, no. 4, pp. 776-813, May 2021, doi: 10.1002/sce.21622.
[4] P. Pantidos, E. Herakleioti, and M.-E. Chachlioutaki, “Reanalysing children’s responses on shadow formation: a comparative
approach to bodily expressions and verbal discourse,” International Journal of Science Education, vol. 39, no. 18, pp. 2508–2527,
Oct. 2017, doi: 10.1080/09500693.2017.1392644.
[5] M. Hackling, K. Murcia, and K. Ibrahim-Didi, “Teacher orchestration of multimodal resources to support the construction of an
explanation in a Year 4 Astronomy topic,” Teaching Science, vol. 59, no. 1, pp. 7–15, 2013.
[6] D. Givry and P. Pantidos, “Toward a multimodal approach of science teaching,” COLLOQUE SIEST MEDITERRANEE TUNIS
2012 Dispositifs, demarches, apprentissage dans l’Enseignement des sciences et technologies, 2012, pp. 123–129.
[7] G. Kaliampos, V. Kada, A. Saregar, and K. Ravanis, “Preshool pupils mental represenations on electricity, simple electrical
circuit and electrical appliances,” European Journal of Education Studies, vol. 7, no. 12, pp. 596–610, 2020.
[8] M. Koutsikou, V. Christidou, M. Papadopoulou, and F. Bonoti, “Interpersonal Meaning: Verbal Text-Image Relations in
Multimodal Science Texts for Young Children,” Education Sciences, vol. 11, no. 5, p. 245, May 2021, doi:
10.3390/educsci11050245.
[9] M. Wood and S. Stocklmayer, “Visual Context and Relevance in Life Cycle Diagrams,” Interdisciplinary Journal of
Environmental and Science Education, vol. 17, no. 1, p. e2224, Nov. 2020, doi: 10.29333/ijese/9154.
[10] Z. C. Zacharia and G. Olympiou, “Physical versus virtual manipulative experimentation in physics learning,” Learning and
Instruction, vol. 21, no. 3, pp. 317–331, Jun. 2011, doi: 10.1016/j.learninstruc.2010.03.001.
[11] S. Goldin-Meadow and M. W. Alibali, “Gesture’s Role in Speaking, Learning, and Creating Language,” Annual Review of
Psychology, vol. 64, no. 1, pp. 257–283, Jan. 2013, doi: 10.1146/annurev-psych-113011-143802.
[12] E. Herakleioti and P. Pantidos, “The Contribution of the Human Body in Young Children’s Explanations About Shadow
Formation,” Research in Science Education, vol. 46, no. 1, pp. 21–42, Feb. 2015, doi: 10.1007/s11165-014-9458-2.
[13] S. Goldin-Meadow, “Taking a Hands-on Approach to Learning,” Policy Insights from the Behavioral and Brain Sciences, vol. 5,
no. 2, pp. 163–170, Aug. 2018, doi: 10.1177/2372732218785393.
[14] J. B. Lopes et al., “Constructing and Using Multimodal Narratives to Research in Science Education: Contributions Based on
Practical Classroom,” Research in Science Education, vol. 44, no. 3, pp. 415–438, Nov. 2013, doi: 10.1007/s11165-013-9381-y.
[15] C. Kontra, D. J. Lyons, S. M. Fischer, and S. L. Beilock, “Physical Experience Enhances Science Learning,” Psychological
Science, vol. 26, no. 6, pp. 737–749, Apr. 2015, doi: 10.1177/0956797615569355.
[16] D. Givry and W.-M. Roth, “Toward a new conception of conceptions: Interplay of talk, gestures, and structures in the setting,”
Journal of Research in Science Teaching, vol. 43, no. 10, pp. 1086–1109, 2006, doi: 10.1002/tea.20139.
[17] Y. Hadzigeorgiou, “Erratum: Imaginative Science Education,” in Imaginative Science Education. Springer International
Publishing, 2016.
ISSN: 2252-8822
Int J Eval & Res Educ, Vol. 11, No. 2, June 2022: 617-627
626
[18] P. Pantidos, “Narrating Science in the Classroom: The Role of Semiotic Resources in Evoking Imaginative Thinking,” Journal of
Science Teacher Education, vol. 28, no. 4, pp. 388–401, May 2017, doi: 10.1080/1046560x.2017.1345557.
[19] T. de Jong, M. C. Linn, and Z. C. Zacharia, “Physical and Virtual Laboratories in Science and Engineering Education,” Science,
vol. 340, no. 6130, pp. 305–308, Apr. 2013, doi: 10.1126/science.1230579.
[20] Z. C. Zacharia, E. Loizou, and M. Papaevripidou, “Is physicality an important aspect of learning through science experimentation
among kindergarten students?” Early Childhood Research Quarterly, vol. 27, no. 3, pp. 447–457, Jul. 2012, doi:
10.1016/j.ecresq.2012.02.004.
[21] M. Kim, W.-M. Roth, and J. Thom, “Children’s gestures and the embodied knowledge of geometry,” International Journal of
Science and Mathematics Education, vol. 9, no. 1, pp. 207–238, Sep. 2010, doi: 10.1007/s10763-010-9240-5.
[22] C. Kontra, S. Goldin-Meadow, and S. L. Beilock, “Embodied Learning Across the Life Span,” Topics in Cognitive Science,
vol. 4, no. 4, pp. 731–739, Sep. 2012, doi: 10.1111/j.1756-8765.2012.01221.x.
[23] I. Han, “Embodiment: A New Perspective for Evaluating Physicality in Learning,” Journal of Educational Computing Research,
vol. 49, no. 1, pp. 41–59, Jul. 2013, doi: 10.2190/ec.49.1.b.
[24] S. Anastopoulou, M. Sharples, and C. Baber, “An evaluation of multimodal interactions with technology while learning science
concepts,” British Journal of Educational Technology, vol. 42, no. 2, pp. 266–290, Feb. 2011, doi: 10.1111/j.1467-
8535.2009.01017.x.
[25] D. Givry and A. Tiberghien, “Studying Students’ Learning Processes Used during Physics Teaching Sequence about Gas with
Networks of Ideas and Their Domain of Applicability,” International Journal of Science Education, vol. 34, no. 2, pp. 223–249,
Jan. 2012, doi: 10.1080/09500693.2011.566289.
[26] R. E. Scherr, H. G. Close, E. W. Close, and S. Vokos, “Representing energy. II. Energy tracking representations,” Physical
Review Special Topics - Physics Education Research, vol. 8, no. 2, Oct. 2012, doi: 10.1103/physrevstper.8.020115.
[27] G. Fragkiadaki and K. Ravanis, “The unity between intellect, affect, and action in a child’s learning and development in scie nce,”
Learning, Culture and Social Interaction, vol. 29, p. 100495, Jun. 2021, doi: 10.1016/j.lcsi.2021.100495.
[28] L. Pozzer-Ardenghi, “Visual Representations in Educational Research,” in A Companion to Research in Education, Springer
Netherlands, 2013, pp. 513–516.
[29] P. Pantidos, K. Ravanis, K. Valakas, and E. Vitoratos, “Incorporating Poeticality into the Teaching of Physics,” Science &
Education, vol. 23, no. 3, pp. 621–642, Jan. 2013, doi: 10.1007/s11191-012-9573-2.
[30] W.-M. Roth, D. Lawless, and K. Tobin, “Towards a Praxeology of Teaching,” Canadian Journal of Education/Revue canadienne
de l éducation, vol. 25, no. 1, p. 1, 2000, doi: 10.2307/1585864.
[31] P. Schrader and E. Rapp, “Does multimedia theory apply to all students? The impact of multimedia presentations on science
learning,” Journal of Learning and Teaching in Digital Age, vol. 1, no. 1, pp. 32–46, 2016.
[32] C. A. Cohen and M. Hegarty, “Visualizing cross sections: Training spatial thinking using interactive animations and virtual
objects,” Learning and Individual Differences, vol. 33, pp. 63–71, Jul. 2014, doi: 10.1016/j.lindif.2014.04.002.
[33] M. Stieff, M. Hegarty, and G. Deslongchamps, “Identifying Representational Competence with Multi-Representational Displays,”
Cognition and Instruction, vol. 29, no. 1, pp. 123–145, Jan. 2011, doi: 10.1080/07370008.2010.507318.
[34] P. Pantidos, “Creating a "Dictionary" of semiotic analysis terms for teaching physics: a framework for the study of teaching
practices employing theater semiotics,” Unpublished dissertation, Department of Educational Sciences and Early Childhood
Education, University of Patras, Greece, 2008.
[35] R. M. Ping and S. Goldin-Meadow, “Hands in the air: Using ungrounded iconic gestures to teach children conservation of
quantity.,” Developmental Psychology, vol. 44, no. 5, pp. 1277–1287, 2008, doi: 10.1037/0012-1649.44.5.1277.
[36] E. Ochs, “Experiencing language,” Anthropological Theory, vol. 12, no. 2, pp. 142–160, Jun. 2012, doi:
10.1177/1463499612454088.
[37] S. Papert and I. Harel, “Situating constructionism,” Constructionism, vol. 36, no. 2, pp. 1–11, 1991.
[38] E. Fioratou and S. J. Cowley, “Insightful thinking,” in Distributed Language, John Benjamins Publishing Company, 2011,
pp. 57–80.
[39] V. P. Glăveanu, “What can be done with an egg? Creativity, material objects, and the theory of affordances,” The Journal of
Creative Behavior, vol. 46, no. 3, pp. 192–208, 2012.
[40] G. Aiello, “Visual Semiotics: Key Concepts and New Directions,” in The SAGE Handbook of Visual Research Methods, SAGE
Publications, Inc., 2020, pp. 367–380.
[41] T. Van Leeuwen, “Multimodality,” in The Handbook of Discourse Analysis, John Wiley & Sons, Inc., 2015, pp. 447–465.
[42] M. W. Alibali, M. J. Nathan, and Y. Fujimori, Gestures in the mathematics classroom: What’s the point? Routledge, 2013.
[43] M. A. Impedovo, C. Andreucci, and J. Ginestié, “Mediation of artefacts, tools and technical objects: an international and French
perspective,” International Journal of Technology and Design Education, vol. 27, no. 1, pp. 19–30, Sep. 2015, doi:
10.1007/s10798-015-9335-y.
[44] M.-E. Chachlioutaki, P. Pantidos, and M. Kampeza, “Changing semiotic modes indicates the introduction of new elements in
children’s reasoning: the case of earthquakes,” Educational Journal of the University of Patras UNESCO Chair, vol. 3, no. 2,
pp. 198–208, 2016.
[45] P. Pantidos, K. Valakas, E. Vitoratos, and K. Ravanis, “Towards applied semiotics: An analysis of iconic gestural signs regarding
physics teaching in the light of theatre semiotics,” Semiotica, vol. 2008, no. 172, Jan. 2008, doi: 10.1515/semi.2008.095.
[46] K. Plakitsi, “Activity Theory in Formal and Informal Science Education,” in Activity Theory in Formal and Informal Science
Education, SensePublishers, 2013, pp. 1–15.
[47] L. Moro, E. F. Mortimer, and A. Tiberghien, “The use of social semiotic multimodality and joint action theory to describe
teaching practices: two cases studies with experienced teachers,” Classroom Discourse, vol. 11, no. 3, pp. 229–251, Mar. 2019,
doi: 10.1080/19463014.2019.1570528.
[48] F. Arzarello and C. Sabena, Analytic-structural functions of gestures in mathematical argumentation processes. Information Age
Publishing, 2014.
[49] G. Fragkiadaki, M. Fleer, and K. Ravanis, “A Cultural-Historical Study of the Development of Children’s Scientific Thinking
about Clouds in Everyday Life,” Research in Science Education, vol. 49, no. 6, pp. 1523–1545, Sep. 2017, doi: 10.1007/s11165-
017-9665-8.
[50] W.-M. Roth and L. Pozzer-Ardenghi, “Pictures in Biology Education,” in Models and Modeling in Science Education, Springer
Netherlands, 2012, pp. 39–53.
[51] M. A. Novack, E. L. Congdon, N. Hemani-Lopez, and S. Goldin-Meadow, “From Action to Abstraction,” Psychological Science,
vol. 25, no. 4, pp. 903–910, Feb. 2014, doi: 10.1177/0956797613518351.
Int J Eval & Res Educ ISSN: 2252-8822
Narration and multimodality: The role of the human body and material objects in … (Panagiotis Pantidos)
627
BIOGRAPHIES OF AUTHORS
Panagiotis Pantidos is assistant professor at the Department of Early Childhood
Education at the National and Kapodistrian University of Athens, Greece. He has obtained a
B.Sc. in physics in 1997, M.A. in Theory and Praxis in Teaching and Evaluation in 2001 and
Ph.D. in science education in 2008 (Thesis: Creating a “Dictionary” of semiotic analysis terms
for teaching physics: a framework for the study of teaching practices employing theater
semiotics). His research interests include scientific knowledge representation, applied theatre
semiotics in science teaching and learning, semiotics of science teaching and learning,
embodied learning in science education, narration and meaning making process in teaching
science. He has provided seminars on semiotics of science teaching in Alexandru Ioan Cuza
University and Aix-Marseille Université. He can be contacted at: ppantidos@ecd.uoa.gr.
George Kaliampos is Bachelor in Physics (University of Patras, Greece), Master
in Science Education (University of Leeds, UK), Masters in Special Needs (University of
Nottingham, UK), PhD in Physics Education in Special Needs Context (University of
Thessaly, Greece). He is currently an instructor at the Distance Learning MA in Special
Education at the Department of Education, University of Nicosia, Cyprus. He also teaches
Science Education as contract instructor at the Department of Educational Sciences and Early
Childhood Education, University of Patras, Greece. He has more than ten-year experience as a
secondary education physics teacher in Greek secondary schools. His current research interests
and publications focus on studying Science Education for diversity students. He can be
contacted at email: kaliampos.g@unic.ac.cy.
Konstantinos Ravanis is professor in didactics of physics at the Department of
Educational Sciences and Early Childhood Education in the University of Patras, Greece. He
has obtained a bachelor in physics and a bachelor in educational sciences at the University of
Patras, M.A. in science education at the Université Paris VII-Denis Diderot, and Ph.D. in
physics education at the University of Patras. He was an invited professor in Université de
Provence, Aix-Marseille Université, Universidad de Buenos Aires, Université de Bretagne
Occidentale and he had a research fellowship from the Jean Piaget Archives at the University
of Geneva. He was director of the Division of Theoretical and Applied Pedagogies (2003-
2008) and has been chairman in the Department of Educational Sciences and Early Childhood
Education (1999-2001, 2005-2006), Vice-Rector (2006-2010) and member of the Council
Foundation (2014) of the University of Patras. He can be contacted at: ravanis@upatras.gr.
