Content uploaded by Marco Magirius
Author content
All content in this area was uploaded by Marco Magirius on Nov 22, 2024
Content may be subject to copyright.
1
Magirius, M. (2024). Multimodality in L1 Education—Introduction to the Special Issue. L1-
Educational Studies in Language and Literature, 24, 1-6.
https://doi.org/10.21248/l1esll.2024.24.3.847
Corresponding author: Marco Magirius, Universität Erfurt, Nordhäuser Str. 63, 99089 Erfurt,
Germany.
Email: marco.magirius@uni-erfurt.de
© 2024 International Association for Research in L1-Education.
MULTIMODALITY IN L1 EDUCATION
Introduction to the Special Issue
MARCO MAGIRIUS
University of Erfurt
Abstract
The introduction serves as the first entry point into the special issue. Based on a semiotically grounded
concept of multimodality, it summarizes the contributions of the articles to the topic. Additionally, the
articles are thematically grouped to highlight the connections between them.
Keywords: multimodality, literature education, semiotics, teacher education
2 M. MAGIRIUS
Shifts in media preferences like the drastic decline of everyday reading of books are
often perceived as threats to literature education. The Special Issue sets a
counterpoint to these laments, grounded on the premise that literature education
should extend beyond narratives conveyed solely through written language. The
articles within this issue investigate multimodality with a specific emphasis on its
implications for literature education.
Was does multimodality mean? The term 'multimodality' encompasses several
dimensions, with perceptual and semiotic multimodality being the two most
prominent. As defined by Sachs-Hombach et al. (2018, p. 12), “a perceptual
dimension of multimodality is present when a stimulus or a constellation of stimuli
is processed through at least two modes of perception” (translated by the author).
Here, modes of perception refer to the five human senses and, in some cases, include
motoric body sensations (ibid.). Within the context of L1 education and literature
studies, semiotic multimodality plays a central role. This term is used, when
narratives are presented through “multi-layered sign systems” to “present worlds
and stories” (Eder 2022, 354). In this case, the various sign systems are conceived as
semiotic “modes,” which are frequently defined as “resources for meaning-making”
(Jewitt & Kress 2003, p. 1f.; see Serafini, 2014, p. 12f). For instance, “[i]mage, writing,
layout, gesture, speech, moving image, soundtrack are examples of [such] modes”
(Kress 2010, 79; see also Bateman at al., 2017, p. 18).
In the semiotically grounded framework of Bateman et al. (ibid., p. 7),
multimodality is defined as “a way of characterising communicative situations […]
which rely upon combinations of different ‘forms’ of communication to be effective”.
This abstract definition is particularly well-suited for educational purposes. For
example, a communicative situation possesses certain “necessary features,”
whereby specific “knowledge [...] among a community of users” must be shared. This
knowledge is about a “particular range of material regularities that are to be
considered to be carrying semiotic activity” (ibid. p. 86). Furthermore, “a scheme for
deriving interpretations from the material regularities identified must also be
shared” (ibid.). From an educational perspective, the pertinent question arises: how
can we empower students to become members of these communities of users? In
order to achieve this goal, the influential New London Group prescribes “the
development of a semiotic toolkit [...] that builds access to the literacy practices and
discourse resources that constitute the contemporary social landscape” (Siegel
2006, 72; see also Wildfeuer, in prep.). This includes much more than a list of stylistic
devices of different media—comics, films, videogames, or novels. The New London
Group (1996) recommends “Overt Instruction” comprising “the use of
metalanguages, languages of reflective generalization that describe the form,
content, and function of the discourses of practice.” Moreover, “immersion in
meaningful practices within a community” including experts is needed, as well as a
critical framing, all resulting in a “reflective practice”. By following this path,
multiliteracy—an extension of the traditional concept of literacy—can be achieved.
MULTIMODALITY IN L1 EDUCATION 3
For teachers, this presents a daunting challenge. They may not be well-versed in
how various semiotic modes contribute to meaning-making or how the modes
generate ambiguities and undecidabilities. They may also lack the terminology
necessary to perceive the idiosyncrasies of a multimodal aesthetic object.
Furthermore, some of their students may have more experience with multimodal
artworks than the teachers themselves. To determine whether these constitute
discouraging conditions for teachers to employ multimodal literature in the
classroom, it is best to inquire with them. This leads to the first main question of the
special issue: What challenges and opportunities do (prospective) teachers think of
and face in dealing with multimodal literature in literature education? How do
articles of this special issue address this question?
1) The article by Poyas and Elkad-Lehman approaches this question regarding
graphic novels. They conducted a questionnaire study involving 48 Jewish
and Arab teachers, some of whom were pursuing a Master’s Degree, while
others were in the early stages of their careers. They completed an online
questionnaire after reading the first 21 pages of “Tunnels” by Rutu Modan
(2020). 44% of the sample reported having no experience with graphic
novels. Although especially the Arab teachers appreciated the task, 58% of
the participants “found it harder to read the graphic novel than to read a
verbal novel in Hebrew”. Even though the participants were quite young
and “grew up in the digital and multimodal age”, the teachers need
additional training to confidently use graphic novels in literature classes.
This is why Poyas and Elkad-Lehmann propose “updating the teacher
training curricula in Israel—and worldwide—in order to equip language arts
teachers with the tools of reading, interpreting and mediating such texts”.
2) Comparable insights can be drawn from the article by Aliagas et al.
concerning the inclusion of digital fiction in literary education. They
interviewed six language arts teachers who were at the beginning of their
career. They voluntarily participated in a project led by researchers to
collaboratively design proposals for “guided reading of multimodal texts,
and especially digital fiction” for use in Catalan or Spanish classes. Aliagas
et al. compared three teacher profiles (very enthusiastic, moderately keen,
and more resistant) in terms of their views regarding digital fiction, as well
as challenges, difficulties or even threads associated with integrating such
literature into the classroom. The results indicate that “although the
introduction of digital fiction into schools might promote new ways of
conceptualizing the learning environment, it does require deep didactic
reflection” to realize the potential of such texts.
In addition to exploring the perspectives of teachers, the special issue features
articles that focus on how students process the specifics of semiotic modes.
Consequently, the second main question is: How can the specifics of different
multimodal media help to initiate aesthetic experiences for learners with varying
abilities and differences in prior knowledge? For aesthetic experiences, subjective
4 M. MAGIRIUS
entanglement and, subsequently, emotions are central (Magirius et al., 2023).
Emotions “are not the adversary of considered and reflective action, but rather an
indispensable compass” (Seel 2018, 132) for the purpose of perceiving the
idiosyncrasies of an aesthetic object. Four articles of the special issue deal with
aesthetic experiences with different semiotic modes.
1) The article by Wittig presents results from her exploratory dissertation
study in which 19 elementary school students performed so-called panel
readings of the comic Lehmriese lebt! [Clay Giant’s Alive!] by Anke Kuhl. In
the panel readings, the children bring chapters of the comic to life by
creating sounds using various objects. To achieve this, they must identify
where the semiotic modes of the comic convey information about sound.
In the article, Wittig meticulously analyzes how four children negotiate the
use of different sounds and how they perform the panel reading. She
demonstrates how panel readings can lead to aesthetic experiences for
learners with varying abilities.
2) While in Wittig's study, the sound of the comics had to be created—since
comics do not produce sound themselves—in Pietsch's article, the
multimodal text—a film shot—incorporates physical acoustics in the form
of sound effects and/or music. Drawing on a Master’s thesis by Seifert
(2019), he analyzes how 17 elementary school students interpret a
character’s feelings and actions in relation to different soundscapes and -
tracks. The same film shot was presented to the class with three different
audio tracks, and after each iteration, the teacher asked, “What kind of
character could that be? What might happen next and why?” The analysis
of the plenary discussion revealed in detail how the students engaged in
meaning-making by utilizing multiliteracy. Pietsch's article is particularly
significant for this special issue, as he emphasizes a concept that is
foundational to research on multimodality: the interplay between different
semiotic modes (see Batesman et al., 2017, 8). He demonstrates that
primary school students can engage with this interplay, evoking a wide
range of emotions and associations.
3) Dammers' article presents a quantitative analysis of data from his
dissertation study. In this study, 48 elementary school students were
presented with seven picture books. Over three recording sessions,
Dammers collected gaze data using eye-tracking instruments. Furthermore,
he conducted brief follow-up interviews with the students. The article
focuses on gaze data from ten students. By comparing “the fixation time of
peripheral and central areas of the picture book”, he explores whether
attention of the students is related to the narrative function of the text
elements. In some aesthetically ambitious books, peripheral areas may
contain details that need to be processed to fully understand the narrative.
However, “[t]he allocation of visual attention tends to correspond to the
expected (conventional) relevance of the print elements”. Thus, an
MULTIMODALITY IN L1 EDUCATION 5
“efficiency-oriented strategy” is already apparent in elementary school and
may inhibit literary (multi-)literacy, as literary literacy requires a prolonged
engagement with the (multimodal) aesthetic object and a deeper
interaction with it.
4) In the study by Castellano-Sanz and Reyes-Torres, a sample of 60
participants is surveyed through questionnaires and interviews regarding
their assessment of a learning unit about Valencian rondalles. The learning
unit, consisting of five sessions, aimed at multiliteracy “and the acquisition
of Catalan, while fostering enriching reading experiences”. One key aspect
of the study is the heterogeneity of the sample. It includes students aged
12 to 18 from different sociolinguistic contexts, as well as teachers who
“bring a wealth of pedagogical experience spanning from kindergarten to
baccalaureate, including specialized areas like vocational training (VT) and
adult education". The learning unit is based on extensions of concepts
proposed by the New London School (see above). Results show that while
all students “valued the creative tasks and showed interest in learning new
vocabulary and understanding rondalles”, especially students from less
diverse backgrounds “did not see the necessity of learning the minority
language”. The authors interpret this phenomenon as “'self-hatred' (Flors-
Mas, 2021)—a reluctance to embrace their minority identity, preferring
instead to align with the dominant culture”. The teachers fully embraced
methods and contents of the learning unit. This in part aligns with the
teachers’ background “as consultants for teacher-training courses related
to multilingualism”.
The brief overview of the articles of the Special Issue highlights the potential that
multimodal texts can hold for literature education and beyond. At the same time, it
becomes evident that implementation, particularly in curricula, is challenging and
necessitates specific conditions. Many of the articles demonstrate that teachers
require training in multimodality to effectively utilize such literary materials in the
classroom. This not only enables students to engage in literary learning and aesthetic
experiences, but also increases the likelihood that students can meaningfully
incorporate their everyday experiences with multimodal texts into the literature
classroom. Certainly, the individual articles present much more nuanced results, and
I encourage you to delve deeply into them.
REFERENCES
Bateman, J., Wildfeuer, J., & Hiippala, Tuomo (2017). Multimodality, foundations, research and analysis.
A problem-oriented Introduction. De Gruyter.
Eder, J. (2022). Film: The affective specificity of audiovisual media. In P. C. Hogan, B. J. Irish, & L. P. Hogan
(Eds.), Routledge Companion to Literature and Emotion (pp. 354–365). Routledge.
Flors-Mas, A., Sorolla, N., Pradilla, M. A., & Vila, F. X. (2021). The recent evolution of first languages in
Catalonia. Between minoritization and language maintenance. Language Problems and Language
Planning, 45(1), 31–55. https://doi.org/10.1075/lplp.20028.flo
6 M. MAGIRIUS
Jewitt, C. & Kress, G. (2003). Multimodal literacy. Peter Lang.
Magirius, M., Meier, C., Kubik, S., & Führer, C. (Eds., 2023). Evaluative ästhetische Rezeption als Grundlage
literarischen Verstehens und Lernens. Theorie und Empirie [Evaluative aesthetic reception as
foundation of literary understanding and learning. Theory and empirical studies]. kopaed.
Kress, G. (2010). Multimodality. A Social Semiotic Approach to Contemporary Communication, Routledge.
New London Group (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational
Review, 66(1), 60–92. https://www.sfu.ca/~decaste/newlondon.htm
Sachs-Hombach, K., Bateman, J., Curtis, Robin, & Ochsner, B. (2018). Medienwissenschaftliche
Multimodalitätsforschung [Research on multimodality from a perspective of media studies].
MEDIENwissenschaft, 35(1), 8–26. https://doi.org/10.17192/ep2018.1.7708
Seel, M. (2018). The arts of cinema. Cornell University Press.
Seifert, L. (2019). Filmmusik in der Grundschule. Durchführung und Evaluation eines Praxisprojekts in
einer vierten Grundschulklasse [Film music in elementary school. Implementation and evaluation of
a hands-on project in a fourth-grade elementary school class]. Unpublished Master’s thesis.
Serafini, F. (2014). Reading the visual. An introduction to teaching multimodal literacy. Teachers College
Press.
Siegel, M. (2006). Rereading the signs. Multimodal transformations in the field of literacy education.
Language Arts, 84(1), 65–77.
Wildfeuer, J. (2025, in prep.). Multimodale Texte lesen. Theoretische und methodische Bausteine aus der
Multimodalitätsforschung [Reading multimodal texts. Theoretical and methodical building blocks
from research on multimodality]. In M. Staiger (Ed.), Schrift / Bild – Lesen. Interdisziplinäre
Perspektiven für die Leseforschung. J. B. Metzler.
