No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
What is a mode? Smell, olfactory perception, and the notion of mode in multimodal mediated theory
Abstract
Moving towards multimodal mediated theory, I propose to define a mode as a system of mediated action that comes about through concrete lower-level actions that social actors take in the world. In order to explain exactly how a mode is a system of mediated action, I turn to a perfume blog and use one blog entry as my starting point. The mode that I primarily focus on in this article is the mode of smell, explicating that the mode of smell is not synonymous with olfactory perception, even though modal development of smell is certainly partially dependent upon olfactory perception.
As I am ostensibly focusing on the one mode, I once again problematize this notion of countability and delineate the purely theoretical and heuristic unit of mode (Norris, 2004). I clarify that modes a) do not exist in the world as they are purely theoretical in nature; b) that modes can be delineated in various ways; and c) that modes are never singular.
Even though the concept of mode is problematical – and in my view needs to always be problematized – I argue that the term and the notion of mode is theoretically useful as it allows us to talk about and better understand communication and (inter)action in three respects: 1. The notion of mode allows us to investigate regularities as residing on a continuum somewhere between the social actor(s) and the mediational means; 2. The theoretical notion of mode embraces socio-cultural and historical as well as individual characteristics, never prioritising any of these and always embracing the tension that exists between social actor(s) and mediational means; and 3. The theoretical notion of mode demonstrates that modal development through concrete lower-level actions taken in the world, is transferable to other lower-level actions taken.
Supplementary resources (2)
... Smell, however, rarely means on its own. It is almost always part of a multisemiotic, multimodal, or multisensorial activity (Howes 2011, Norris 2013. Smelling flowers in a garden or perfume in a bathroom, for example, is likely as much a tactile, visual, and/or aural experience as it is an olfactory one. ...
... Smelling flowers in a garden or perfume in a bathroom, for example, is likely as much a tactile, visual, and/or aural experience as it is an olfactory one. Compartmentalizing smell (and the sociocultural relevance of smell) from the other senses and from other semiotic resources may be useful for analytic or conceptual purposes, but it risks reducing our understanding of the process of meaning-making to a sense-by-sense or mode-by-mode account (Howes 2011, Norris 2013. While the cultural meaning of smell undoubtedly needs to be studied "on its own terms" (see Drobnik 2006, 4, above), it also needs to be understood in relation to and in combination with other meaning-making resources, to understand its potential in "multiplying the set of possible meanings that can be made" in any multisemiotic activity (Lemke 1998, 92, see also Royce 2002, 2007, O'Halloran 2005, 159, Baldry and Thibault 2006. ...
... Multisemiotic or multimodal accounts of smell tend to focus on perfumery or aromatherapy (e.g. van Leeuwen 2005, Norris 2013, van Leeuwen and Djonov 2015. Perfumery offers a sophisticated and well-established 'language of smell', with its three basic note types (top/head, middle/heart, and base) and the complex harmonies created by different combinations of notes and chords. ...
... The aim of multimodal analysis is to examine what the actions are mediated by, how semiotic resources work together in multimodal action and how those resources relate to each other in such events. According to Norris (2013b), the modes used, the way they are used and the hierarchy of modes in one situation or by one participant in a situation cannot be assumed to take place in another situation or by other participants. Modes and the way they are used come into being, emerge and are strongly situated, however, they have a history through the experiences participants have hadin their own networks or through their joint experience (Norris 2013b, see also Kress andVan Leeuwen 2001 andVan Leeuwen 2005 for social semiotic approach to the historicity of semiotic resources). ...
... According to Norris (2013b), the modes used, the way they are used and the hierarchy of modes in one situation or by one participant in a situation cannot be assumed to take place in another situation or by other participants. Modes and the way they are used come into being, emerge and are strongly situated, however, they have a history through the experiences participants have hadin their own networks or through their joint experience (Norris 2013b, see also Kress andVan Leeuwen 2001 andVan Leeuwen 2005 for social semiotic approach to the historicity of semiotic resources). Norris (2013b, 155-156) defines a mode as "a system of mediated action that comes about through concrete lower-level actions that social actors take in the world", and "a system of mediated action with regularities". ...
... The "finely tuned" mode of gaze is a mediated action with certain type of regularities and is in a hierarchical position in relation to other modes (e.g. typing and text) in this nexus of practice (Norris 2013b). ...
Through detailed multimodal analyses, this article shows how participants of an English language course create and manage sites of attention for achieving collaboration across multiple spaces. The ethnographic data for the study comes from an English language course ‘Academic reading’ offered for university students majoring in Finnish Sign Language. The paper examines a classroom situation where the participants interact simultaneously with each other in a chatroom environment and physical environment via several mediational means, such as Finnish Sign Language and typed text in a chatroom that appears on students’ tablets and on a large screen in the classroom. The emerging interactional patterns coordinate fragmented interaction into mutual sites of attention so that the participants are able to collaborate. Further, the paper reflects how gaze plays a crucial role in coordinating actions together. Gaze itself is seen as a mediated action which begins to display regularities, and ‘comes into being’ through long and short life trajectories. The longer trajectory of the mode of gaze is traced to the patterned ways of using gaze in signing communities of practice.
... Metaphor Theory (CMT, Lakoff & Johnson, 1980), (ii) cinematic metaphoricity (Coëgnarts & Kravanja, 2012a, 2012bKappelhoff & Müller, 2011;Müller & Kappelhoff, 2018;Ortiz, 2014), and (iii) multimodality in films (Bagchi, 2011;Bateman & Wildfeuer, 2014;Forceville, 2009Forceville, , 2017Kress, Jewitt, Ogborn & Tsatsarelis, 2000;Norris, 2013). ...
... It must be noticed that there is contradiction not only about what a mode is but also in the way of labeling the term: mode, semiotic mode, modality, articulatory modality, sensory modality, module, or even channel. The definition of what a mode is and entails is a rather controversial and difficult task (Klug & Stöckl, 2014;Jewitt, 2013;Norris, 2013), and its elucidation is crucial for the understanding of multimodal data. This section presents the notion of mode according to its three main approaches: (i) modes as perception channels, (ii) modes as semiotic gestalts, and (iii) modes as communicative means. ...
... I suggest, in line with Norris (2013), that our senses, our perception channels, build the basis for a mode of communication to develop. We perceive the world (whether real or unreal), through our sensory channels (sight, hearing, taste, touch, and smell). ...
The analysis and identification of figurative language is one of the largest research areas in cognitive linguistics, and metaphor is one of these tropes. This thesis develops The Filmic Metaphor Identification Procedure (FILMIP), a modified and dynamic version of another method for the identification of metaphors in still pictures (VISMIP, Šorm & Steen, 2018), and presents its application to a corpus of several TV commercials where distinct communicative modes interact to construe filmic metaphors in a cross-modal fashion. The method is then tested for reliability and validity, obtaining highly significant results. FILMIP offers a valuable contribution not only to metaphor scholars but also to researchers focused on other fields of study such as multimodality, discourse analysis, communication, branding, or even film theory.
-----------------------------------------
DOI TO CITE THIS THESIS: http://dx.doi.org/10.6035/14110.2019.23206
... Simultaneously, a methodological framework prioritizing mediation, given the inextricable effect of mediating technologies is directly congruent with the technological specificities of these new interaction types. While a complete explication of Multimodal Mediated Theory and Multimodal Interaction Analysis (Norris 2004(Norris , 2006(Norris , 2009(Norris , 2011(Norris , 2013Pirini 2014) is beyond the scope of a single article, below I detail some central methodological tools which are directly pertinent to the discussion which follows. ...
... Approaching the study of familial interaction through contemporary video-conferencing software and technologies from a Multimodal Mediated Theory perspective (Norris 2004(Norris , 2011(Norris , 2013(Norris , 2014Author 2013;Author et al. 2015); and employing Multimodal (Inter)action Analysis (MIA) (Norris 2004(Norris , 2006(Norris , 2009(Norris , 2011(Norris , 2013Pirini 2014) as a methodological framework has a number of distinct advantages. MIA provides an explicit framework for the analysis of verbal and non-verbal communication without overtly prioritising any single mode a priori (Norris 2004(Norris , 2011. ...
... Approaching the study of familial interaction through contemporary video-conferencing software and technologies from a Multimodal Mediated Theory perspective (Norris 2004(Norris , 2011(Norris , 2013(Norris , 2014Author 2013;Author et al. 2015); and employing Multimodal (Inter)action Analysis (MIA) (Norris 2004(Norris , 2006(Norris , 2009(Norris , 2011(Norris , 2013Pirini 2014) as a methodological framework has a number of distinct advantages. MIA provides an explicit framework for the analysis of verbal and non-verbal communication without overtly prioritising any single mode a priori (Norris 2004(Norris , 2011. ...
This article provides a preliminary answer to exactly why video-conferencing is evaluated as better than traditional telephony for long-distance familial interaction by allocating analytical attention to the showing of objects during interaction. While it is acknowledged that ‘showing’ constitutes an interactive move less contingent on linguistic maturation, more importantly, the showing of objects, artefacts or entities during video-conferencing interactions exemplifies an agentive and volitional production of identity elements on behalf of young children. Thus, while some have pointed to shortcomings of conversation-like activities mediated by video-conferencing in favour for more activity-driven tasks, I make a case for drawing upon pre-existing components of the material surround as a means to more comprehensively and longitudinally engage younger children in video-conferencing interaction.
... In this article I utilise multimodal (inter)action analysis (Norris 2004a(Norris , 2011(Norris , 2013 to conduct an analysis of the beginning of a tutoring session to show how tutor and student come to co-produce a higher-level action. I utilise several tools from multimodal (inter)action analysis to conduct my analysis, including higher and lower-level actions, modal density and the foreground/background continuum of attention/awareness, semantic/pragmatic means (Norris 2004a) and modal configurations (Norris 2009). ...
... In addition I determine how these actions produce an interactive substrate, which provides a basis for more fleeting material intersubjectivity. Building on previous work in intersubjectivity (Matusov 1996(Matusov , 2001 and utilising the theoretical basis of multimodal (inter)action analysis (Norris 2013;Scollon 1998;Wertsch 1998), these theoretical/methodological tools expand upon multimodal (inter)action analysis as a framework for analysing social interaction. ...
... Methodology I utilise concepts from multimodal (inter)action analysis (Norris 2004a(Norris , 2011(Norris , 2013 to develop a theoretical and methodological tool for the analysis of intersubjectivity. As I pointed out above, multimodal (inter) action analysis builds on mediated discourse analysis (Scollon 1998(Scollon , 2001Wertsch 1998) and takes the mediated action as its unit of analysis. ...
Researchers seeking to analyse how intersubjectivity is established and maintained face significant challenges. The purpose of this article is to provide theoretical/methodological tools that begin to address these challenges. I develop these tools by applying several concepts from multimodal (inter)action analysis to an excerpt taken from the beginning of a tutoring session, drawn from a wider data set of nine one-to-one tutoring sessions. Focusing on co-produced higher-level actions as an analytic site of intersubjectivity, I show that lower-level actions that co-constitute a higher-level action can be delineated into tiers of materiality. I identify three tiers of materiality: durable, adjustable and fleeting. I introduce the theoretical/methodological tool tiers of material intersubjectivity to delineate these tiers analytically from empirical data, and show how these tiers identify a multimodal basis of material intersubjectivity. Building on this analysis I argue that the durable and adjustable tiers of material intersubjectivity produce the interactive substrate, which must be established in order for actions that display fleeting materiality to produce intersubjectivity. These theoretical/methodological tools extend the framework of multimodal (inter)action analysis, and I consider some potential applications beyond the example used here.
... Since the original development, empirical enterprises employing Multimodal (Inter)action Analysis have generated a wealth of insights both theoretically in the ongoing development of Multimodal Mediated Theory (Geenen, 2013a(Geenen, , 2014Norris, 2013) and analytically with the development of various methodological tools. Multimodal (Inter)action Analysis as an integrated framework provides tools for analysing the attention/awareness of social actors in any specific site of engagement through modal density (Norris, 2004(Norris, , 2011, the production of modal configurations (Norris, 2016(Norris, , 2017Pirini, 2016Pirini, , 2017 which help analysts discern the salience of particular modes during any segment of interaction and methods for the analysis of actions as related across time and space to various scales of action (Norris, 2017). ...
... For a complete and comprehensive guide to analytical protocol and transcript generation, see Norris (2018). We could not deal comprehensively with topics like the mediational interrelationship (Geenen, 2013b(Geenen, , 2014, the notion of practice (Norris, 2004(Norris, , 2011Scollon, 2001) or other developments, such as what constitutes a communicative mode and how they develop and are during ontogenesis (see Norris, 2013). ...
Multimodal (Inter)action Analysis was developed to study social interaction based upon the theoretical notion of mediated action. Building on this core concept, Multimodal (Inter)action Analysis includes several theoretical/methodological tools. These tools facilitate analysis which moves flexibly between micro-level moments of interaction and macro-level practices and discourses. In this chapter, the application of mediated action to multimodal analysis is discussed, before the central theoretical/methodological tools are introduced. Tight links are made between the tools used in Multimodal (Inter)action Analysis and the core theoretical tenets, to support robust multimodal interaction research.
... Coffee and cinnamon buns On the deployment and intersemiotic potential of smell Our sense of smell is mediated by complex physicochemical and biological interactions. This sense is part of the material basis for what we might call a mode of smell, a socially mediated semiotic system (van Leeuwen 2005, Norris 2013, Fryer 2013. The resources of such modes or systems, to the extent that they are recognized as such, are rarely if ever deployed alone, but as part of a multisemiotic arrangement (van Leeuwen 2005, Kress 2010, Norris 2013. ...
... This sense is part of the material basis for what we might call a mode of smell, a socially mediated semiotic system (van Leeuwen 2005, Norris 2013, Fryer 2013. The resources of such modes or systems, to the extent that they are recognized as such, are rarely if ever deployed alone, but as part of a multisemiotic arrangement (van Leeuwen 2005, Kress 2010, Norris 2013. In this paper, I discuss the meaning potential of smell by considering how social actors produce, manipulate, combine, and organize certain smells to create sophisticated 'messages' that can be interpreted and evaluated by others. ...
Full paper here: https://prezi.com/jvyyxpwcsgil/coffee-and-cinnamon-buns/
Abstract
Our sense of smell is mediated by complex physicochemical and biological interactions. This sense is part of the material basis for what we might call a mode of smell, a socially mediated semiotic system (van Leeuwen 2005, Norris 2013, Fryer 2013). The resources of such modes or systems, to the extent that they are recognized as such, are rarely if ever deployed alone, but as part of a multisemiotic arrangement (van Leeuwen 2005, Kress 2010, Norris 2013). In this paper, I discuss the meaning potential of smell by considering how social actors produce, manipulate, combine, and organize certain smells to create sophisticated 'messages' that can be interpreted and evaluated by others. As an example, I examine the case of a real-estate agent's use of specific smells, created among other things by preparing coffee and cinnamon buns, as part of a house viewing. I
consider the potential meanings encoded by those smells in that particular context of situation. I also discuss the relative semantic weight of those resources as part of a multisemiotic event or activity that includes the co-deployment of verbal, visual, and spatial resources. By considering the relative amounts or degrees of meaning potential instantiated through those different semiotic systems, I demonstrate how smell might be deployed to make meanings that are potentially complementary to as well as incongruent or inharmonious with those committed verbally, visually, and spatially (see, for example, Hood 2008, Martin 2011, and Painter, Martin, and Unsworth 2013 on 'commitment'). Reasons for making those particular semiotic choices and the kinds of values social actors might assign to them will also be discussed.
... O terceiro conceito -influenciado por ideias de Vygotsky, Leont'ev e Luria -é o de ação mediada, que abrange três níveis de interação: (1) ações de nível superior, que abarcam; (2) ações de nível inferior, que são as menores unidades de significado de dado modo; e (3) ações congeladas, que são congeladas no tempo e podem integrar conversas futuras (Norris, 2004(Norris, , 2011. Norris (2013) ilustra que a escolha de certo perfume pode requerer uma ação de nível superior: cheirar perfumes. Essa ação pode, ainda, congregar distintas ações de nível inferior, como a movimentação dos braços e das mãos em direção ao nariz que comporia o modo gesto. ...
Este artigo se propõe a analisar o processo de articulação de distintas semioses para a (co)construção de sentidos na leitura e na produção textual com alunos/as autistas e não autistas em uma turma de educação inclusiva de espanhol como língua adicional. Constatamos, a partir de nossa orientação teórico-metodológica, que o estudante autista negociou semioses verbais e não verbais na (co)construção de sentidos com os/as colegas nas tarefas de leitura e de produção textual analisadas.
... Forceville and Urios-Aparisi (2009) supports this claim and adheres to it in what he defines as a multimodal metaphor. According to the author, a multimodal metaphor is a metaphor in which the target and the source domains are represented in different communicative acts, semiotic resources (Pun 2008), perception channels (Norris 2013), or "modes" (Forceville and Urio-Apaisi 2009). These modes are mainly spoken and written language, visuals, sound, gestures, smell, taste, and touch. ...
Research on multimodal communication is complex because multimodal analyses require methods and procedures that offer the possibility of disentangling the layers of meaning conveyed through different channels. We hereby propose an empirical evaluation of the Filmic Metaphor Identification Procedure (FILMIP, Bort-Mir, L. (2019). Developing, applying and testing FILMIP: the filmic metaphor identification procedure, Ph.D. dissertation. Universitat Jaume I, Castellón.), a structural method for the identification of metaphorical elements in (filmic) multimodal materials. The paper comprises two studies: (i) A content analysis conducted by independent coders, in which the reliability of FILMIP is assessed. Here, two TV commercials were shown to 21 Spanish participants for later analysis with the use of FILMIP under two questionnaires. (ii) A qualitative analysis based on a percentage agreement index to check agreement among the 21 participants about the metaphorically marked filmic components identified on the basis of FILMIP's seven steps. The results of the two studies show that FILMIP is a valid and reliable tool for the identification of metaphorical elements in (filmic) multimodal materials. The empirical findings are discussed in relation to multimodal communication open challenges.
... Then, MIA moves on from MDA to multimodal mediated theory, noting that mediated actions appear on different levels (lower-level, higher-level and frozen) (Norris, 2004a). Multimodal mediated theory further suggests that modes are systems of mediated actions (Norris, 2013) and discourses are practices (mediated actions with a history) with an institutional and/or ideological dimension (Norris, 2020). When looking at Figure 1, we see how the philosophical and the theoretical strata seemingly sit above method and methodology. ...
This paper presents a concise introduction to Multimodal (inter)action analysis (MIA), which began to be developed in the early 2000s in tandem with technological advances for visual qualitative research. By now, MIA has grown into a fully-fledged research framework, including multimodal philosophy, theory, method and methodology for the study of human action, interaction and identity. With systematic phases from data collection to transcription (including transcription conventions) and data analysis, this framework allows researchers to work in a data-driven and replicable manner moving past common interpretive paradigms (Norris 2019, 2020).
... Goodwin & Cekaite (2018), for instance, illustrate in their study on family communication how bodily behavior is an important contributor to shaping the interactional order and constituting family members' situated relationships. In another social setting, Norris (2013) investigates the multimodal aspects of everyday regularities such as putting on perfume and getting dressed. In doing so, she illustrates how such "higher level actions" are inherently made up of an abundance of chained "lower-level actions" and argues that these multimodal dynamics are essential in all communication and (inter)action (ibid: 55). ...
In this article, we investigate the use of social media in contemporary family interaction from a linguistic ethnographic perspective. Inspired by Auer’s (1998) work on code-switching in conversation, we study how family members choose and sometimes alternate between digitally mediated and face-to-face modes of communication in various family settings. Based on ethnographic observations, the participants’ metapragmatic reflections, and their interactional orientations to mode choices, we show how such choices serve social and metapragmatic functions in the interaction between family members who are present in the same house or even in the same room. Accordingly, we argue in favor of situating peoples’ polymedia repertoires in a broader framework of communicative repertoires.
... We implemented a mixed-method study [73] by combining methods that complement one another and shed light on important questions in our research [74][75][76][77][78][79]. We had a multimodal perspective for the analysis of our data (questionnaires, interviews, observations and field notes) [80][81][82][83]. ...
There are both culturalist and structuralist approaches to the integration of the second-generation immigrants into mainstream society. These approaches focus on cultural, linguistic and socioeconomic assimilation. Successful societal membership is associated with psychosocial adaptation, hybrid identity, selective acculturation or biculturalism, which is an individual's adjustment to new psychological and social conditions. Individual identity is related to the sense of belonging, integration and engagement in the current space. Self-identity is fluid and flexible; it comprises individual and collective identity, habitus or unconscious identity, agency and reflexivity, which is re-evaluated and adjusted throughout the life trajectory of a migrant and connected to citizenship and solidarity. This study investigated heritage language use, maintenance and transmission, as well as language and cultural identity and social inclusion of second-generation immigrants in Cyprus with various L1 backgrounds. The analysis of the data (e.g. questionnaires, interviews, focus group discussions, observations) showed that second-generation immigrants have a hybrid language and cultural identity, as well as multifarious perceptions regarding citizenship, inclusion and belonging. These immigrants try to assimilate to the target society, but at the same time they have a strong link with the community of residence, their L1 country and their heritage or home language. The participants also use mixed/multiple languages at home and elsewhere.
... que fazem parte de seu potencial semiótico(KRESS, 2010).Com relação à segunda vertente teórica (a análise multimodal interacional), retomamos, sob essa perspectiva, o conceito de modo, que consiste em um sistema de ação mediada, fruto de usos sociais, históricos e culturais, que se vincula fortemente ao contexto de produção em decorrência de nossa atuação no mundo(JEWITT (2011(JEWITT ( [2009NORRIS, 2013). Tal sistema prevê ações menores (nível mais baixo) e ações maiores (nível mais alto)(NORRIS, 2004(NORRIS, , 2006(NORRIS, [2009), que poderiam ser representados, por exemplo, por uma interação face a face (nível mais alto), em que estão presentes gestos, mudanças posturais, enunciados(NORRIS, [2009). ...
This production is another collective work resulting from
academic-scientific investigations of professors linked to the
International Scientific Research Project (PICNAB), network
which brings together researchers from the universities of
Nantes (France), Aveiro (Portugal) and Brasília (Brazil), and counts,
since 2016, with the support of the Research Support Foundation of the
Government of the Federal District (FAP-DF) and the CAPES-Print /
University of Brasilia. The Network's main objectives are to build artistic (literary and theatrical), linguistic research and pedagogical that envision new socio / intercultural practices before the challenges of contemporary reality, which has limits
tenuous, migratory movements and fluid identities. Thus,
researchers and researchers study issues concerning the
teacher training that will deal with the education of children
and young members of an educational system that needs to be
rethought and resized in their teaching practices, both in terms of
Brazil, as well as in Portugal and France, countries that experience the day
day of this new reality. In this sense, studies add not only
undergraduate students, but also graduate students (master's
and doctorate), linked to the programs of the three universities.
... While we have seen an increased number of EMCA studies adopting a multimodal perspective on interaction, multimodality is central to other influential research paradigms investigating discourse and interaction (see e.g., contributions in Jewitt 2009). For example, multimodal discourse analysis van Leeuwen 1996, 2001) and multimodal mediated theory (Norris 2004(Norris , 2013 investigate multimodality in a variety of texts, images and other expressions that transcend verbal language. ...
This article investigates how a significant part of the learning process within show dance classes is performed, namely multimodal demonstrations. In particular, it focuses on how the mirror is used as a tool during demonstrations. The data consist of video recordings and ethnographic fieldwork among young adults practicing show dance. Three single cases undergo in-depth analysis using an EMCA (ethnomethodology conversation analysis) approach. The analysis shows that the understanding of a multimodal demonstration is watched into being and that the mirror functions as an extension of the teacher. In addition, the use of the mirror changes over the course of the ongoing demonstration. The participants agree on how to use the mirror and that there is a symmetrical order for how the mirror is used, which means that the responding participant follows the way of using (or not using) the mirror that is introduced by another participant. As such, this article contends that use of the mirror is a well-established practice. The article concludes by highlighting the positive effect on the learning situation when a mirror is used for demonstration in show dance classes.
... The assumption is, for example, that a gesture or gaze can play a superordinate or equal role to the mode of language in interaction. Modes and the way they are used come into being, emerge and are strongly situated; however, they have history through the experiences participants have had in their own networks or through their joint experiences (Norris, 2013). Norris (2013, pp. ...
This paper attends to languaging in the context of visually oriented communities of sign language users through the concept of chaining. I define chaining as the patterned, routine ways of interlinking different linguistic and multimodal elements. The goal of this paper is to discuss the concepts of chaining, languaging and remodalisation through practical examples. The material for the paper comes from two ethnographic research projects which examine the use of multiple signed and spoken languages via different modalities in educational contexts. Both projects take a point of departure in ethnography and multimodal approaches towards interaction stemming from mediated discourse analysis. By examining the multimodal and multilingual practices, this paper aims at providing teachers and language learners with tools with which to reflect upon complex and often unnoticed resources used in everyday interaction.
... The methodological approach employed herein originally derives from the adoption of a Vygotskyan approach to genetic development and the incorporation of Wertsch's mediated action theory as theoretical precursors to Scollon's (2001) mediated discourse theory/analysis and Norris's extension of these principles in Multimodal (Inter)action Analysis (2004, 2011, 2013a, 2013b) and Multimodal Mediated Theory (Norris 2013a(Norris , 2013bGeenen 2013). ...
In this article, I detail incremental microgenetic alterations in the development of one particular socio-interactive aptitude: making a relevant interactive contribution. Taking heed of Clark’s (2014) call for the need to reorient our attention to investigate the pragmatics of interaction by accounting for the multiple communicative modes through which this is acccomplished I detail the ways in which parental facilitation and a flexible participatory configuration, made possible by video-conferencing technology, create conditions enabling the agentive re-introduction of a psycho-socially relevant topic. Paramount are the ways in which residual interactive specificities in introduction, co-production and multimodal configurations re-manifest suggesting a more symbiotic relationship between traditional notions of ‘message’ and ‘production’. During the microgenesis of interactive aptitudes, children are not just learning what constitutes psycho-socially relevant topoi, they also acquire an understanding of exactly how to make the contribution through multimodal ensembles.
... The analysis will also take into account the evaluative properties of the texts under scrutiny (Hunston and Thompson 2000;Bednarek 2009aBednarek , 2009b, by investigating the linguistic markers signalling the legal appraisal of claimants' stories and the values emerging from how immigration judges handle these cases. This paper takes a multimodal perspective (Norris, 2004(Norris, , 2011a(Norris, , 2012a(Norris, , 2013a(Norris, , 2013b(Norris, , 2013cGeenen, 2013aGeenen, , 2013bGeenen, Norris & Makboon, 2015;Makboon, 2015;Pirini, 2014Pirini, , 2015Pirini, , 2016, building upon mediated discourse analysis (Scollon, 1998Norris and Jones, 2005a, b) to examine how identities are produced in family videoconferencing (inter)actions. ...
Book of Abstracts of the 6th Critical Approaches to Discourse Analysis across Disciplines Conference (CADAAD 2016)
... According to Norris, the regularities of the modes are sometimes closer to the social actor and sometimes closer to the mediational means, but modes do not occur without the social actor. Consequently, the focus of the study is on how the speakers act, and how these actions depend on the modal development (Norris 2013). In the case of this paper, for example, I analyze the modes of spoken language (and its intonation), gesture, posture and gaze. ...
This paper is part of a larger scale project where I explore the structure of academic lecture. The focus of the study here presented is to investigate the structure and organization of a university lecture through the introduction of new topics. One of the tools traditionally referred to as an organizer of discourse is metadiscourse (Crismore et al. 1993. Metadiscourse in persuasive writing: A study of texts written by American and Finnish university students. Written Communication, 10:39–71; Vande Kopple. 1985. Some exploratory discourse on metadiscourse. College Composition and Communication, 36(1):82–93). Although metadiscourse has been studied from a wide range of perspectives (Hyland. 2005. Metadiscourse: exploring interaction in writing. London, England: Continuum), these analyses have most of the time been conducted from a purely linguistic point of view and neither the speaker as a social actor nor metadiscourse as part of a multimodal interaction are taken into account. That being so, the aim of this study is to explore the role played by
... Note that attention to handling objects does not necessarily mean that authors have paid attention to tactility. Indeed, multimodal interaction is multi-sensory, but, as Mondada (2016, p. 355) pointed out, the visual turn in multimodality research (made possible by video recording interactions) has led to (or obscured) 'another form of reductionism, that of embodiment to audible-visible features': touching, tasting, smelling are understudied in the field of multimodality research (see also Norris, 2013). Multisensoriality has received more attention within linguistic landscaping (see Pennycook, 2017), a field of study that focuses less on language in interaction. ...
This paper presents a critical examination of key concepts in the study of (signed and spoken) language and multimodality. It shows how shifts in conceptual understandings of language use, moving from bilingualism to multilingualism and (trans)languaging, have resulted in the revitalisation of the concept of language repertoires. We discuss key assumptions and analytical developments that have shaped the sociolinguistic study of signed and spoken language multilingualism as separate from different strands of multimodality studies. In most multimodality studies, researchers focus on participants using one named spoken language within broader embodied human action. Thus while attending to multimodal communication, they do not attend to multilingual communication. In translanguaging studies the opposite has happened: scholars have attended to multilingual communication without really paying attention to multimodality and simultaneity, and hierarchies within the simultaneous combination of resources. The (socio)linguistics of sign language has paid attention to multimodality but only very recently have started to focus on multilingual contexts where multiple sign and/or multiple spoken languages are used. There is currently little transaction between these areas of research. We argue that the lens of semiotic repertoires enables synergies to be identified and provides a holistic focus on action that is both multilingual and multimodal.
... Gangsta Granny's brain also constantly fails throughout the text: 'I am going to put another cardigan on, she said, even though she was already wearing two'; 'She probably doesn't understand what you're saying half the time' . Also, she smells of cabbage -smell as a semiotic resource is invoked here as negative appreciation/evaluation. Norris (2013) points out that 'smell' is a semiotic mode and words that refer to the biological capacity of human beings to smell afford different meaning potentials. In our case, Gangsta Granny would be interpreted differently if she smelt of roses. ...
In everyday life it is now common to find our actions linked to sound, especially using technology, such as when we use mobile devices, or operate more recently manufactured cars, technology in the workplace or simply in an elevator. While we may attend little to these noises, like any semiotic resource, they can communicate very specific meanings and carry ideologies. In this paper, using multimodal critical discourse analysis, we analyse the sounds and music in two proto-games that are played on mobile devices: Genie Palace Divine and Dragon Island Race. While visually the two games are highly gendered, we show that an investigation of the sounds players can make during gameplay reveals very specific insights into the ways that sound positions players in the world. In each game we ask: what is foregrounded and what backgrounded as regards sound? Sound can be used to signal the personal and impersonal and specific kinds of social relations which, we show, is highly gendered. It can also signal priorities, ideas and values, which in both cases, we show, relate to a world where there is simply no time to stop and think.
... Multimodal (inter)action analysis (Norris 2004(Norris , 2009(Norris , 2011a(Norris , 2011b(Norris , 2013(Norris , 2014(Norris , 2015 originating from mediated discourse analysis (Scollon 1998(Scollon , 2001 is based in the sociological interest of humans acting in the world that we find in the work of Goffman (1963); incorporates the interest in intercultural interaction that we find in the work of Gumperz (1982); includes an interest in power in interaction that we find in the work of Wodak (1989); delves into the microanalysis of interaction that we find in the work of Tannen (1984), Schiffrin (1987), or Hamilton (1998); has a strong interest in applied linguistics that we find in the work of van Lier (1996); is strongly influenced by socio-cultural psychology as we find in the work of Wertsch (1998); and is grounded in social semiotic thought that we find in the writings of van Leeuwen and Kress (van Leeuwen 1999;Kress 2000;Kress andvan Leeuwen 1998, 2001). With these foundations, multimodal (inter)action analysis (Norris 2004(Norris , 2011 has developed into a strong theoretical framework with an abundance of methodological tools (Norris 2004(Norris , 2009(Norris , 2011(Norris , 2013a(Norris , 2013b(Norris , 2014Geenen 2013;Makboon 2015;Pirini 2015Pirini , 2016) that make the analysis of (always) multimodal (inter)action possible, opening up research into new and promising directions. ...
This article presents theoretical concepts and methodological tools from multimodal (inter)action analysis that allow the reader to gain new insight into the study of discourse and interaction. The data for this article comes from a video ethnographic study (with emphasis on the video data) of 17 New Zealand families (inter)acting with family members via skype or facetime across the globe. In all, 84 social actors participated in the study, ranging in age from infant to 84 years old. The analysis part of the project, with data collected between December 2014 and December 2015, is ongoing. The data presented here was collected in December 2014 and has gone through various stages of analysis, ranging from general, intermediate to micro analysis.
Using the various methodological tools and emphasising the notion of mediation, the article demonstrates how a New Zealand participant first pays focused attention to his engagement in the research project. He then performs a semantic/pragmatic means, indicating a shift in his focused attention. Here, it is demonstrated that a new focus builds up incrementally: As the participant begins to focus on the skype (inter)action with his sister and nieces, modal density increases and he establishes an emotive closeness. At this point, the technology that mediates the interaction is only a mundane aspect, taken for granted by the participants.
Objective: To evaluate inferential and macrostructural ability through three types of multimodal artifacts: verbal, graphic-mathematical, and complete text (which integrates all semiotic modes). Theoretical Framework: This section explores the key concepts of multimodality theory, as well as the notions of inferences and multimodal ability, which are fundamental to addressing the research problem. The main theories and concepts supporting the research are presented, providing a solid foundation for understanding the study's context. Method: A mixed methodology was employed, combining quantitative and qualitative approaches. The quantitative phase adopted a correlational ex post facto design, while the qualitative phase was carried out through a focus group. The quantitative study followed a factorial experimental between-subjects design, considering two dimensions: the participants' professional program and the semiotic systems present in the comprehension test. Results and Discussion: It was concluded that the type of multimodal artifact does not significantly affect the efficient identification of inferences or macrostructural reduction. This suggests that students exhibit a low level of multimodal ability. Research Implications: The practical and theoretical implications of the research are analyzed, providing insights into how the results can influence or be applied to practices related to multimodal comprehension. Originality/Value: The research highlights that universities, in general, do not pay sufficient attention to multimodality, which negatively impacts students' multimodal competencies.
Multimodal (inter)action analysis offers a powerful and robust methodology for the study of action and interaction between social actors, their environment, and the objects and tools within. Yet its implementation in the analysis of synchronous multimodal online data sets, e.g. (inter)actions via videoconferencing, is limited. Drawing on our research in understanding teacher-learner (inter)actions in instruction-giving fragments in synchronous multimodal online language lessons, we describe and illustrate the ways in which we adapted and extended some of the methodological and analytical tools. These include (1) the use of a grounded theory approach in delineating and identifying higher-level actions, (2) the embodiment and disembodiment of frozen actions, (3) electronic print mode, (4) semiotic lag, (5) semiotic (mis)alignment, (6) modal density (mis)alignment, and (7) how modal density can be achieved by brisk modal shifts in addition to through modal intensity and complexity. We conclude by a call for further educational research in online teaching platforms using the framework to have richer understandings of the (inter)actions between social actors with particular roles and identities (teachers-learners), their environment, and the objects and tools within, which bring their “own material properties, feel and techniques of use, affordances and limitations” (Chun, Dorothy, Richard Kern & Bryan Smith. 2016. Technology in language use, language teaching, and language learning. The Modern Language Journal 100. 64–80: 65).
Red Chinese movies depict heroic individuals who make significant contributions to the nation, and these films play a vital role in constructing the national image of new China. This paper explores the images of heroic individuals in red Chinese movies using multimodal action theory. The findings indicate that these individuals are characterized by their selfless sacrifices for their country, fearless perseverance, embodiment of international communist ideals, and unwavering devotion and warmth. The study also reveals that facial expressions, body movements, and language serve as critical mediational means in shaping these characters, while other forms of mediation are underutilized. Recommendations include employing more mediational means to create more nuanced and complex characters in future red Chinese movies.
Based on the multimodal discourse analysis, the contribution attempts to describe the genre of online video tutorial, focusing primarily on the involved modes and their interplay. The research has been done on the corpus consisting of online video tutorials on differential and integral calculus (e.g. first-order linear differential equations) which are available on the Internet and made by native speakers of English. In order to comment on differences between online and offline practices a referential group of standard lectures on infinitesimal calculus has been investigated as well. A case study focusing on forms of explication from the multimodal perspective has been carried out; seven presentational formats were compared both quantitatively and qualitatively to find out how various communication techniques are employed.
Dooly (2017, p. 169) defines telecollaboration in education as “the use of computer and/or digital communication tools to promote learning through social interaction and collaboration, thus moving the learning process beyond the physical boundaries of classrooms.” As digital communication technologies advance, newer and more sophisticated cutting-edge ICT tools are being used for telecollaboration, including virtual reality (VR). Researchers have applied different models and approaches of multimodal analysis to understand the specific features of VR on students' language learning (Dubovi, 2022; Friend & Mills, 2021) and intercultural communication (Rustam, et al., 2020). Nevertheless, very little has been done to look into language teacher telecollaboration via VR technologies. This present study recruited student teachers of an additional language (LX) (Dewaele, 2017) from different geographical locations and cultural backgrounds to participate in a project aiming at cultivating the participants' critical views on LX teaching and intercultural communication skills. The participants interacted and discussed LX teaching/learning issues in VR environments. Their interactions were video recorded and analyzed. By applying Multimodal (inter)action Analysis (MIA) (Norris, 2014) as the analytical framework, this study systematically unpacked the thematical saliencies and significant moments of the participants' intercultural interaction in the three VR meetings. Based on the findings, suggestions and caveats for future designing and researching intercultural telecollaboration in VR environments are provided.
Multimodal discourse analysis is an approach to discourse which focuses on how meaning is made through the use of multiple modes of communication as opposed to just language. The concept of “mode” in multimodal discourse analysis should not be confused with the concept of “modality” in grammar (which refers to the expression of possibility and obligation in language), or with the concept of “mode” in Halliday's tripartite model of context, in which mode refers to the “channel” of linguistic communication. Some scholars, while influenced by Halliday's work, developed methods of analysis that sharply diverged from both social semiotics and systemic functional analysis. Among the most notable has been the Genre and Multimodality framework (sometimes abbreviated GeM), developed chiefly by John Bateman. Another important tradition in multimodal discourse analysis has focused more on the analysis of face-to-face interaction, drawing its inspiration from work in sociology, anthropology, and psychiatry. More recent trends in multimodal discourse analysis are pushing scholars to consider more fully the complex ways different modes interact in communication, including how meanings change as they move across modes.
The flexible and pedagogically informed use of technology for the service of socioculturally oriented perspectives on language learning has been an important goal in education for some time. However, change in pedagogical approaches and practices has been slow in schools. Neither has technology become an established resource in language teacher education. This study sheds light on how pre-service teachers in northern Finland design an online project for English promoting learners’ meaningful participation in language activities, as well as multimodal and intermedial practices. Nexus analysis is used to examine the research materials gathered during the design process. The findings highlight competing perspectives arising from the pre-service teachers’ print-oriented and teacher-led experiences of language learning at school, and from their active intermedial practices online. The chapter proposes a way to utilize a design-based approach to developing language teacher education and envisioning practices for future language pedagogy.KeywordsLanguage teacher educationDesign-driven approachNexus analysisDiscourse
Using multimodal (inter)action/conversation analysis, the present contribution inventories the repertoire of higher-level actions that constitute musical instruction in orchestra rehearsals. The study describes the modal complexity of the instructional actions as built from a varied combination of speech, gesture, gaze, vocalizing and body posture/movement. A high modal intensity of speech and vocalizing is explained with recourse to their contextually useful modal reaches. While some modes, like vocalizing and body posture appear to be action-specific, others turn out to be pervasive default modes. Besides modal intensity, the study also attends to the transitioning between higher-level actions through gaze and the role of the score as frozen action. The analyses help demystify orchestra rehearsals as a special type of professional communicative interaction, which builds on a rich multimodal texture motivated by recurring instructional functions. The methodological rationale demonstrated will be suited to exploring the social variation of instructional interaction in orchestra rehearsals.
Multimodal interaction analysis is a holistic methodological framework that allows the analyst to integrate the verbal with the nonverbal, and to integrate these with material objects and the environment as they are being used by individuals acting and interacting in the world. In short, multimodal interaction analysis allows a researcher to study real people interacting with others, with technology, and with the environment. The point of view taken in multimodal interaction analysis is that all actions in fact are interactions and that all of these (inter)actions are linked to people (referred to as “social actors” in multimodal interaction analysis)—no matter whether you are investigating the real or the virtual world, someone buying ice cream, or using a software program. Multimodal interaction analysis is carried out through a series of analytical phases and steps. Researchers utilize multimodal transcription conventions and analytical tools, which are theoretically founded, allowing for replicability of analyses and reliability of findings. Multimodal interaction analysis allows the integration of all communicative modes, where communicative modes are defined as systems of mediated actions. In this view, all modes of communication together build one system of communication, which comes about through actions and interactions that people produce. This coherent framework and process of analysis can be taught and learned and is proving useful for research examining what people do, how they communicate, and how they interact.
In this paper, we present an in depth study of the interplay between various semiotic modes in the specific instructional setting of a scientific café. We analyzed the multimodal performance of five female instructors delivering a monologue explanation during instruction on the following dimensions: speech, gesture (hand gestures, head orientation and gaze) and use of written didactical material. Results first point out the crucial role played by referential hand gesture together with gaze-body behavior both in representing new concepts ( conceptual mediation ) and in building bridges between information displayed in several modes ( semiotic mediation ). They also show cross-individual differences in instructors’ multimodal performance, that we propose to interpret as three diverse modes of mediating knowledge, guiding being the only one providing both conceptual and semiotic mediation.
This article re-examines the often misrepresented role of Okinawan agency by focusing on the divergence between mainstream framings of Okinawa, actual policies directed towards the Ryūkyū Islands and the sociopolitical reality on these outlying islands. In so doing, it interrogates the various narratives of Okinawa and the key terms that have articulated them in the post-reversion era. It thereby adds explanatory power to extant structuralist and critical literatures, which have tended to suffer from monolithic descriptions of structural power and polemic approaches to American and Japanese governance of the islands. Specifically, by analyzing a series of illustrative issue areas such as sexual and economic exploitation, environmental protection and military security, the article uses an adapted form of critical discourse analysis (CDA) to trace how framings and policy have shifted since reversion to Japanese rule. This concentrates primarily on prime-ministerial statements from the National Diet and other, mostly Japanese-language, materials relating to Okinawa’s governance. These are contrasted with short case studies highlighting the disconnection between rhetoric and reality. Ultimately, the evidence points to a decoupling of mainstream narratives from the on-the-ground reality. The article thereby provides a nuanced understanding and expression of Okinawa’s complex interests and agency.
Defining and understanding a positive and inclusive Māori or Pacific ethnic identity is difficult. Yet doing so is necessary in order to enhance the wellbeing of Māori and Pacific people in New Zealand. This paper argues that analysing the art of a Māori and Samoan visual artist using frozen actions as the analytical tool, reveals their fluid ethnic identity. Actions such as hanging the art, producing the art and researching the art are embedded as frozen actions in the art itself. These identity telling actions reveal a fluid ethnic identity, a positive and inclusive ethnic identity which combines ideas about the social environment and ethnicity.
The present study belongs to an extensive project that explores how academic knowledge is mediated through new generic structures and publishing formats and provides data comprising research video articles from JoVE, the international Journal of Visualized Experiments. In order to deal with the implications of video formats in web contexts upon academic user engagement and knowledge building processes, we adopt a multimodal (inter)action approach in our analysis. We show how exploiting the complex hypermodal configurations contribute to changes in the balance of various types of knowledge and to the potential building of new ones. We also show that by embedding the video in a hypermodal context urges academic users to increase their engagement with the article in unprecedented ways.
This book merges recent trends in game studies and multimodal studies to explore the relationship between the interaction between videogames’ different modes and the ways in which they inform meaning for both players and designers. The volume begins by laying the foundation for integrating the two disciplines, drawing upon social semiotic and discourse analytic traditions to examine their relationship with meaning in videogames. The book uses a wide range of games as examples to demonstrate the medium’s various forms of expression at work, including audio, visual, textual, haptic, and procedural modes, with a particular focus on the procedural form, which emphasizes processes and causal relationships, to better showcase its link with meaning–making. The second half of the book engages in a discussion of different multimodal configurations and user generated content to show how they contribute to the negotiation of meaning in the player experience, including their role in constructing and perpetuating persuasive messages and in driving interesting and unique player decisions in gameplay. Making the case for the benefits of multimodal approaches to game studies, this volume is key reading for students and researchers in multimodal studies, game studies, rhetoric, semiotics, and discourse analysis.
This article develops a new methodological tool, called scales of action, which allows the empirical investigation of ubiquitous actions such as driving on the one hand, and the highly complex relationships between (for example) drives and other actions in everyday life on the other hand. Through empirical analysis of ethnographic data of drives performed by a German artist and an American IT specialist, the article illustrates how talk and driving are embedded differently in different cultural contexts. Examining the actions of the two drivers before, during, and after a drive further demonstrates that chronologically performed actions are not necessarily sequential in nature. Using a mediated discourse theoretical approach and building upon multimodal (inter)action analysis, the article provides analysts with a tool that captures the inherent complexities of everyday actions. Through the notion of scales of action and their composition, this article sheds new light upon the complexity and cultural differences of drives and car talk in middle class Germany and North America.
In this article I present an analysis of three extracts from a business coaching session captured on video. In business coaching the coach aims to help the client generate solutions to their own issues, often by finding different perspectives. However, there has been a lack of empirical studies focusing on the coaching interaction. Here I set out firstly to describe how the coach carried out the act of coaching, and to illustrate the use of higher and lower-level actions and modal density to focus on the detail of an (inter)action, while not losing sight of the whole. I analyse all the relevant communicative modes (Norris, 2011a), and focus closely on specific lower-level actions in the interaction. I use modal density (Norris, 2004) as a methodological tool to consider these lower-level actions as constituents of higher-level actions, and as a measure of participants’ relative engagement in various higher-level actions. Overall, I show that modal density and lower and higher-level actions can be used as useful tools for the analysis of business coaching at the level of the interaction.
Building on the argument that practices between teacher and learners in classrooms may differ (Scollon and Scollon, 1981; Brice Heath, 1983 [1996]; Street, 1984; Gee, 1996; Barton and Hamilton, 1998), I look at how literacy focused school classroom teaching/learning practices instilled into an individual have a long-term effect. Using a multimodal (inter)action analytical approach (Norris, 2004, 2014) and the site of engagement as my analytical tool that brings together concrete actions, practices and discourses as a coherent whole, I examine actions, practices and discourses produced and reproduced by an art teacher and a new art student in a small private art school in Germany. While the art teacher draws on and re-produces the practice of painting, the new art student draws on and reproduces the practices and discourses that she learned in formal schooling, forcing her to produce and understand modal configurations that do not align with the creative practice that she is learning. This paper has potential educational and social ramifications as it illustrates that formal schooling may have a negative effect upon creativity by focusing the schooled individual upon results and on language/listening. These foci directly translate into modal behaviour which disadvantages the individual when trying to learn a creative practice, where the process and showing/seeing are emphasised. As the world becomes more multimodal and creative, we may want to engage in more research to rethink what and how children are taught.
ResearchGate has not been able to resolve any references for this publication.