No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Communicating knowledge, getting attention, and negotiating disagreement via videoconferencing technology: A multimodal analysis
Abstract
This article takes a multimodal approach to examine how two young men communicate knowledge, shift attention, and negotiate a disagreement via videoconferencing technology. The data for the study comes from a larger ongoing project of participants engaging in various tasks together. Linking micro, intermediate and macro analyses through the various methodological tools employed, the article presents multimodal (inter)action analysis (Norris, 2004, 2011, 2013a, 2013b) as a methodology to gain new insight into the complexity of knowledge communication via videoconferencing technology, which is relevant to many settings from education to employment, from organizations to gaming.
... It considers interactions between social actors and other social actors, objects, or the environment as actions. It offers a broad socio-cultural perspective which is compatible with detailed micro-analysis of interaction, whilst also allowing the researcher, when required, to address the macro and intermediate ( Norris & Pirini, 2016 ). We now summarise the key analytical tools from MIA employed in our analytical approach. ...
... Modal density is also a measure of social actors' phenomenological attention/awareness ( Norris & Pirini, 2016 ). Norris (2020 ) describes that attention and awareness, considered as two different sides of the same concept, can be analysed as a continuum: foreground -mid-ground -background. ...
... Modal density determines the relative position of the actions in the social actors' attention/awareness continuum. When the HLA carries high modal density, the performer (person A) is more focused upon that action ( Norris & Pirini, 2016 ), which is foregrounded in their attention/awareness. Norris also proposes the notion of (inter)actional attention whereby interlocutors "read off the verbal and nonverbal actions that a person performs" ( 2020, p. 12) as expressions of what the performer (person A) experiences, perceives, thinks, and feels and, then in turn, the interlocutor (person B) performs particular actions in relation to how they perceive person A's performed actions. ...
... Within this framework, all actions are considered interactions between social actors and other social actors, objects, or the environment. It offers a broad socio-cultural perspective which is compatible with detailed micro analysis of interaction, allowing the researcher to address the macro, intermediate and micro analyses as required (Norris & Pirini, 2016). We now explain multimodal (inter)action analysis' key analytical tools. ...
... Modal density is also a measure of social actors' phenomenological attention/awareness (Norris & Pirini, 2016). Social actors can pay "various levels of attention to simultaneously performed higher-level mediated actions" (Norris, 2019: 247). ...
... The modal density of these actions determines their relative position in social actors' attention/awareness continuum: foreground, mid-ground and background. When the higher-level action has high modal intensity, the social actor is more focused upon that action (Norris & Pirini, 2016), which is foregrounded in his/her attention/awareness. ...
Online language teaching is gaining momentum worldwide and an expanding body of research analyses online pedagogical interactions. However, few studies have explored experienced online teachers' practices in videoconferencing particularly while giving instructions, which are key to success in task-based language teaching (Markee, 2015). Adopting multimodal (inter)action analysis (Norris, 2004, 2019) to investigate the multimodal construction of instructions in a single case study, we examine instruction-giving as a social practice demonstrated in a specific site of engagement (a synchronous online lesson recorded for research purposes). Drawing on the higher-level actions (instruction-giving fragments) we have identified elsewhere (Satar & Wigham, 2020), in this paper we analyse the lower-level actions (modes) that comprise these higher-level actions, specifically focusing on the print mode (task resource sheets, URLs, text chat, and online collaborative writing spaces) wherein certain higher-level actions become frozen. Our findings are unique in depicting the modal complexity of sharing task resources in synchronous online teaching due to semiotic misalignment and semiotic lag that precludes the establishment of a completely shared interactional space. We observe gaze shifts as the sole indicator for learners that the teacher is multitasking between different higher-level actions. Further research is needed to fully understand the interactional features of online language teaching via videoconferencing to inform teacher training policy and practice.
... The shift toward online courts has extensively reformatted the legal communication platform, transformed the courtroom layout, and eliminated whole-body movements (Guichon and Cohen 2016;Hampel and Stickler 2012;Norris and Pirini 2017). Considering the significant role that modes play in legal discourse for physical courts, how metafunctional meanings are constructed in videoconferencing technology-based online courts requires rigorous examination. ...
... Moreover, it allows multimodal texts to be annotated multiple times, as meanings often arise from combined semiotic interactions (Jewitt, Bezemer, and O'Halloran 2016). However, for online trials, spoken language is arguably the most important mode used, and it is impossible to incorporate all modes into this study's discussion (Norris and Pirini 2017). This study takes the metafunctional analysis of He's (2020), which adapted SF-MDA to legal discourse analysis, as a reference. ...
... Despite acknowledged contributions from multimodal studies (e.g., Norris and Pirini, 2016), the role and nature of empirical breaks during reading performances is poorly understood. Typically, these breaks are discussed in negative terms, and conceived as distractive, interruptive, and economically inefficient by reference to, for instance, how digital reading environments cue inattentiveness (Delgado and Salmerón, 2021), constrain attention span (Wolf, 2018), or how a reader struggles to get into what Csikszentmihalyi (1975Csikszentmihalyi ( , 1990 coined as "flow." ...
... The model we propose of imaginative reading consisting of processes unfolding at different timescales bears an interesting resemblance with, and can be enriched through, the analytical tools provided by multimodal interaction analysis (MIA) as developed by scholars like S. Norris and J. Pirini (see Norris and Pirini, 2016). Indeed, MIA provides a framework to perform detailed analysis of mediated actions (social actions) which comprise higher and lower level actions. ...
This paper applies an embodied perspective to the study of reading and has a twofold aim: (i) to discuss how reading is best understood in terms of cultural-cognitive performance that involves living bodies who actively engage with reading materials, and (ii) to spark a dialogue with neighboring disciplines, such as multimodality studies and movement studies, which likewise pivot on how practices and performances involve moving bodies: life is something we do. An embodied cognitive perspective considers how performance is constrained by and draws on expertise such as lived experience as well as the material affordances available in the situation. Such a perspective is crucial for reading research as this domain has been, and largely still is, dominated by the view that reading is a silent, disembodied activity that takes place in the reader's brain by means of neural mechanisms. However, recent studies of reading practices are starting to develop new explanations emphasizing the multimodal engagement in reading as crucial for managing the activity. While this perspective is still empirically underexplored, we seek to highlight how reading is managed by readers' dynamic, embodied engagement with the material. We call this engagement cognitive pacemaking, an action-perception phenomenon we argue should be considered as the key mechanism for controlling attention. We present here a framework to understand reading in terms of pacemaking by emphasizing attentional shifts constituted by embodied modulations of lived temporality. Methodologically, we combine a close reading of a classic literary text, with the focus on attentional modulation with a qualitative study of university students reading different short texts. We highlight how meaning emerges not primarily from linguistic decoding and comprehension, but also from cognitive-cultural, multimodal engagement with the text. Finally, we conclude that empirical reading research should focus on how embodied reading differs across contexts, genres, media and personalities to better scaffold and design reading settings in accordance with those aspects.
... The analysis of modal configurations relies on first assessing the primary meaning of a higher-level action, and then determining which lower-level actions are hierarchically most important to the meaning the higher-level action produces. Using this approach, Norris and Pirini (2017) demonstrate how agreement and disagreement operate multimodally in a knowledge communication task. Here, disagreement was co-ordinated multimodally, but disagreement expressed through language lagged behind disagreement expressed through other modes. ...
... The site of engagement is an important conceptual tool allowing analysts to move between micro, intermediate and macro analyses (Norris, 2011;Norris & Pirini, 2017). The notion was first introduced by Scollon (1998) as the "window opened up through the intersection of social practices" (p. ...
Multimodal (Inter)action Analysis was developed to study social interaction based upon the theoretical notion of mediated action. Building on this core concept, Multimodal (Inter)action Analysis includes several theoretical/methodological tools. These tools facilitate analysis which moves flexibly between micro-level moments of interaction and macro-level practices and discourses. In this chapter, the application of mediated action to multimodal analysis is discussed, before the central theoretical/methodological tools are introduced. Tight links are made between the tools used in Multimodal (Inter)action Analysis and the core theoretical tenets, to support robust multimodal interaction research.
... The unit of analysis is midlevel action [25]. Although meetings are higher level actions, explanations constitute lower level actions, which are smaller interactional meaning units formed by different modes [71] or assemblages. Some actions operate on an even lower level. ...
... Some actions operate on an even lower level. For instance, in the mode of gesture, a gesture unit is also regarded as a lower level action [71]. The midlevel actions described in this article constitute actions in which multimodal resources are assembled in explanations of the product (Extracts 2 and 5) and production process (Extract 3), consensus-seeking communication (Extract 3), and solution-finding communication (Extract 4). ...
bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Background:
Engineers increasingly work and advance their careers in international business settings. As technical managers, they need management and technical skills when working with different stakeholders with whom they may not share a common first language. Studies have revealed that informal oral communication skills are of prime importance for global engineers who face challenges in building shared meaning and formulating clear messages in meetings with non-native speakers of English. This article proposes that studying the use of multimodal resources (spoken language, gaze, gestures, and objects) in meetings can unpack how work tasks are accomplished in business through different communicative strategies.
Literature review:
This paper focuses on engineers’ and technical managers’ needs and challenges in professional and intercultural communication where English is used as a business lingua franca (BELF) in multimodal meetings. While multimodal conversation and discourse analytic studies highlight the dynamic nature of meeting interaction, previous technical and professional communication and BELF research on multimodality is limited.
Research questions:
1. How do technical managers use multimodal resources to articulate their ideas in BELF meetings with their peers? 2. How does the use of multimodal resources contribute to the construction of shared meaning in explanatory, consensus-seeking, and solution-finding communication?
Methodology:
This study reports on two case studies and multimodal discourse analysis of video-recorded meetings among technical managers and their peers in four companies. The use of multimodal resources is analyzed in explanatory, consensus-seeking, and solution-finding communication.
Results and conclusions:
In BELF meetings, assemblages of spoken language, gestures, tools, whiteboard, and documents contribute to constructing shared meaning. This study has implications for global professional and engineering communication. Future research should further examine multimodality in BELF meetings.
... Multimodal (inter)action analysis was used to analyze the conversation transcription (see. Norris & Pirini, 2016). Norris's (2019, p. 164) suggestion that "a lower-level mediated action does not ever exist by itself" was also considered in the data analysis because generally, humans would produce the utterance in higherlevel action (since it always involved many different modes (e.g., spoken, gesture, and facial expression). ...
Technological advancement has enabled language learners to employ verbal and nonverbal cues in computer-mediated communication (CMC). These cues can support language use for learners wishing to communicate more effectively in English. Interactive alignment is one phenomenon that shows how humans tend to collaborate in their language use by adapting, priming, and reusing verbal and nonverbal cues to achieve mutual understanding. Informed by a sociocognitive framework, this study explored and documented English language learners' multimodal interactive alignment during their CMC task engagement through Instagram. We collected data from 30 first-year Indonesian business school learners who participated in seven online CMC tasks using Instagram chat features: text chat, voice chat, and video chat. To examine various interactive alignments (e.g., how interlocutors adapt, prime, and reuse verbal and nonverbal cues to achieve mutual understanding) that occurred during multimodal task communication, we employed multimodal (inter)action analysis. Findings revealed that learners adapted and reused various nonverbal features (e.g., emojis, GIFs, facial expressions, gestures) and verbal cues (e.g., expression, lexical) to convey and comprehend meaning during CMC task completion. Caveats about using various nonverbal alignment patterns for supporting better English online communication were also noted. The study highlights how language learners use the full repertoire of semiotic resources in CMC to maximize their online language learning.
... Multimodal (Inter)action analysis has already proved quite valuable in unravelling some of the complexities which emerge through new technologically-mediated forms of social interaction. In the fractured interactional ecologies which emerge when communication is complexly mediated by new technologies, non-verbal modes take on a new importance and appear to be pivotal for exemplifying divergence in stance or disagreement (Norris and Pirini 2017), can be vital for the production of identity (Geenen 2017) and can be exploited to help facilitate smoother interactions with pre-verbal children and toddlers (Geenen 2018(Geenen , 2020. Other work has shown how non-verbal actions can be pivotal to accomplishing collaborative tasks via video-conferencing technology (Geenen et al. 2021) but that the distribution of attention and the interactional demands can also result in miscommunication and misunderstanding (Norris and Geenen 2022). ...
... Note. # n refers to the image number within the excerpted transcript being discussed According to Norris & Pirini (2016), multimodal (inter)action analysis is "a holistic analytical framework that understands the multiple modes in (inter)action as all together building one system of communication" (p. 24). ...
This study examines how English-as-lingua-franca (ELF) learners employ semiotic resources, including head movements, gestures, facial expression, body posture, and spatial juxtaposition, to negotiate for meaning in an immersive virtual reality (VR) environment. Ten ELF learners participated in a Taiwan-Spain VR virtual exchange project and completed two VR tasks on an immersive VR platform. Multiple datasets, including the recordings of VR sessions, pre-and post-task questionnaires, observation notes, and stimulated recall interviews, were analyzed quantitatively and qualitatively with triangulation. Built upon multimodal interaction analysis (Norris, 2004) and Varonis and Gass' (1985a) negotiation of meaning model, the findings indicate that ELF learners utilized different embodied semiotic resources in constructing and negotiating meaning at all primes to achieve effective communication in an immersive VR space. The avatar-mediated representations and semiotic modalities were shown to facilitate indication, comprehension, and explanation to signal and resolve non-understanding instances. The findings show that with space proxemics and object handling as the two distinct features of VR-supported environments, VR platforms transform learners' social interaction from plane to three-dimensional communication, and from verbal to embodied, which promotes embodied learning. VR thus serves as a powerful immersive interactive environment for ELF learners from distant locations to be engaged in situated languacultural practices that goes beyond physical space. Pedagogical implications are discussed.
... HLAs and is also a part of larger scale HLA (Norris & Pirini, 2017). Thus, the episodes of interaction and the lecturer's monologue are part of the lecture. ...
This study analysed the multimodal interactive discourse of one English-medium instruction (EMI) lecturer to engage students in a digital environment. It examined the first live online class given to a group of international students living in different countries. A methodology based on the multimodal (inter)action analysis approach was followed to study how interaction unfolded and was promoted and managed. Results showed the complexity of classroom interaction in this digital environment, the importance of lecturer waiting time, the high modal density and functional diversity of the follow-up/ feedback stage and the most frequent discourse functions expressed during the interaction. The results will be of interest to designers of EMI training courses concerned with student engagement in virtual settings. Some suggestions are given regarding the need to know how to foster EMI lecturers’ awareness of multimodal interactive discourse.
... Multimodal (inter)action analysis is thus a coherent and comprehensive research framework for the analysis of qualitative video-based data. 2 All the pieces in this framework fit together (Norris 2012;Pirini 2014b), allowing the researcher to build a coherent picture of whatever human action, interaction or identity is being studied. In this way, we have made strides in examining space and place or children's acquisition (Geenen 2013;Geenen 2017Geenen , 2018; identity (Norris 2005(Norris , 2007(Norris , 2008(Norris , 2011Norris and Makboon 2015;Matelau-Doherty and Norris 2021); video conferences (Norris 2017a;Norris and Pirini 2017); business coaching, high school tutoring and intersubjectivity (Pirini 2013(Pirini , 2014a(Pirini , 2016, to name but a few areas in which the framework has been used. What we at the AUT Multimodal Research Centre are finding is that with a coherent framework such as MIA, there is much potential to discover new insight and knowledge about any kind of human action, interaction, and identity. ...
This paper presents a concise introduction to Multimodal (inter)action analysis (MIA), which began to be developed in the early 2000s in tandem with technological advances for visual qualitative research. By now, MIA has grown into a fully-fledged research framework, including multimodal philosophy, theory, method and methodology for the study of human action, interaction and identity. With systematic phases from data collection to transcription (including transcription conventions) and data analysis, this framework allows researchers to work in a data-driven and replicable manner moving past common interpretive paradigms (Norris 2019, 2020).
... It can be a meeting in an office, translating or giving feedback. Higher-level actions are also found at the level of discourse and practice (Norris & Pirini, 2017). The concept of frozen mediated actions covers previously performed actions that are embedded within an object, such as a document which has been written and printed through someone's mediated actions. ...
This paper presents an analysis of three roleplayed interpreted institutional meetings in which sight translation is part of the interaction. The analysis is based on multimodal (inter)action analysis and utilises the analytical tool of modal density as indication of attention/awareness. This analytical framework is novel in interpreting studies. The data include filmed material from an experimental setting and participants’ reflections about the situation. The findings show variations in sight translation practices and that the shift from interpreting to sight translation affects interactional patterns, particularly social actors’ attention and agency. In my discussion of agency in sight-translated interaction, I argue that interpreters, in addition to translating, need to pay attention to interactional issues related to attention and agency caused by the interpreting method.
... Within this scholarly body of work, researchers have utilized multimodal analytical frames that draw from various theoretical underpinnings, and differ across several factors, including, how modes are theorized, the role of objects and artifacts relative to analysis, and how sociocultural and historical aspects of analysis are considered (Jewitt et al. 2016;Pirini et al. 2018). The methodology presented herein is grounded in multimodal interaction analysis (MIA) (Norris 2004(Norris , 2012Norris and Pirini 2017), which takes a holistic approach to analysis grounded in the perspective of the mediated interaction (Wertsch 1998), with the unit of analysis being individual actions (Pirini et al. 2018). As our research is grounded in the embodied perspectives introduced above, we take a more micro-interactional view than prior uses of MIA. ...
Science teaching and learning are discursive practices, yet analysis of these practices has frequently been grounded in theorizations that place language at the forefront of interaction and meaning-making. Such language-centric analytic approaches risk overlooking key embodied, enacted aspects of students’ engagement in science practices. This manuscript presents a case of a plurilingual student’s participation in science inquiry to demonstrate how multimodal interaction analysis can be used to examine the highly diverse array of communicative resources that she draws upon while participating in science, including gestures, facial expressions, vocal intonations, and languages. Grounded in dialogic theorizations of language, we first detail the multimodal interaction approach, and second, we show how multimodal interaction analysis beginning first with her embodied engagement, then coupled with her subsequent written and spoken engagement, reveals robust views of her engagement in science practices. Key to this methodological approach is multilayered analysis that backgrounds verbal or spoken communication to allow for an identification of embodied interaction resources employed. We emphasize how this analytical method allows us to conceptualize science as a practice that unfolds through and in interaction, as compared to a static body of concepts to be learned.
... Multimodal (Inter)action Analysis (Matelau, 2014;Norris, 2004Norris, , 2011Norris, , 2013Norris & Pirini, 2017;Pirini, 2014aPirini, , 2014bPirini, , 2016Pirini, , 2017Pirini, Norris, Geenen, & Matelau, 2014) is a holistic approach to the analysis of multimodal action and interaction. With its strongest theoretical origins in Mediated Discourse Analysis (Scollon, 1998(Scollon, , 2001, the framework embraces the mediated action, defined as a social actor acting with/through mediational means (Scollon, 1998(Scollon, , 2001Wertsch, 1998) as a unit of analysis. ...
This chapter introduces the five most prominent approaches to multimodal data analysis: Multimodal (Inter)action Analysis, Mediated Discourse Analysis, Systemic Functional Multimodal Discourse Analysis, Social Semiotics, and Multimodal Conversation Analysis. The chapter begins by discussing the origin of each approach, methods of data collection and analysis, and embedded theoretical foundations. The theoretical differences between the approaches are then examined with a focus on the way each approach treats the individual, the artefact, and the notion of mode during analysis. With the variety of available approaches to multimodal data analysis, it is important that researchers link data collection and analysis to coherent theoretical underpinnings.
... Given the growing significance of multiliteracies in the age of new media, language learners no longer read, write and interact only on paper using writing and speech, but they also do so electronically and through multiple modes of meaning-making. This shift in writing practices has led many researchers to investigate the role and impact of multimodality in language learning, predominantly in the area of online language teaching (e.g., Guichon, 2017;Hampel andStickler, 2012, Satar andWigham, 2017), learner interactions (Lamy, 2007;Lamy and Flewitt, 2011;Norris and Pirini, 2016;Satar, 2016), and more recently in online collaborative writing (see the Special Issue in Journal of Second Language Writing edited by Li and Storch, 2017). Given this development, research on multimodal electronic feedback in foreign and second language writing (SLW) is a timely, but still under-researched topic (Ene and Upton, 2014). ...
Spoilt for choice: A plethora of modes for electronic feedback on second language writing
... Therefore, the term 'interaction' is used in its wider sense including interaction between an actor and his/her environment and among different modes that are employed. Multimodal (inter)action analysis draws on concepts from a variety of theoretical approaches, including mediated discourse, social interaction studies and social semiotics, and it allows the researcher to deploy numerous methodological tools to move between macro, intermediate and micro analysis (Jewitt et al., 2016;Norris & Pirini, 2016) in order to describe a range of communication modes and the " situated interplay between modes at a given moment in social interaction " (Jewitt et al., 2016: 115). In employing a multimodal (inter)action analysis, we engage with the contributions of gestures, gaze and proxemics in the visual mode. ...
This paper focuses on instruction-giving practices, a crucial but under-researched aspect of online language tutorials. The context for this qualitative study is a telecollaborative exchange focussing on French as a foreign language. We investigate trainee teachers’ instructions for a role-play rehearsal task during webconferencing-supported language teaching sessions.
Multimodal (inter)action analysis (Jewitt, Bezemer, & O’Halloran, 2016; Norris, 2004) of the data from three sessions reveals how the trainees mark different stages in the instructions using gaze and webcam proximity, allocate roles helped by the use of gaze (Satar, 2013) and gestures (Guichon & Wigham, 2016; McNeill, 1992), and introduce key vocabulary using word-stress, gaze and text chat strategies. The paper sheds light on the need to demonstrate clear boundaries between instructions and beginning of the task and the need, in future online teacher training programmes, to prepare trainees to direct learners’ attention to the resources needed for task accomplishment, explain how the task will be accomplished using the online resources and harness the potential of semiotic resources during this teaching phase.
Relationships are increasingly conducted through digital and mobile communication technologies. This chapter focuses on mobile messaging apps such as WhatsApp and SMS, exploring how ‘mobile conversations’ shape, and are shaped by, the offline activities and encounters in which interlocutors engage, and the potential impact on the way we engage with, and relate to, others. To investigate this, we advance a new perspective on digital discourse which combines mobile messaging data with interviews and time-use diaries. Six distinct ‘engagement styles’ were identified from quantitative and interactional analysis of the data. We show how these engagement styles encourage distinct patterns of interaction, drawing on a combination of quantitative and interactional analysis with a focus on message length, use of multimodal elements such as emoji and images, positioning cues such as deixis and vocatives, and involvement strategies. Our new perspective on digital discourse shows how mobile conversations cannot be fully understood without consideration of the wider (physical and digital) contexts in which they are sent and the ways people juggle relationship concerns with competing demands on their attention.
School Children increasingly engage in designing and making activities, with the aim to promote creativity, increase peer collaboration, and engage with novel technologies. However, there is limited exploration of the impact of these activities. This study inquires what kinds of identities children (co)produce during design and making and how they evolve. Nexus analysis is used as a research strategy, drawing on multimodal research materials accumulated and gathered during a design and making project conducted with 11–12-year-old pupils. The research lens views social action as an intersection of participants’ histories, their mutual interaction orders and discourses in place. The analysis entails scrutinizing the detailed aspects of identity production that are bound with wider discourses and interactions. Our findings show how identities of a maker, designer, change agent, performer, leader, content producer, and empathizer emerged, and how the competing orientations of being pupils and engaging in design and making that invite diverse types of identity production among the pupils, at the same time gaining agency as designers and makers, and ownership of their designs.
Este capítulo presenta los principios básicos de los estudios sobre multimodalidad, incluye una definición de los conceptos clave y de las metodologías de investigación más importantes, y analiza cómo contribuye la multimodal en la enseñanza de segundas lenguas. El estudio describe el concepto básico de no centralización lingüística de la lengua y de su enseñanza basada en los diferentes modos comunicativos y la contribución de estos en la construcción de significado. El capítulo está estructurado en tres secciones principales. En primer lugar, una revisión de la literatura previa estudia el concepto de multimodalidad. Se analizan los marcos teóricos y metodológicos más utilizados en el estudio de la multimodalidad, destacando de este modo los conceptos y los objetivos más importantes de enfoques tales como el análisis del discurso multimodal desde la sistémica funcional, la semiótica social, el análisis de la (inter)acción multimodal y el análisis conversacional. A continuación, se examina la contribución que la investigación multimodal ha hecho al campo de la enseñanza de segundas lenguas, identificando tendencias y necesidades investigadoras. La sección se cierra con una propuesta de nuevas líneas de investigación y enfoques para explorar la multimodalidad desde una perspectiva interdisciplinar. Finalmente, el capítulo se centra en la integración de la multimodalidad en la enseñanza del español, y su contribución en la mejora del proceso de enseñanza-aprendizaje a través del desarrollo de la competencia comunicativa multimodal. A modo de ejemplo, se propone la pedagogía basada en el género como un enfoque centrado en la multimodalidad para desarrollar la multialfabetización de los alumnos, así como su creatividad y pensamiento crítico.
Multimodal (inter)action analysis offers a powerful and robust methodology for the study of action and interaction between social actors, their environment, and the objects and tools within. Yet its implementation in the analysis of synchronous multimodal online data sets, e.g. (inter)actions via videoconferencing, is limited. Drawing on our research in understanding teacher-learner (inter)actions in instruction-giving fragments in synchronous multimodal online language lessons, we describe and illustrate the ways in which we adapted and extended some of the methodological and analytical tools. These include (1) the use of a grounded theory approach in delineating and identifying higher-level actions, (2) the embodiment and disembodiment of frozen actions, (3) electronic print mode, (4) semiotic lag, (5) semiotic (mis)alignment, (6) modal density (mis)alignment, and (7) how modal density can be achieved by brisk modal shifts in addition to through modal intensity and complexity. We conclude by a call for further educational research in online teaching platforms using the framework to have richer understandings of the (inter)actions between social actors with particular roles and identities (teachers-learners), their environment, and the objects and tools within, which bring their “own material properties, feel and techniques of use, affordances and limitations” (Chun, Dorothy, Richard Kern & Bryan Smith. 2016. Technology in language use, language teaching, and language learning. The Modern Language Journal 100. 64–80: 65).
Dieser Beitrag richtet den Fokus auf Multimodalität in der digitalen Kommunikation am Beispiel von Variation im digitalen Humor von Memes. Zunächst wird auf multimodale digitale Kommunikation in Chats und sozialen Medien eingegangen, in welcher verschiedene Modalitäten wie Bilder, Sprache, Text, Musik und Videos in einer digitalen Kommunikationseinheit als Zeichenmodalitäten kombiniert werden. Ein besonderer Schwerpunkt liegt auf Memes als kommunikative Artefakte und ihrer vielfachen Eigenschaft, Humor zu erzeugen. Diese Studie zu Variation bei digitalem Humor zeigt auf, wie einzelne Modalitäten der Memes mit demographischen Faktoren der Nutzenden in Beziehung stehen und wie verschiedene multimodale Humortypen entstehen. Zudem werden die Vielzahl und Vielfalt der verwendeten Zeichenmodalitäten sowie ihrem Zusammenspiel dargestellt und Wege der Analyse ihrer Funktionen im Rahmen interaktiver digitaler Kommunikation aufgezeigt.
The present study centers on exploring how different modes such as speech, gesture, gaze, body posture, and head movement are employed by an EFL teacher and his students during a Critical Learning Episode (CLE) (Davis et al.,2009). CLEs are brief instances of classroom interaction where the instructor and the researcher believe that learning is being fostered or inhibited. This article is part of a larger qualitative multiple-case study that took place at a private Colombian University. The lesson was videotaped and then analyzed within a Multimodal (Inter)action Analysis framework (Norris, 2020). Findings indicate that CLEs are created in a highly embodied, multimodal, and ecological manner through different modal configurations. Besides speech and writing, modes such as gestures, posture, gaze, and head movement played not a marginal, but prominent role, in performing various pedagogical classroom activities such as enhancing shared/focused attention, strengthening alignment, helping teachers and learners to visually make meaning of morphological and syntactical units, and serving as devices to check for understanding.
The purpose of the proposed paper – which places itself within the field of Postcolonial Critical Discourse Studies (see Esposito, E. (2021). Politics, ethnicity and the postcolonial nation – a critical analysis of political discourse in the Caribbean. John Benjamins Publishing Company) – is to analyse the multi-semiotic practices contributing to the characterisation of the protagonist of Trinidadian short-film “Doubles with Slight Pepper” as a cinematic portrayal of the hybrid Indo-Trinidadian identity. By way of a Multimodal Critical Discourse Analysis (Kress, G. and van Leeuwen, T. (1996). Reading images: the grammar of visual design. Routledge, London, Kress, G. and van Leeuwen, T. (2001). Multimodal discourse. The modes and media of contemporary communications discourse. Hodder Educations, London, Kress, G. and van Leeuwen, T. (2006). Reading images: the grammar of visual design, 2nd ed. London: Routledge; O’Halloran, K.L. (2004). Multimodal discourse analysis. Continuum, London), the study aims to cast light on the strategies exploited by the filmmaker (1) to depict the protagonist (Dhani) as an in-betweener whose ambitions are inhibited by his social status stemming from generations of subjugation and misuse by colonialists; and (2) to promote Indo-Trinidadian cultural specificities. Following an introduction to the key concept of individual and collective identity with a focus on Trinidad and Tobago, and an outline of diasporic cinema as applied to the Indo-Trinidadian community, the Multimodal Critical Discourse Analysis is carried out on the verbal, non-verbal, and visual sub-corpora gathered up in sequences according to the main three identitarian traits exhibited in the short-film: ‘religion/folklore’, ‘food’, and ‘lineage’.
FULL TEXT HERE: https://rfmv.fr/numeros/5/articles/11-visual-transcription/ The communication and language sciences have decidedly taken a multimodal turn and a proliferation of work in previously language-dominated fields is focusing on the contribution of non-verbal communicative modes in social interaction. While this has proven empirically and theoretically fruitful, it throws an additional kink in an already complicated issue: transcription practices and the interpretive and representational process accompanying said practices. In this article, we review and champion a methodological framework that provides analytical tools and a transcription protocol for the generation of visual transcripts. Visual transcription, we contest, is far more congruent with the multimodal ethos and we both detail the method while applying it to a data segment from a larger project investigation teamwork and collaborative problem-solving in dyadic videoconferencing interactions. The analytical focus on action coupled with the accompanying transcription method reveals complex and fluctuating distributions of attention and interactional awareness. Non-verbal actions reveal contributions to the task at hand which are not realized in talk and the analysis details an intricate ebb and flow of attentional orientations which are realized through non-verbal means.
This paper examines the use of video chat (VC) with a focus on expectations and construction of attention. It is based on micro analyses of recorded VC sessions (gathered between 2013 and 2015) and thematic analysis of 29 semi-structured interviews about VC practices (conducted in 2014 and 2015). Building on multimodal (inter)action analysis (Norris, S. (2004). Analysing multimodal interaction: a methdological framework. Routledge, Norris, S. (2016). Concepts in multimodal discourse analysis with examples from video conferencing. Yearbook of the Poznan Linguistic Meeting 2: 141–165) and key concepts from nexus analysis (Scollon, R. and Scollon, S.W. (2004). Nexus Analysis: Discourse and the emerging internet. Routledge), I examine how focused attention is constructed in VCs and how these practices are shaped by experiences with other forms of communication. I demonstrate that unlike other forms of distance communication, typical VC encounters require a full investment of attention. This can be formulated as an interactional maxim: focus your attention on the VC interaction . I discuss how other activities can be interwoven with a VC and examine the exceptional practice of lapsed VC encounters (previously open connections or always-on video). I argue that participants display an orientation towards the maxim when pursuing other courses of action, and that lapsed encounters operate under a different value system than typical focused VC encounters. Finally, I reason that VC is reserved for close relationships because of the required investment of attention.
Virtual learning environments are increasingly important in higher education. These instructional settings render a view of academic genres that have evolved towards the development of technology-mediated communication. This chapter focuses on the digital academic genre of synchronous videoconferencing lecture to find out how a multimodal-in-context approach can be applied to examine the constraints and affordances of interaction in this digital environment. It begins by positioning genre analysis from a multimodal perspective and outlining the main features that characterise interaction in live lectures. Then, the theoretical and methodological research framework is introduced, which integrates concepts central to multimodality that belong to different perspectives: meaning functions (systemic functional multimodal discourse analysis), modal affordance and multimodal ensemble (social semiotics), modal density and higher-level actions (multimodal interaction analysis), and sequential and simultaneous actions (multimodal conversation analysis). Drawing on this eclectic framework, the analysis of interaction in this digital genre is illustrated with examples taken from a synchronous videoconferencing lecture given in an English-medium master’s degree programme. The chapter contributes to the study of spoken interactive genres. This methodology facilitates multimodal analysis of the considerable number of communicative modes that interplay in social interaction. Its application is limited to neither instructional nor to virtual settings.
Multimodal synchronous online language teaching is an area of growing interest for research and practice. Emerging research investigates online language teachers' semio-pedagogical skills and competencies, which includes giving instructions to inform learners how to complete the task. However, the few studies that exist have explored trainee teachers' instruction-giving practices, while other work on instructions is grounded in face-to-face classroom settings. Using a qualitative design, this paper investigates experienced teachers' delivery of task instructions for the same task in small group multimodal synchronous online language teaching via videoconferencing. Employing grounded theory and multimodal interaction analysis, we depict both a comprehensive overview and a detailed micro-analysis of higher-level and lower-level actions that comprise task instructions-as-process. Our findings identify 13 higher-level actions and propose a framework for understanding the nature of instructions-as-process in online language teaching. We offer multimodal interaction analyses of selected higher-level actions (communicating key task information, suggesting ways into task, launching the task) to illustrate the multimodal elements (lower-level actions) utilised by the teachers. In our conclusions, we offer pedagogical and research directions and discuss challenges in identifying success and best practice in delivering task instructions.
Multimodal interaction analysis is a holistic methodological framework that allows the analyst to integrate the verbal with the nonverbal, and to integrate these with material objects and the environment as they are being used by individuals acting and interacting in the world. In short, multimodal interaction analysis allows a researcher to study real people interacting with others, with technology, and with the environment. The point of view taken in multimodal interaction analysis is that all actions in fact are interactions and that all of these (inter)actions are linked to people (referred to as “social actors” in multimodal interaction analysis)—no matter whether you are investigating the real or the virtual world, someone buying ice cream, or using a software program. Multimodal interaction analysis is carried out through a series of analytical phases and steps. Researchers utilize multimodal transcription conventions and analytical tools, which are theoretically founded, allowing for replicability of analyses and reliability of findings. Multimodal interaction analysis allows the integration of all communicative modes, where communicative modes are defined as systems of mediated actions. In this view, all modes of communication together build one system of communication, which comes about through actions and interactions that people produce. This coherent framework and process of analysis can be taught and learned and is proving useful for research examining what people do, how they communicate, and how they interact.
This article presents theoretical concepts and methodological tools from multimodal (inter)action analysis that allow the reader to gain new insight into the study of discourse and interaction. The data for this article comes from a video ethnographic study (with emphasis on the video data) of 17 New Zealand families (inter)acting with family members via skype or facetime across the globe. In all, 84 social actors participated in the study, ranging in age from infant to 84 years old. The analysis part of the project, with data collected between December 2014 and December 2015, is ongoing. The data presented here was collected in December 2014 and has gone through various stages of analysis, ranging from general, intermediate to micro analysis.
Using the various methodological tools and emphasising the notion of mediation, the article demonstrates how a New Zealand participant first pays focused attention to his engagement in the research project. He then performs a semantic/pragmatic means, indicating a shift in his focused attention. Here, it is demonstrated that a new focus builds up incrementally: As the participant begins to focus on the skype (inter)action with his sister and nieces, modal density increases and he establishes an emotive closeness. At this point, the technology that mediates the interaction is only a mundane aspect, taken for granted by the participants.
Israeli political talk-show debates are notoriously fierce and overtly confrontational. To understand the structures and origins of this discursive style, we apply a historical pragmatics perspective, comparing debates of current political events on a popular talk-show to a classic and historically cherished form of traditional Jewish argumentation—the oral study of the premodern Talmud—as performed through paired-study debate (xavruta) in contemporary Talmudic academies. The institutional environments and deeper social significance of the two speech events we compared are highly divergent: Political talk shows represent the uneasy coexistence of real-life conflict and antagonistic game, while xavruta interactions make use of a superficially adversarial format to maximize mutual comprehension between interlocutors and ultimately enhance sociability. Yet on the level of performance—in rhetorical strategy and confrontational style—they have marked similarities. Transcribed recordings of debates in these two arenas of argument were analyzed and compared, and the analysis yields a series of marked similarities in discursive attributes between the two. These similarities include: i. a marked preference for disagreement, ii. high dialogicity of the exchanges in the sense of nuanced listening and responding, iii. acceptability of occasional disruptions in the dialogicity of the conversation-flow without its breakdown, and iv. high complexity of logic and structure in argument and argumentation. Given the direction of the historical timeline, these findings suggest the possibility of a carry-over of discursive styles from the religious/scholarly milieu to the public sphere of ideological and political debate. The survival of this unique discursive style from antiquity to the present, both within and across the scholarly, educational, and public spheres, and across media of communication, would demonstrate the resilience of traditional cultural patterns in the face of radical technological, political, and ideological change.
Meetings are increasingly seen as sites where organizing and strategic change take place, but the role of specific discursive strategies and related linguistic-pragmatic and argumentative devices, employed by meeting chairs, is little understood. The purpose of this article is to address the range of behaviours of chairs in business organizations by comparing strategies employed by the same chief executive officer (CEO) in two key meeting genres: regular management team meetings and ‘away-days’. While drawing on research from organization studies on the role of leadership in meetings and studies of language in the workplace from (socio)linguistics and discourse studies, we abductively identified five salient discursive strategies which meeting chairs employ in driving decision making: (1) Bonding; (2) Encouraging; (3) Directing; (4) Modulating; and (5) Re/Committing. We investigate the leadership styles of the CEO in both meeting genres via a multi-level approach using empirical data drawn from meetings of a single management team in a multinational defence corporation. Our key findings are, first, that the chair of the meetings (and leading manager) influences the outcome of the meetings in both negative and positive ways, through the choice of discursive strategies. Second, it becomes apparent that the specific context and related meeting genre mediate participation and the ability of the chair to control interactions within the team. Third, a more hierarchical authoritarian or a more interpersonal egalitarian leadership style can be identified via specific combinations of these five discursive strategies. The article concludes that the egalitarian leadership style increases the likelihood of achieving a durable consensus. Several related avenues for research are outlined.
This article reviews a range of conversation analytic findings concerning the role of information imbalances in the organization of conversational sequences. Considering sequences launched from knowing and unknowing epistemic stances, it considers the role of relative epistemic stance and status as warrants for the production of talk and as forces in the process of sequence production and decay.
There is evidence that co-speech gestures communicate information to addressees and that they are often communicatively intended. However, we still know comparatively little about the role of gestures in the actual process of communication. The present study offers a systematic investigation of speakers’ gesture use before and after addressee feedback. The findings show that when speakers responded to addressees’ feedback gesture rate remained constant when this feedback encouraged clarification, elaboration or correction. However, speakers gestured proportionally less often after feedback when providing confirmatory responses. That is, speakers may not be drawing on gesture in response to addressee feedback per se, but particularly with responses that enhance addressees’ understanding. Further, the large majority of speakers’ gestures changed in their form. They tended to be more precise, larger, or more visually prominent after feedback. Some changes in gesture viewpoint were also observed. In addition, we found that speakers used deixis in speech and gaze to increase the salience of gestures occurring in response to feedback. Speakers appear to conceive of gesture as a useful modality in redesigning utterances to make them more accessible to addressees. The findings further our understanding of recipient design and co-speech gestures in face-to-face dialogue.
From emails relating to adoption over the Internet to discussions in the airline cockpit, the spoken or written texts we produce can have significant social consequences. The area of mediated discourse analysis considers the actions individuals take with texts – and the consequences of those actions.
Discourse in Action brings together leading scholars from around the world in the area of mediated discourse analysis and reveals ways in which its theory and methodology can be used in research into contemporary social situations. Each chapter explores real situations and draws on real data to show how the analysis of concrete social actions broadens our understanding of discourse. Taken together, the chapters provide a comprehensive overview to the field and offer a range of current studies that address some of the most important questions facing students and researchers in linguistics, education, communication studies and other fields.
Mimicry has been observed regarding a range of nonverbal behaviors, but only recently have researchers started to investigate
mimicry in co-speech gestures. These gestures are considered to be crucially different from other aspects of nonverbal behavior
due to their tight link with speech. This study provides evidence of mimicry in co-speech gestures in face-to-face dialogue,
the most common forum of everyday talk. In addition, it offers an analysis of the functions that mimicked co-speech gestures
fulfill in the collaborative process of creating a mutually shared understanding of referring expressions. The implications
bear on theories of gesture production, research on grounding, and the mechanisms underlying behavioral mimicry.
KeywordsCo-speech gestures–Mimicry–Collaborative referring–Grounding–Common ground
This paper examines embodied procedures for producing disagreement turns in the midst of the children's game of hopscotch. Turn shape, intonation, and body positioning are all critical to the construction of stance towards a player's move in the game. In particular, in formulating a player's move as “out” foul calls can state unambiguously, without doubt or delay that a violation has occurred. Turn initial tokens in disagreement turns include cries of “OUT!”, negatives (“No!”), or response cries (nonlexicalized, discrete interjections such as “Ay!” or “Eh!”). Players make use of pitch leaps, vowel lengthening, and dramatic contours (for example, LHL contours) to vocally highlight opposition in the turn preface. Whereas the normal pitch range of a speaker's talk in ordinary conversation can be between 250 and 350 Hz, in opposition moves the pitch may be considerably higher, around 600 Hz. Affective stance is also displayed through gestures such as extended points towards the person who has committed the foul or the space where the foul occurred. Explanations or demonstrations (frequently embodied re-enactments of the player's past move) constitute additional critical components of disagreement moves as they provide the grounds for the opposition. Disagreement moves and trajectories within children's games provide demonstrations of the practices through which girls build and display themselves as agents in the constitution of their social order. Data for this study consists of videotaped interaction of working class fifth grade girls on the playground: second generation Mexican and Central Americans in Los Angeles, and African American Southern migrant children. Ethnic differences in the display of opposition are observable within the groups studied.
Abstract There is extensive literature describing the characteristics of a good leader in the area of organizational communication and business management. However, the research tends to be based on secondary, survey or reported data, typically interviews and questionnaires. Moreover, the predominant image of a “good” leader tends to be a charismatic, inspirational, decisive, authoritative, ‘hero’. The Language in the Workplace database provides a large corpus of authentic spoken interaction which allows examination of how effective leaders behave in a wide range of face-to-face interactions at work, and identifies a diverse range of leadership styles. The analysis reveals that effective leaders select from a range of strategies available to challenge, contest or disagree with others, paying careful attention to complex contextual factors, including the type of interaction, the kind of community of practice or workplace culture in which they are operating, and the relative seriousness of the issue involved. The analysis identifies four distinct strategies which leaders use to deal with potential conflict. These strategies lie along a continuum from least to most confrontational: Conflict avoidance; diversion; resolution through negotiation; and resolution by authority. The findings suggest that good leaders “manage” conflict: i.e. they choose strategies which address both their transactional and relational goals in order to achieve a desirable outcome. Keywords: Disagreement, Discourse analysis, Argument, Leadership, Meetings.
Using data from more than ten years of research, David McNeill shows that gestures do not simply form a part of what is said and meant but have an impact on thought itself. Hand and Mind persuasively argues that because gestures directly transfer mental images to visible forms, conveying ideas that language cannot always express, we must examine language and gesture together to unveil the operations of the mind.
Die im vorliegenden Band versammelten Studien stützen sich auf ungewöhnlich umfangreiches empirisches Material, das im Rahmen eines kommunikationswissenschaftlich-medizinischen Modellprojekts an der Universität Duisburg-Essen, Campus und Klinikum Essen, erhoben wurde. Am Beispiel von Videokonferenzen isolierter, krebskranker Kinder mit ihren Familien und Freunden entstand ein übertragbares Modell für „Telekommunikation von Kindern im Krankenhaus mit Eltern, Lehrern, Freunden (TKK-ELF)“. Seine Entwicklung und kommunikationswissenschaftliche Evaluation werden hier ebenso dokumentiert wie die daran anschließenden interaktions- und gesprächsanalytischen Arbeiten der Essener Forschungsgruppe zur Theorie und zur methodischen Erforschung technisch vermittelter (multimedialer) interpersonaler Kommunikation und ihrer oft multiplen Multimodalität.
Der Inhalt
• Das Modellprojekt TKK-ELF
• Wege zum isolierten Patienten
• Raum- und Patientenbesuche
• Kodierung von Bildinhalten
• Phantasiespiel über Skype
• Multimodale Kommunikation im Interaktionsverbund
• Telesupport und Fernhandeln
• Reden gegen Angst
Die Zielgruppen
• Kommunikationswissenschaftler, Gesprächsanalytiker, Medienforscher
•
Krankenhausmediziner, Psycho- und Kinderonkologen, Pädagogen, Pflegekräfte, Angehörige KrebskrankerDie Herausgeber
Prof. Dr. Jens Loenhoff ist Inhaber des Lehrstuhls für Kommunikationswissenschaft an der Universität Duisburg-Essen.
Prof. em. Dr. H. Walter Schmitz war bis 2013 Inhaber des Lehrstuhls für Kommunikationswissenschaft an der Universität Duisburg-Essen.
This article develops a new methodological tool, called scales of action, which allows the empirical investigation of ubiquitous actions such as driving on the one hand, and the highly complex relationships between (for example) drives and other actions in everyday life on the other hand. Through empirical analysis of ethnographic data of drives performed by a German artist and an American IT specialist, the article illustrates how talk and driving are embedded differently in different cultural contexts. Examining the actions of the two drivers before, during, and after a drive further demonstrates that chronologically performed actions are not necessarily sequential in nature. Using a mediated discourse theoretical approach and building upon multimodal (inter)action analysis, the article provides analysts with a tool that captures the inherent complexities of everyday actions. Through the notion of scales of action and their composition, this article sheds new light upon the complexity and cultural differences of drives and car talk in middle class Germany and North America.
This chapter explores the link between gender, occupational roles and relative status in spoken interaction in the workplace. The analysis is based on extracts from business meetings where disagreement occurs between speakers, and focuses on key methodological issues concerning the interpretative process. More specifically, the discussion considers whether gender is relevant to the interpretation of the talk. To this end, two different analytical approaches are applied — a strict Conversational Analytical (CA) approach that relies solely on transcripts of the data to interpret the meaning of the interaction, combined with an analysis that draws on wider interpretative resources, such as participant feedback, information about the companies the participants work for, observation and employment statistics.
Researchers seeking to analyse how intersubjectivity is established and maintained face significant challenges. The purpose of this article is to provide theoretical/methodological tools that begin to address these challenges. I develop these tools by applying several concepts from multimodal (inter)action analysis to an excerpt taken from the beginning of a tutoring session, drawn from a wider data set of nine one-to-one tutoring sessions. Focusing on co-produced higher-level actions as an analytic site of intersubjectivity, I show that lower-level actions that co-constitute a higher-level action can be delineated into tiers of materiality. I identify three tiers of materiality: durable, adjustable and fleeting. I introduce the theoretical/methodological tool tiers of material intersubjectivity to delineate these tiers analytically from empirical data, and show how these tiers identify a multimodal basis of material intersubjectivity. Building on this analysis I argue that the durable and adjustable tiers of material intersubjectivity produce the interactive substrate, which must be established in order for actions that display fleeting materiality to produce intersubjectivity. These theoretical/methodological tools extend the framework of multimodal (inter)action analysis, and I consider some potential applications beyond the example used here.
Multimodal interaction analysis (Norris, 2004) is a holistic methodological framework that allows the analyst to integrate the verbal with the nonverbal, and to integrate these with material objects and the
environment as they are being used by individuals acting and interacting in the world. In short, multimodal interaction analysis allows a researcher to study real people interacting with others, with technology, and with the environment.
Moving towards multimodal mediated theory, I propose to define a mode as a system of mediated action that comes about through concrete lower-level actions that social actors take in the world. In order to explain exactly how a mode is a system of mediated action, I turn to a perfume blog and use one blog entry as my starting point. The mode that I primarily focus on in this article is the mode of smell, explicating that the mode of smell is not synonymous with olfactory perception, even though modal development of smell is certainly partially dependent upon olfactory perception.
As I am ostensibly focusing on the one mode, I once again problematize this notion of countability and delineate the purely theoretical and heuristic unit of mode (Norris, 2004). I clarify that modes a) do not exist in the world as they are purely theoretical in nature; b) that modes can be delineated in various ways; and c) that modes are never singular.
Even though the concept of mode is problematical – and in my view needs to always be problematized – I argue that the term and the notion of mode is theoretically useful as it allows us to talk about and better understand communication and (inter)action in three respects: 1. The notion of mode allows us to investigate regularities as residing on a continuum somewhere between the social actor(s) and the mediational means; 2. The theoretical notion of mode embraces socio-cultural and historical as well as individual characteristics, never prioritising any of these and always embracing the tension that exists between social actor(s) and mediational means; and 3. The theoretical notion of mode demonstrates that modal development through concrete lower-level actions taken in the world, is transferable to other lower-level actions taken.
Taking the action, rather than the utterance or the text, as the unit of analysis, this article isolates different modes, investigating the interdependent relationships, illustrating that the visual mode of gestures can take up a hierarchically equal or a super-ordinate position in addition to the commonly understood sub-ordinate position in relation to the mode of spoken language. Building on McNeill, Birdwhistell, Eco, and Ekman and Friesen, and using a multimodal interaction analytical approach (Norris), I analyse in detail three separate everyday (inter)actions in which a deictic gesture is being performed and spoken language is used by the social actor performing the gesture. With these examples, I build on previous work in multimodal analysis of texts and multimodal interaction analysis, illustrating that the verbal is not necessarily more important than the visual (Kress and Van Leeuwen; Norris; Scollon), demonstrating that verbal and visual modes can be utilized together to (co)produce one message (Van Leeuwen), and showing that a mode utilized by a social actor producing a higher-level discourse structure hierarchically supersedes other modes in interaction (Norris).
This article identifies some limitations of discourse analysis by analyzing interactions between five boys in which the TV and the computer are featured as mediational means. The incorporation of several modalities into transcripts and a shift in focus from primarily language to human action facilitate a better understanding of the multi-modal interaction involved. The use of conventional transcripts with a focus on language demonstrates that movie-and computer-mediated interactions appear fragmented; by contrast, an inclusion of images into the transcripts, representing central interactions and/or images of a movie or computer screen, demonstrates the significant visual modes that are imperative to the ongoing talk. Just as written words correspond to the oral language, images can exemplify the global interaction among the participants, or they can represent the images on the screen. In addition, viewing an image is much faster than reading a description, so that these images also display the fast pace of the movie-and/or computer-mediated interaction.
In this article I present an analysis of three extracts from a business coaching session captured on video. In business coaching the coach aims to help the client generate solutions to their own issues, often by finding different perspectives. However, there has been a lack of empirical studies focusing on the coaching interaction. Here I set out firstly to describe how the coach carried out the act of coaching, and to illustrate the use of higher and lower-level actions and modal density to focus on the detail of an (inter)action, while not losing sight of the whole. I analyse all the relevant communicative modes (Norris, 2011a), and focus closely on specific lower-level actions in the interaction. I use modal density (Norris, 2004) as a methodological tool to consider these lower-level actions as constituents of higher-level actions, and as a measure of participants’ relative engagement in various higher-level actions. Overall, I show that modal density and lower and higher-level actions can be used as useful tools for the analysis of business coaching at the level of the interaction.
During the activities of everyday life social actors always produce multiple simultaneous higher level actions. These necessarily operate at different levels of attention and awareness. Norris (2004, 2011) introduces modal density as a tool for analysing the attention/awareness of social actors in relation to higher level actions they produce, positioning actions in the foreground, midground and background of attention. Using modal density to analyse an opening and a closing in high school tutoring sessions, I show social actors transitioning into and out of producing the same higher level actions at the foreground of their attention/awareness. Through this analysis I identify two potentially unique aspects of one-to-one tutoring. Firstly I show one way that a tutor helps a student take on the practices of being a good student, and secondly I show the influence that students have over tutoring. I argue that movements into and out of a shared focus of attention are potentially useful sites for analysis of social interaction.
Contemporary research investigating the phenomena of lifestyle sport has highlighted the centrality of space, spatiality and spatial appropriation. Lifestyle sports tend to manifest in liminal and/or unbounded spaces with practitioners drawing upon the affordances of the natural environment in new and unique ways. Simultaneously, practitioners employ continuously evolving technical tools (equipment) in the undertaking of these activities. This article articulates the ways in which the specialised equipment employed as mediational means affect the perception, interpretation and valuation of physical components of the natural and man-made environment.
In this article, I introduce the notion of actionary pertinence and concept of locational element through the analysis of the ways in which general geographic areas (spaces) become actual kitesurfing locations (places) for specific social actors through mediated action and in direct connection to the mediational means through which action occurs. Drawing upon data generated through a year-long video ethnography, I argue that components of the physical environment become locational elements for specific social actors in and through mediated action. This explicitly extends theorisation regarding how mediational means affect and structure the nature of the action they mediate (Vygotsky 1978, 1987; Wertsch 1991, 1998), and extends this argument by exemplifying how mediational means also affect and structure the perception and interpretation of physical components of the environment which have a bearing on mediated action. Furthermore, I articulate the ways in which these locational elements are considered, read, re-read and interpreted in terms of their actionary pertinence. Thus, the materiality of locational elements manifests as salient and/or relevant through the bearing an element has on specific mediated actions and practices.
Previous research has pointed to students’ diverging access to academic discourse practices outside school while lacking empirical insights into how such differences in communities’ communicative repertoires are interactionally brought about. Focusing explanatory discourse, the present study addresses this issue by analyzing the local sequential negotiation of interactional identities and epistemic stance-taking in preadolescents’ family talk and peer talk. Drawing on microanalysis informed by conversation analysis and discourse analysis, it examines how interactants establish local relevance for explanations to occur or not occur and demonstrates that the interactional identity of an explainer as well as knowledgeable stances may be readily adopted as well as rejected. Findings demonstrate that for some children, explanatory discourse in talk with intimates is linked to the interactive disclosure of not-knowing, irrelevance and inability to explicate knowledge for others. The findings indicate that being able to provide explanations in classroom talk might also be a question of identificatory compatibility with regard to students’ out-of-school interactional experiences and identities.
Epistemics in interaction refers to how participants display, manage, and orient to their own and others’ states of knowledge.
This article applies recent conversation analytical work on epistemics to classrooms where language and content instruction
are combined. It focuses on Epistemic Search Sequences (ESSs) through which students in peer interaction collectively resolve
emerging knowledge gaps while working on pedagogic tasks. ESSs are initiated when a speaker displays an ‘unknowing’ epistemic
stance by making an information request about some aspect of language or the content being worked on. We examine three different
types of ESS: those in which a ‘knowing’ response is accepted by the initiator of the sequence; those in which there is an
‘unknowing’ response; and those where ‘knowing’ responses are contested. The findings have implications for understanding
peer interaction in content-based classrooms in three areas: the affordances of peer interaction for learning in contrast
with teacher-led ‘known-answer’ sequences; how learners manage rights and responsibilities around knowing or not knowing;
and how learners discover and work on their own learning objects.
Building on the argument that practices between teacher and learners in classrooms may differ (Scollon and Scollon, 1981; Brice Heath, 1983 [1996]; Street, 1984; Gee, 1996; Barton and Hamilton, 1998), I look at how literacy focused school classroom teaching/learning practices instilled into an individual have a long-term effect. Using a multimodal (inter)action analytical approach (Norris, 2004, 2014) and the site of engagement as my analytical tool that brings together concrete actions, practices and discourses as a coherent whole, I examine actions, practices and discourses produced and reproduced by an art teacher and a new art student in a small private art school in Germany. While the art teacher draws on and re-produces the practice of painting, the new art student draws on and reproduces the practices and discourses that she learned in formal schooling, forcing her to produce and understand modal configurations that do not align with the creative practice that she is learning. This paper has potential educational and social ramifications as it illustrates that formal schooling may have a negative effect upon creativity by focusing the schooled individual upon results and on language/listening. These foci directly translate into modal behaviour which disadvantages the individual when trying to learn a creative practice, where the process and showing/seeing are emphasised. As the world becomes more multimodal and creative, we may want to engage in more research to rethink what and how children are taught.
Aims and Scope
Multimodality is a fast-growing interdisciplinary approach that aims to analyze the interplay of multiple modes such as gaze, gesture or spoken language that are utilized in interaction, and to examine the multimodal production and consumption of communicated messages.
This Reader provides a comprehensive text of current research into multimodality, outlining in-depth delineation of each primary theoretical and methodological approach, as well as personal accounts of scholars, who are responsible for the various approaches’ advancements. The book additionally offers a plethora of analysis chapters, written by scholars from across the world, with vastly diverse themes ranging from buying popcorn, protests in Oman, coaching sessions and identity, to kitesurfing, typography, TV news, billboards, workplace practices, or analyzing web pages, Facebook, comic books, and more.
Flexible and easy to use, the Reader includes key terms, suggested further readings, and a project idea for each chapter. The key terms for the chapters also comprise the extensive alphabetical glossary. Brief introductions for the analysis chapters, written by the editors, summarize the topic, explain the methodology used, outline the thematic orientation, and link each chapter to other chapters in the book.
Showcasing multimodal analysis in detail, this Reader is essential reading for undergraduate and postgraduate students, for emergent researchers, and for advanced scholars who wish to gain insight into the current state of multimodal research.
Problem solving (PbS) talk has been associated with disagreement and conflict as interactants oppose each other's views and express diverse opinions. Although disagreement and conflict have been regarded in earlier work as potentially negative acts more recent work points to the importance of context and local practices instead of a priori categorizations of what the interactants perceive as un/acceptable linguistic behaviour. The paper draws on data from two projects on workplace discourse, one focusing on multinational companies situated in Europe and one on small/medium firms (SMEs). The dataset consists of recordings of meetings, ethnographic observations and interviews. The analysis of the data shows that ‘deviating opinions’ are not only ‘acceptable’ but also unmarked and they form an inherent part of the PbS process. At the same time linguistic behaviour perceived as face threatening or intentionally impolite is typically rare. The paper closes by drawing a theoretical distinction between marked and unmarked disagreement. The latter is perceived as task bound and does not pose a threat to the management of the meeting participants’ complex identities and relationships.
Disagreement is an important socio-pragmatic skill for the workplace context, and newcomers risk causing offence if they fail to adhere to community norms. Applying a Community of Practice framework, embedded in broader societal constraints, I argue the relevance of group norms and shared practices for establishing how and whether disagreement occurs. Focussing on skilled migrant interns entering the New Zealand workplace, the analysis illustrates ways in which these newcomers are unintentionally hindered in their ability to learn and contribute to community norms. Their interlocutors’ failure to endorse their attempts to disagree serves to thwart their socio-pragmatic development. In the examples, the role of co-construction in disagreement, and the interactive nature of these events, is thus highlighted. Because all participants play a part in the ongoing (re)negotiation of the norms of a Community of Practice, a hypothesised ‘tolerance’ by in-group members in the interactions (whereby disagreements are reinterpreted as unintentional errors) means that these skilled migrants are restricted in their access to their new communities.
This article is an introduction to the theoretical and methodological backgrounds of
multimodal (inter)action theory. The aim of this theory is to explain the complexities
of (inter)action, connecting micro- and macro levels of analysis, focusing on the social
actor. The most important theoretical antecedent, mediated discourse analysis (see
Scollon 1998, 2001b), is presented with its key concepts mediated action and modes.
It is shown how action is used as the unit of analysis and how modes are understood
in multimodal (inter)action analysis – as complex cultural tools, as systems of
mediated action with rules and regularities and different levels of abstractness. Subsequently,
methodological basics are introduced, such as lower-level, higher-level and frozen
action; modal density, which specifies the attention/awareness of the social actor; and
horizontal and vertical simultaneity of actions. Horizontal simultaneity can be plotted on
the heuristic model of foreground-background continuum of attention/ awareness. Vertical
simultaneity of actions comprises the central layer of discourse (immediate actions), the
intermediate layer (long-term actions) and the outer layer (institutional or societal contexts).
In short, it is sketched how multimodal (inter)action analysis aims to answer questions
about the interconnection of the different modes on a theoretical as well as on a
practical level.
Multimodality is an innovative approach to representation, communication and interaction which looks beyond language to investigate the multitude of ways we communicate: through images, sound and music to gestures, body posture and the use of space.
The Routledge Handbook of Multimodal Analysis is the first comprehensive ‘research tool kit’ for multimodal analysis, with 22 chapters written by leading figures in the field on a wide range of theoretical and methodological issues. It clarifies terms and concepts, synthesizes the key literature with in-depth exploration and illustrative analysis, and tackles challenging methodological issues.
The Handbook includes chapters on key factors for Multimodality such as technology, culture, notions of identity and macro issues such as literacy policy. The handbook takes a broad look at multimodality and engages with how a variety of other theoretical approaches have looked at multimodal communication and representation, including visual studies, anthropology, conversation analysis, socio-cultural theory, socio-linguistics and new literacy studies.
Detailed multimodal analysis case studies are also included, along with an extensive glossary of key terms, to support those new to multimodality and allow those already engaged in multimodal research to explore the fundamentals further.
The Routledge Handbook of Multimodal Analysis is essential reading for undergraduate and postgraduate students as well as researchers involved in the study of multimodal communication.
In this monograph, the author offers a new way of examining the much discussed notion of identity through the theoretical and methodological approach called multimodal interaction analysis. Moving beyond a traditional discourse analysis focus on spoken language, this book expands our understanding of identity construction by looking both at language and its intersection with such paralinguistic features as gesture, as well as how we use space in interaction. The author illustrates this new approach through an extended ethnographic study of two women living in Germany. Examples of their everyday interactions elucidate how multimodal interaction analysis can be used to extend our understanding of how identity is produced and negotiated in context from a more holistic point of view.
Our perception of our everyday interactions is shaped by more than what is said. From coffee with friends to interviews, meetings with colleagues and conversations with strangers, we draw on both verbal and non-verbal behaviour to judge and consider our experiences.
Analyzing Multimodal Interaction is a practical guide to understanding and investigating the multiple modes of communication, and provides an essential guide for those undertaking field work in a range of disciplines, including linguistics, sociology, education, anthropology and psychology. The book offers a clear methodology to help the reader carry out their own integrative analysis, equipping them with the tools they need to analyze a situation from different points of view. Drawing on research into conversational analysis and non-verbal behaviour such as body movement and gaze, it also considers the role of the material world in our interactions, exploring how we use space and objects - such as our furniture and clothes - to express ourselves. Considering a range of real examples, such as traffic police officers at work, doctor-patient meetings, teachers and students, and friends reading magazines together, the book offers lively demonstrations of multimodal discourse at work.
Illustrated throughout and featuring a mini-glossary in each chapter, further reading, and advice on practical issues such as making transcriptions and video and audio recordings, this practical guide is an essential resource for anyone interested in the multiple modes of human interaction.
Taking the action, rather than the utterance or the text, as the unit of analysis, this article isolates different modes, investigating the interdependent relationships, illustrating that the visual mode of gestures can take up a hierarchically equal or a super-ordinate position in addition to the commonly understood sub-ordinate position in relation to the mode of spoken language. Building on McNeill, Birdwhistell, Eco, and Ekman and Friesen, and using a multimodal interaction analytical approach (Norris), I analyse in detail three separate everyday (inter)actions in which a deictic gesture is being performed and spoken language is used by the social actor performing the gesture. With these examples, I build on previous work in multimodal analysis of texts and multimodal interaction analysis, illustrating that the verbal is not necessarily more important than the visual (Kress and Van Leeuwen; Norris; Scollon), demonstrating that verbal and visual modes can be utilized together to (co)produce one message (Van Leeuwen), and showing that a mode utilized by a social actor producing a higher-level discourse structure hierarchically supersedes other modes in interaction (Norris).
This book explores what speech, music and other sounds have in common. It gives a detailed description of the way perspective, rhythm, textual quality and other aspects of sound are used to communicate emotion and meaning. It draws on a wealth of examples from radio (disk jockey and newsreading speech, radio plays, advertising jingles, news signature tunes), film soundtracks (The Piano, The X-files, Disney animation films), music ranging from medieval plain chant to drum 'n' bass and everyday soundscapes.
It is almost a truism that disagreement produces conflict. This article suggests that perceptions of bias can drive this relationship. First, these studies show that people perceive those who disagree with them as biased. Second, they show that the conflict-escalating approaches that people take toward those who disagree with them are mediated by people's tendency to perceive those who disagree with them as biased. Third, these studies manipulate the mediator and show that experimental manipulations that prompt people to perceive adversaries as biased lead them to respond more conflictually-and that such responding causes those who engage in it to be viewed as more biased and less worthy of cooperative gestures. In summary, this article provides evidence for a "bias-perception conflict spiral," whereby people who disagree perceive each other as biased, and those perceptions in turn lead them to take conflict-escalating actions against each other (which in turn engender further perceptions of bias, continuing the spiral).
Multimodal (inter)action analysis
- Sigrid Norris
- P Albers
- T Holbrook
- A Flint
Norris, Sigrid. 2013a. Multimodal (inter)action analysis. In: Albers P, Holbrook T, Flint A.S. New
Methods in Literacy Research. London, New York: Routledge.
Communicating Religious Belief: Multimodal Identity Production of Thai Vegetarians
- Boonyalakha Makboon
Makboon, Boonyalakha. 2015. " Communicating Religious Belief: Multimodal Identity Production of
Thai Vegetarians. " Auckland, NZ: AUT University.
Kitesurfing: Action, (Inter)action and Mediation
- Jarret Geenen
Geenen, Jarret. 2014. " Kitesurfing: Action, (Inter)action and Mediation. " PhD Thesis, Auckland, NZ:
AUT University.
Interaction -Language and modal configurations in multimodal actions in the art classroom Handbook 'Language in Multimodal Contexts
- Sigrid Norris
Norris, Sigrid. (2016 b). Interaction -Language and modal configurations in multimodal
actions in the art classroom. In: Klug, Nina-Maria and Stöckl, Hartmut (eds). Handbook
'Language in Multimodal Contexts'. Berlin, Boston: de Gruyter Mouton.