No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Multimodal Interaction Analysis
Abstract
Multimodal interaction analysis is a holistic methodological framework that allows the analyst to integrate the verbal with the nonverbal, and to integrate these with material objects and the environment as they are being used by individuals acting and interacting in the world. In short, multimodal interaction analysis allows a researcher to study real people interacting with others, with technology, and with the environment. The point of view taken in multimodal interaction analysis is that all actions in fact are interactions and that all of these (inter)actions are linked to people (referred to as “social actors” in multimodal interaction analysis)—no matter whether you are investigating the real or the virtual world, someone buying ice cream, or using a software program. Multimodal interaction analysis is carried out through a series of analytical phases and steps. Researchers utilize multimodal transcription conventions and analytical tools, which are theoretically founded, allowing for replicability of analyses and reliability of findings. Multimodal interaction analysis allows the integration of all communicative modes, where communicative modes are defined as systems of mediated actions. In this view, all modes of communication together build one system of communication, which comes about through actions and interactions that people produce. This coherent framework and process of analysis can be taught and learned and is proving useful for research examining what people do, how they communicate, and how they interact.
... My analysis draws on multimodal interaction analysis, (Norris, 2004) whose framework offers a coherent approach for dealing with data from different communicative modes during social interaction. The data gathered included the multimodal texts produced by the students, audio-recordings of their conversations as they made these, video-recordings of their presentation of their texts to the class and written notes from the lesson. ...
... Multimodal Interaction Analysis offers a unit of analysis which is applicable to all modes within an interaction: the mediated action (Norris, 2004;Norris, 2016). These actions take three main forms: higher-level actions, lower-level actions and frozen actions. ...
... Analysis drew on the step-by-step approach to multimodal transcription outlined by Norris (2004) to scrutinise complex interactions. To transcribe the use of gesture and gaze, I took still frames from the video for each change of position with, where possible, a mid-change shot, to give a sense of the unfolding physical elements of the interaction. ...
... Classroom CA is not alone in its interest in multimodality. Other approaches including but not limited to Multimodal Interaction Analysis (Mejía-Laguna, 2023;Norris, 2020;Satar & Wigham, 2020Wigham & Satar, 2021;Wilmes & Siry, 2021), Systemic Functional Multimodal Discourse Analysis (Halliday, 2014;Lim, 2019Lim, , 2023O'Halloran & Lim, 2014), New Materialism (Early et al., 2015), and Language Socialization (Burdelski & Howard, 2020) have considered the role visual resources play in classroom interaction. The current study shares at least one major conceptual similarity with these frameworks: semiotic resources such as gaze, gesture, facial expression, and objects are not a priori subordinate to language but are potentially equally or more important than linguistic resources to a given situated action (Mondada, 2018). ...
... The first reason for our choice of CA is its emic, unmotivated, and radically empirical perspective. Instead of taxonomies of potential action (Lim, 2019) or a hierarchy of lower-and higher-level actions (Mejía-Laguna, 2023;Norris, 2020), CA provides tools to document a situated account of participants' orientation to the unfolding interaction by asking 'why that now' (Schegloff & Sacks, 1973). Relying on participants' observable orientation to interaction provides an empirical account of situated pedagogical practice instead of a top-down analysis imposed by the researcher. ...
This conversation analysis (CA) study extends our understanding of the complexity of three turn instructional sequences by investigating the multimodal turn design of a teacher's third turn repetitions (TTRs) and the actions accomplished in the third turn position as well as subsequent post-expansions. The videorecorded data are from an undergraduate Korean as a foreign language classroom at a large US university. The analysis reveals how a teacher coordinates resources such as language, prosody, gaze, gesture, body movements, and objects during and immediately following TTRs to mitigate negative evaluation, direct student attention to trouble sources, and intimate answers. The findings show that actions accomplished by talk, i.e. negative evaluation, and actions accomplished by multimodal resources like gaze, i.e. directing attention, may be undertaken simultaneously. The article contributes to understandings of teaching as complex and contingent interactional work by unpacking in fine-grained detail the moment-by-moment multimodal unfolding of pedagogical practice. We conclude by discussing implications for teacher preparation, namely the central role microanalysis of videorecorded classroom interaction should play.
... Multimodal and multisensory research is an exciting, interdisciplinary field. It includes studies of sound (Hall et al., 2008; Raimbault and Dubois, 2005; Van Leeuwen, 1999), images (Kress and Van Leeuwen, 1996; Machin, 2007; Banks, 2001; Pink, 2001) embodied emplacement (Casey, 1996; Ingold, 2000), human gesture, gaze and movement (Goodwin, 2000; Mondada, 2006; Lancaster, 2001), multimodal discourse (Scollon and Scollon, Scollon, 2001; Norris, 2004;), interaction with objects and environments (Hurdley, 2010; Pahl, 2004; Heath and Vom Lehn, 2008); visual participatory methods (Lomax et al 2011) and children's use of multimodal resources (Lancaster, 2007; Flewitt, 2006; Jewitt, 2006; Pahl, 2006; Rowsell, 2011). Such research investigates the 'multiplicity of how people communicate' (Bezemer and Mavers, 2011: 192): how talk, gesture, gaze and elements of material surroundings combine together in the production of meaning. ...
... Other approaches combine insights from one or more paradigm. Multimodal interactional analysis (e.g. Norris, 2004), for example, draws on interactional sociolingistics (or conversation analysis, a branch of ethnomethodology) and social semiotic multimodality, and social semioticians and interactional linguists have researched and written together (e.g. Bezemer et al 2012). ...
This article discusses three qualitative research traditions concerned with ‘multimodal’ and ‘multisensory’ methods, namely: i) ethnomethodology, ii) multisensory ethnography and iii) social semiotics. These have been selected not because they are the only research domains in which qualitative multimodal methodology is currently developing, but because a comparison between them allows for discussion of methodological and theoretical issues of key importance for advancing the field. Each is argued to rely on a distinctive underlying epistemological commitment – to the study of action, experience and communication, respectively. Each combines linguistic and non-linguistic data to try and get closer to the object of research, but involve different ontologies of closeness. The differences include how the object of research is defined, the role of context, the nature and locus of meaning, the nature of evidence and the relationship between researcher and the object/subject of research. These in turn have implications for how time and space are conceived. The discussion ends by indicating how the respective insights and advantages of each might be synthesised to suggest a new, integrated perspective for this kind of qualitative research.
... To do so, a multimodal approach is followed, employing concepts from different schools of thought: following Multimodal Discourse Analysis (O'Halloran, 2021), the semiotic resources (modes) used in these videos and their combinations in multimodal ensembles are explored. Furthermore, the concept of modal density from Multimodal Interaction Analysis (Bernad-Mechó, 2022;Norris, 2022) is applied. Modal density refers to the number of modes employed at a given moment and/or their intensity. ...
... Prior to the reflective discussion with the teachers following the lesson trials, the researchers examined the classroom recordings. The sequences of teacher-student exchanges where a FL-informed activity was being enacted were identified and the data were analysed drawing on multimodal interaction analysis (S. Norris, 2004). For the analysis, the researchers created transcripts capturing the spoken dialogue in one column with targeted screen captures from the video recordings of classroom actions carried out by the teacher and students. ...
Learning science involves learning how to use specific, conceptual language in talking, reading and writing for reasoning and problem solving (Lemke, 1990). However, research has argued that language is a major barrier to students learning science (Wellington Osborne 2001). In response, this chapter reports on a project in a Singapore secondary school, where researchers worked with five science teachers at different stages over two years to trial a functional literacy approach to the teaching of lower secondary science. The functional literacy approach was informed by systemic functional theory and drew on the work of (Rose Martin, 2012) and (Rose, 2015) for language learning in the science context. This chapter focuses on the shifts in language awareness for teaching science from the teacher reflections collected during the process of implementing ideas from the functional literacy approach to address students’ needs. The reflections indicated a shift from initial reservations to an eventual recognition of the value of how awareness and attention to language can support students’ learning of scientific concepts, develop communicative classrooms and make the knowledge process visible.
... Occupying a lengthy 10.4-s pause, both Tom and Johnny silently stare at Tom' s phone screen. The statuesque frozen action reveals both participants regard the dictionary as the authority (Norris, 2020). ...
Peer interaction constitutes a focal site for understanding learning orientations and autonomous learning behaviors. Based on 10 h of video-recorded data collected from small-size conversation-for-learning classes, this study, through the lens of Conversation Analysis, analyzes instances in which L2 learners spontaneously exploit learning opportunities from the on-task public talk and make them relevant for private learning in sequential private peer interaction. The analysis of extended negation-for-meaning practices in peer interaction displays how L2 learners orient to public repair for their learning opportunities in an immediate manner and in so doing, how different participation framework is being utilized to maximize their learning outcomes. As these extended repair practices are entirely managed by learners themselves, they yield both efficient and inefficient learning outcomes. Findings reveal that learners frequently resort to their peers to recycle the focal trouble words for learning opportunities, shifting their participating role from the on looking audience to active learners. By reporting the rather under-researched post-repair negotiation-for-meaning sequence in peer interactions, the study highlights the relevance between on-task classroom activities and private learning, contributing to understanding private learning behaviors in the language classroom and learning as a co-constructed activity locally situated in peer interaction.
... This involved the researchers observing the classrooms as "outsiders" of the immediate school contexts to study the classroom interactions at hand. Multimodal interaction analysis (Norris, 2004) was carried out on the video recordings of the classrooms. For the analysis, the researchers created transcripts capturing the spoken dialogue in one column with screen captures from the video recordings of VRs drawn by the teacher and students and ready-made VRs projected in presentations at the front of the class in another column. ...
This paper was part of a larger study to examine one primary and one secondary teacher’s classroom. The study analysed the classroom discourse to describe the types of interaction and illustrate how the teachers used talk moves to mediate talk for learning science around visual representations (VRs). The study employed ethnographic approaches to collect and analyse classroom data as “snapshots of practice” in two Singapore classrooms. The findings identified that both teachers featured interactive/authoritative discourse, with the primary teacher using talk moves to unpack science terms through eliciting student observations of diagrams followed by talk moves for students to make connections to the meanings represented in the associated graph. The secondary teacher featured an interactive/dialogic approach, using talk moves around student-generated VRs for students to predict, reason and revise their representations. The findings highlighted the value of examining talk from dialogic and authoritative approaches to better understand how teachers provide opportunities for students to learn science through talk around VRs.
... 1 Multimodality, multimedia, and "meaningmaking" from a Peircean semiotic perspective Multimodality has become a hot topic in applied semiotics, linguistics, and communication studies. Among the key concepts and topics in this field of current research are "multimodal research" (Kress 2010), "multimodal analysis" (Jewitt 2009), "multimodal communication" (Böck 2013), "multimodal discourse" (Kress and Van Leeuwen 2001;Iedema 2003), multimodal grammar (Fricke 2012), "multimodal interaction" (Norris 2004), or "multimodal composition" (Lutkewitte 2013). At the University of London, there is a semiotic Center for Multimodal Research, and worldwide, England, Australia, New Zealand, and Germany are centers of multimodality studies. ...
The paper begins with a survey of the state of the art in multimodal research, an international trend in applied semiotics, linguistics, and media studies, and goes on to compare its approach to verbal and nonverbal signs to Charles S. Peirce’s approach to signs and their classification. The author introduces the concept of transmodality to characterize the way in which Peirce’s classification of signs reflects the modes of multimodality research and argues that Peirce’s classification of the signs takes modes and modalities in two different respects into consideration, (1) from the perspective of the sign and (2) from the one of its interpretant. While current research in multimodality has its focus on the (external) sign in a communicative process, Peirce considers additionally the multimodality of the interpretants, i.e., the mental icons and indexical scenarios evoked in the interpreters’ minds. The paper illustrates and comments on the Peircean method of studying the multi and transmodality of signs in an analysis of Peirce’s close reading of Luke 19:30 in MS 599, Reason’s Rules , of c. 1902. As a sign, this text is “monomodal” insofar as it consists of printed words only. The study shows in which respects the interpretants of this text evince trans and multimodality.
... The originality of AVT is in its mode of transmission, as it is received through two channels at once, acoustic and visual: their synchrony is essential (Bartrina and Espasa 2005). It involves «a systematic interaction between oral, written and non-verbal communication» (Lambert 1994: 23) and, therefore, constitutes living proof of the disappearing boundaries between language and other communicative modes such as images, music or gestures (Kress and van Leeuwen 2001;Norris 2004), looking at language from a less restricted perspective. ...
... Die Leiblichkeit des Sprechens manifestiert sich nicht nur im vokalen Ausdruck . Seit einigen Jahren wächst in der Gesprächsforschung die Einsicht, dass Gespräche nicht einfach im Austausch verbaler Turns zwischen Spre-chern und Hörern bestehen, sondern dass es sich um Interaktionen von multimodal Beteiligten handelt (Goodwin 2000; Norris 2004; Schmitt (Hg.) 2007). Der klassische konversationsanalytische Ansatz erweist sich unter dieser neuen Perspektive als reduktionistisch. ...
What could create more displeasure in the performing arts than silence about power, sexual harassment, bullying, and discrimination? Swedish performing arts has been described as a place where a code of silence rules, a code that has been challenged after the Swedish #MeToo movement of 2017 in film and performing arts. In this article, we aim to theoretically and empirically examine what silence around sexual harassment in the performing arts is, using feminist theory. And further, we discuss how it creates negative affect in the performing arts work environment. Silence or “theatre without action” as described by Rancière, has traditionally been regarded as a valuable creative tool in rehearsal work. However, the #MeToo movements have shone a light on how silence contributes to the prevalence of sexual harassment, bullying, and discrimination in film and the performing arts. Building on discourse analysis of qualitative individual interviews and recordings of rehearsals with employees of four performing arts institutions in Sweden, we examine the discursive and affective shape of silence in the performing arts today. Challenging the idea that great art comes out of suffering, we discuss how silence in the performing arts has consequences that risk being harmful, not only for the employees, but for art itself, as well as the audiences experiencing it.
Background: Patient safety remains a critical concern in the high-stakes environment of the operating room (OR). Human factors and non-technical skills (NTS) play pivotal roles in surgical performance and in preventing errors. Various assessment tools and methodologies have been developed to evaluate NTS among OR personnel, reflecting the growing recognition of their importance. Objectives: To map and analyze NTS among OR personnel during real-life surgeries using medical video recording systems and to evaluate NTS assessment tools described in existing literature. Methods: Four databases, PubMed, Web of Science, Medline and Embase, were searched for relevant studies. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) protocol was followed. Eligibility criteria included studies focusing on video recordings providing an overview of the entire OR with a focus on NTS. Results: Twenty-four articles were included in this scoping review. A total of eight NTS were assessed, most frequently communication (79%), leadership (54%), teamwork (50%) and situational awareness (50%). Observational methods varied, ranging from one or more cameras to more sophisticated medical data recorders, such as the OR Black Box (ORBB). Additionally, a wide variety of assessment methods were used, mostly validated tools (NOTSS, SPLINTS, ANTS), but also broader systems-based frameworks (SEIPS) or self-developed tools. Conclusions: This scoping review highlights the diverse methodologies used to measure NTS in the OR using medical video recordings, indicating a need for further research to address challenges and standardize practices. Despite the proven potential benefits of video recording in the OR, legal, ethical and logistical challenges may serve as possible barriers to actual implementation.
This research thesis reported on the findings of a three-year, three-stage mixed-methods study examining instructors' use of non-linguistic semiotic resources to establish a Community of Inquiry (CoI) in their online subjects. Seven online TESOL instructors and their students at three institutions in two countries were studied and their OLEs were analysed through the lens of multimodality.
The results indicate a shift in CoI presences occurred after the delivery of a professional development intervention, though the non-linguistic semiotic resources that instructors talk about and those that students perceive as contributing to the CoI may differ. Furthermore, the aggregated results of the CoI survey may inform reflective practice undertaken by instructors. The overall findings of the study suggest that the choices instructors make regarding semiotic resource use reflect their beliefs and values in terms of TESOL pedagogy, modelling and the role of English in a global context.
This article presents theoretical concepts and methodological tools from multimodal (inter)action analysis that allow the reader to gain new insight into the study of discourse and interaction. The data for this article comes from a video ethnographic study (with emphasis on the video data) of 17 New Zealand families (inter)acting with family members via skype or facetime across the globe. In all, 84 social actors participated in the study, ranging in age from infant to 84 years old. The analysis part of the project, with data collected between December 2014 and December 2015, is ongoing. The data presented here was collected in December 2014 and has gone through various stages of analysis, ranging from general, intermediate to micro analysis.
Using the various methodological tools and emphasising the notion of mediation, the article demonstrates how a New Zealand participant first pays focused attention to his engagement in the research project. He then performs a semantic/pragmatic means, indicating a shift in his focused attention. Here, it is demonstrated that a new focus builds up incrementally: As the participant begins to focus on the skype (inter)action with his sister and nieces, modal density increases and he establishes an emotive closeness. At this point, the technology that mediates the interaction is only a mundane aspect, taken for granted by the participants.
To understand the role of language in public life and the social process in general, we need first a closer understanding of how linguistic knowledge and social factors interact in discourse interpretation. This volume is a major advance towards that understanding. Professor Gumperz here synthesizes fundamental research on communication from a wide variety of disciplines - linguistics, sociolinguistics, anthropology and non-verbal communication - and develops an original and broadly based theory of conversational inference which shows how verbal communication can serve either between individuals of different social and ethnic backgrounds. The urgent need to overcome such barriers to effective communication is also a central concern of the book. Examples of conversational exchanges as well as of longer encounters, recorded in the urban United States, village Austria, South Asia and Britain, and analyzed to illustrate all aspects of the analytical approach, and to show how subconscious cultural presuppositions can damagingly affect interpretation of intent and judgement of interspeaker attitude. The volume will be of central interest to anyone concerned with communication, whether from a more academic viewpoint or as a professional working, for example, in the fields of interethnic or industrial relations.
In this article, I detail incremental microgenetic alterations in the development of one particular socio-interactive aptitude: making a relevant interactive contribution. Taking heed of Clark’s (2014) call for the need to reorient our attention to investigate the pragmatics of interaction by accounting for the multiple communicative modes through which this is acccomplished I detail the ways in which parental facilitation and a flexible participatory configuration, made possible by video-conferencing technology, create conditions enabling the agentive re-introduction of a psycho-socially relevant topic. Paramount are the ways in which residual interactive specificities in introduction, co-production and multimodal configurations re-manifest suggesting a more symbiotic relationship between traditional notions of ‘message’ and ‘production’. During the microgenesis of interactive aptitudes, children are not just learning what constitutes psycho-socially relevant topoi, they also acquire an understanding of exactly how to make the contribution through multimodal ensembles.
Building on multimodal (inter)action analysis as a theoretical and methodological framework, this article introduces and develops the theoretical/methodological tool called primary agency. Taking the mediated action as a unit of analysis, agency can be analysed as a feature of action. However, there is a lack of empirical approaches for the study of agency, and an overemphasis on language as the most important site for identifying agentive action. I develop primary agency through an analysis of three co-produced higher-level actions from a research project into high school tutoring. These are the higher-level actions of conducting research, tutoring and reading a text. Applying co-production and the modal density foreground/background continuum I explore how the researcher, the tutor and the student co-produce these higher-level actions. Through this analysis, I identify the most significant mediational means for each higher-level action, and the social actor with ownership or agency over these mediational means. I define this social actor as the one with primary agency over the co-produced higher-level action. Finally, my analysis outlines the implications of primary agency for co-produced higher-level actions, including the role of the researcher, the attention/awareness participants pay to overarching research projects, and links between primary agency and successful learning.
This paper is part of a larger scale project where I explore the structure of academic lecture. The focus of the study here presented is to investigate the structure and organization of a university lecture through the introduction of new topics. One of the tools traditionally referred to as an organizer of discourse is metadiscourse (Crismore et al. 1993. Metadiscourse in persuasive writing: A study of texts written by American and Finnish university students. Written Communication, 10:39–71; Vande Kopple. 1985. Some exploratory discourse on metadiscourse. College Composition and Communication, 36(1):82–93). Although metadiscourse has been studied from a wide range of perspectives (Hyland. 2005. Metadiscourse: exploring interaction in writing. London, England: Continuum), these analyses have most of the time been conducted from a purely linguistic point of view and neither the speaker as a social actor nor metadiscourse as part of a multimodal interaction are taken into account. That being so, the aim of this study is to explore the role played by
This article provides a preliminary answer to exactly why video-conferencing is evaluated as better than traditional telephony for long-distance familial interaction by allocating analytical attention to the showing of objects during interaction. While it is acknowledged that ‘showing’ constitutes an interactive move less contingent on linguistic maturation, more importantly, the showing of objects, artefacts or entities during video-conferencing interactions exemplifies an agentive and volitional production of identity elements on behalf of young children. Thus, while some have pointed to shortcomings of conversation-like activities mediated by video-conferencing in favour for more activity-driven tasks, I make a case for drawing upon pre-existing components of the material surround as a means to more comprehensively and longitudinally engage younger children in video-conferencing interaction.
This article takes a multimodal approach to examine how two young men communicate knowledge, shift attention, and negotiate a disagreement via videoconferencing technology. The data for the study comes from a larger ongoing project of participants engaging in various tasks together. Linking micro, intermediate and macro analyses through the various methodological tools employed, the article presents multimodal (inter)action analysis (Norris, 2004, 2011, 2013a, 2013b) as a methodology to gain new insight into the complexity of knowledge communication via videoconferencing technology, which is relevant to many settings from education to employment, from organizations to gaming.
This article develops a new methodological tool, called scales of action, which allows the empirical investigation of ubiquitous actions such as driving on the one hand, and the highly complex relationships between (for example) drives and other actions in everyday life on the other hand. Through empirical analysis of ethnographic data of drives performed by a German artist and an American IT specialist, the article illustrates how talk and driving are embedded differently in different cultural contexts. Examining the actions of the two drivers before, during, and after a drive further demonstrates that chronologically performed actions are not necessarily sequential in nature. Using a mediated discourse theoretical approach and building upon multimodal (inter)action analysis, the article provides analysts with a tool that captures the inherent complexities of everyday actions. Through the notion of scales of action and their composition, this article sheds new light upon the complexity and cultural differences of drives and car talk in middle class Germany and North America.
Researchers seeking to analyse how intersubjectivity is established and maintained face significant challenges. The purpose of this article is to provide theoretical/methodological tools that begin to address these challenges. I develop these tools by applying several concepts from multimodal (inter)action analysis to an excerpt taken from the beginning of a tutoring session, drawn from a wider data set of nine one-to-one tutoring sessions. Focusing on co-produced higher-level actions as an analytic site of intersubjectivity, I show that lower-level actions that co-constitute a higher-level action can be delineated into tiers of materiality. I identify three tiers of materiality: durable, adjustable and fleeting. I introduce the theoretical/methodological tool tiers of material intersubjectivity to delineate these tiers analytically from empirical data, and show how these tiers identify a multimodal basis of material intersubjectivity. Building on this analysis I argue that the durable and adjustable tiers of material intersubjectivity produce the interactive substrate, which must be established in order for actions that display fleeting materiality to produce intersubjectivity. These theoretical/methodological tools extend the framework of multimodal (inter)action analysis, and I consider some potential applications beyond the example used here.
In this article, we take a multimodal (inter)action analytical approach, showing how objects in everyday life are identity telling. As social actors surround themselves with objects, multiple actions from producing the objects to acquiring and placing them in the environment are embedded within. Here, we investigate examples from two different ethnographic studies, using the notion of frozen actions. One of our examples comes from a five-month long ethnographic study on identity production of three vegetarians in Thailand (Makboon, forthcoming); and the other example comes from a four-month long ethnographic study of three working parents on the East coast of North America (Norris, 2006, 2008). We illustrate the frozen actions embedded in particular objects and argue that the analysis of frozen actions allows us to partially understand how identity is produced and experienced by social actors in everyday life.
Moving towards multimodal mediated theory, I propose to define a mode as a system of mediated action that comes about through concrete lower-level actions that social actors take in the world. In order to explain exactly how a mode is a system of mediated action, I turn to a perfume blog and use one blog entry as my starting point. The mode that I primarily focus on in this article is the mode of smell, explicating that the mode of smell is not synonymous with olfactory perception, even though modal development of smell is certainly partially dependent upon olfactory perception.
As I am ostensibly focusing on the one mode, I once again problematize this notion of countability and delineate the purely theoretical and heuristic unit of mode (Norris, 2004). I clarify that modes a) do not exist in the world as they are purely theoretical in nature; b) that modes can be delineated in various ways; and c) that modes are never singular.
Even though the concept of mode is problematical – and in my view needs to always be problematized – I argue that the term and the notion of mode is theoretically useful as it allows us to talk about and better understand communication and (inter)action in three respects: 1. The notion of mode allows us to investigate regularities as residing on a continuum somewhere between the social actor(s) and the mediational means; 2. The theoretical notion of mode embraces socio-cultural and historical as well as individual characteristics, never prioritising any of these and always embracing the tension that exists between social actor(s) and mediational means; and 3. The theoretical notion of mode demonstrates that modal development through concrete lower-level actions taken in the world, is transferable to other lower-level actions taken.
Our perception of our everyday interactions is shaped by more than what is said. From coffee with friends to interviews, meetings with colleagues and conversations with strangers, we draw on both verbal and non-verbal behaviour to judge and consider our experiences.
Analyzing Multimodal Interaction is a practical guide to understanding and investigating the multiple modes of communication, and provides an essential guide for those undertaking field work in a range of disciplines, including linguistics, sociology, education, anthropology and psychology. The book offers a clear methodology to help the reader carry out their own integrative analysis, equipping them with the tools they need to analyze a situation from different points of view. Drawing on research into conversational analysis and non-verbal behaviour such as body movement and gaze, it also considers the role of the material world in our interactions, exploring how we use space and objects - such as our furniture and clothes - to express ourselves. Considering a range of real examples, such as traffic police officers at work, doctor-patient meetings, teachers and students, and friends reading magazines together, the book offers lively demonstrations of multimodal discourse at work.
Illustrated throughout and featuring a mini-glossary in each chapter, further reading, and advice on practical issues such as making transcriptions and video and audio recordings, this practical guide is an essential resource for anyone interested in the multiple modes of human interaction.
This book explores what speech, music and other sounds have in common. It gives a detailed description of the way perspective, rhythm, textual quality and other aspects of sound are used to communicate emotion and meaning. It draws on a wealth of examples from radio (disk jockey and newsreading speech, radio plays, advertising jingles, news signature tunes), film soundtracks (The Piano, The X-files, Disney animation films), music ranging from medieval plain chant to drum 'n' bass and everyday soundscapes.