No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Multimodal interaction from a conversation analytic perspective
Abstract
This special issue of the Journal of Pragmatics has its origins in the International Conference on Conversation Analysis 10 (ICCA10), which took place in Mannheim (Germany) in July 2010. More than 650 scholars attended the conference, whose theme was ‘‘multimodal interaction’’. This volume includes papers based on the four plenary talks given at ICCA10 and four additional contributions related to the conference theme.
... Since the beginning of the 21st century, the use of video recordings as data (e.g., Heath et al., 2010;Knoblauch et al., 2006;Nevile, 2004) has become increasingly common in EMCA. Investigations have, in addition to talk, focused more on the multimodal features of social interaction both in mundane and institutional contexts (Deppermann, 2013;Mondada, 2014Mondada, , 2016aMondada, , 2016bMondada, , 2019b. Analyses have focused on, for example, gestures, gaze behaviour, facial expressions, body postures, object manipulations, movement in space (Deppermann, 2013;C. ...
... Investigations have, in addition to talk, focused more on the multimodal features of social interaction both in mundane and institutional contexts (Deppermann, 2013;Mondada, 2014Mondada, , 2016aMondada, , 2016bMondada, , 2019b. Analyses have focused on, for example, gestures, gaze behaviour, facial expressions, body postures, object manipulations, movement in space (Deppermann, 2013;C. Goodwin, 2017;Nevile, 2015;Streeck et al., 2011), and, more recently, on "sensoriality" (e.g., Cekaite, 2015;M.H. Goodwin, 2017;Mondada, 2019aMondada, , 2020. ...
... Asynchronicities between modalities do not seem to be accidental. Activities performed in one modality may not have the exact boundaries of action as those performed in another modality (Deppermann, 2013). Additionally, sequentiality may not be organized strictly successively; it relies on the prior and subsequent actions in real time, and coordinated, simultaneous multimodal interactions are intertwined (Mondada, 2016). ...
... Furthermore, actions are organized not only by individual speakers but also within social interaction (Mondada, 2016). Thus, we must consider co-participants' simultaneous multimodal activities and managing action sequences in CA (Deppermann, 2013;Schegloff, 2007). We use these principles from multimodality to frame the current investigation. ...
This study investigates how learners collaboratively construct embodied geometry knowledge in shared VR environments. Three groups of in-service teachers collaboratively explored six geometric conjectures with various virtual objects (geometric shapes) under the guidance of a facilitator. Although all the teachers were in different physical locations, they logged into a single virtual classroom with their respective groups and were able to see and manipulate the same geometric shapes as well as see their collaborators’ avatars and actions on the shapes in real time in the shared virtual space. This paper introduces a novel multimodal data analysis method for analyzing participants’ interactive patterns in collaborative forms of actions, gestures, movements, and speech. Results show that collaborative speech has a strong simultaneous relationship with actions on virtual objects and virtual hand gestures. They also showed that body movements and positions, which often focus on virtual objects and shifts in these movements away from or around the object, often signal key interactional collaborative events. In addition, this paper presents five emergent multimodality interaction themes showing participants’ collaborative patterns in different problem-solving stages and their different strategies in collaborative problem-solving. The results show that virtual objects can be effective media for collaborative knowledge building in shared VR environments, and that structured activity design and moderate realism may benefit shared VR learning environments in terms of equity, adaptability, and cost-effectiveness. We show how multimodal data analysis can be multi-dimensional, visualized, and conducted at both micro and macro levels.
... In recent times, the paradigm of embodiment 4 has increasingly led semiotic, cognitive, and interactional fields of research to focus their work on multimodal talk-in-interaction (Deppermann, 2013;Müller et al., 2013;Goodwin, 2007;Schröder, 2017Schröder, , 2022Schröder & Streeck, 2022). Although the use of this label can be criticised for maintaining a centrality for talk, it is nonetheless laudable that increasing attention in conversation analysis has been brought to the relevance of the material environment, embodied activities, as well as environmentally coupled gestures (Mondada, 2019;Goodwin, 2018). ...
... Although the use of this label can be criticised for maintaining a centrality for talk, it is nonetheless laudable that increasing attention in conversation analysis has been brought to the relevance of the material environment, embodied activities, as well as environmentally coupled gestures (Mondada, 2019;Goodwin, 2018). Along these lines, social interaction is conceived as locally organised and achieved incrementally both in simultaneity and in temporal-sequential unfolding, as well as through the co-deployment of embodied multisensorial resources by the co-participants in situ (Goodwin, 2000(Goodwin, , 2007Mondada, 2013Mondada, , 2019Deppermann, 2013). ...
Our contemporary reality has given rise to new forms and styles of interaction. Human beings face the challenge of coming to terms with globalisation, hybridity, transnational circulation of practices and people, and the development of virtual spaces on the one hand and, on the other hand, with new kinds of borders arising out of these processes as an attempt to keep communities “intact.” This book addresses this new reality in a bottom-up approach that originates from the editors’ collaboration on the project Intercultural Communication in Interaction: Multimodal Approaches, funded by the Worldwide Universities Network Research Development Fund in 2019. It gathers different voices and approaches to the study of multimodal communication in intercultural interaction through the contributions both of authors who had already significantly contributed to key research in the area and of authors who took their work in new directions as a result of connections they made over the course of the project.
... Tomando esta última clasificación, es posible extraer una primera comparativa entre el discurso de los MOBA y la conversación coloquial: En resumen, aunque hay similitudes, ambos presentan diferencias clave. Primero, el canal, puesto que la conversación es principalmente oral (y kinésica [Poyatos, 2018;Depperman, 2013;Depperman y Streeck, 2018]) mientras que el videojuego se nutre de todas las entradas de información disponibles en la interfaz, incluyendo los movimientos de sus avatares (Yus, 2010(Yus, y 2011Greiner, Caravella y Roth, 2014;Herring, 2017). En segundo lugar, la temática es un elemento indeterminado en el primer género, hecho que contrasta con el MOBA, ya que es una clase de discurso topic-driven (Herrin et al., 2009). ...
La comunicación es un fenómeno complejo que varía según la situación, hecho aún más frecuente desde la irrupción de los medios digitales. Este artículo explora un aspecto de esta nueva dinámica de interacción virtual: la interacción en los videojuegos MOBA en relación con la conversación cara a cara. En concreto, se compararán la frecuencia, densidad y extensión de las intervenciones a través de un corpus propio recogido con muestras de ambos discursos. El análisis se basa en el modelo Val.Es.Co (2010, 2014; Pons, 2022), que facilita el estudio de rasgos situacionales y segmentación de la oralidad. De esta forma, tomando en consideración las características generales de los videojuegos (Stanton, 2015; Hiloko et al. 2021; Badia, 2023) y estudios previos sobre discursos mediados por ordenador (Herring, 2007; 2017; Yus, 2010 y 2011), se podrá estudiar la extensión de las intervenciones y la frecuencia de inicio de nuevos diálogos en ambos contextos.
... These bodily movements play an important role as key interactional resources with which participants make sense of each other. How embodied actions act as interactional resources to achieve both professional and mundane social activities remains a relevant topic of investigation among scholars who are interested in analyzing multimodality in human interaction (Deppermann, 2013;Heath & Luff, 2013;Streeck et al., 2011). Previous studies have revealed that bodily actions are organized in an orderly manner to achieve specific interactional goals (Sacks & Schegloff, 2002). ...
Despite the growing interest in examining the roles of multimodal practices in L2 interaction and language learning (Hall & Looney, 2019; Jacknick, 2021; Lilja, 2022), few studies have been conducted on tracking down teacher’s use of recurrent embodied practices utilized in an educational setting over lessons and how students orient to it. This study examines a teacher’s systematic use of a specific gesture and embodiment through closely observing classroom interactions between an experienced EFL teacher and young learners in Japan. The analysis focuses on a recurrent hand gesture, which will be termed as a microphone gesture, that is utilized mainly as an interactional resource to allocate turns and moderate speaker shifts. The aim of the study is twofold: a) to describe the orderliness of the embodied practice employed by the teacher in terms of managing turn-taking and b) to show how the gesture is used to achieve pedagogical goals. 教室会話におけるマルチモーダルな実践の記述への関心が高まっているにも関わらず (Hall & Looney, 2019, Jacknick, 2021, Lilja, 2022)、教育現場で教師が授業中に使用するジェスチャーを追跡し、学習者がそれに対してどのように志向しているかについての研究はこれまであまり行われていない。そこで、本稿では教師と生徒間のやりとりを詳細に分析することで、教師がマルチモーダル実践を体系的に使用していることを検証する。特に、本教室で繰り返し使われるハンドジェスチャー：マイクロフォン・ジェスチャーに焦点を当て、話者の順番交替を調整するための相互作用的資源として、どのように利用されているかを分析する。特に、a) 相互行為における順番交替の観点から、教師が採用する身体的実践の秩序性を記述すること、b) 同時にジェスチャーがどのような教育目的を達成しているのかを明らかにすることを目的とする。
... Numerous studies have demonstrated the applicability of CA concepts to online interactions. This is evident in special issues of various journals, including those by Deppermann (2013), Bou-Franch and Garcés-Conejos Blitvich (2014), Gerhardt et al. (2014), Arminen et al. (2016), and Giles et al. (2017). Specifically, Giles (2015) argued for adopting a digital CA framework to study online data. ...
... Numerous studies have demonstrated the applicability of CA concepts to online interactions. This is evident in special issues of various journals, including those by Deppermann (2013), Bou-Franch and Garcés-Conejos Blitvich (2014), Gerhardt et al. (2014), Arminen et al. (2016), and Giles et al. (2017). Specifically, Giles (2015) argued for adopting a digital CA framework to study online data. ...
... technical functions (playing louder, with more intensity, etc.), but also other functions concerning the sequential, action formational and social character of musical instruction and performance settings. Using the method of Multimodal Conversation Analysis (Deppermann 2013;Mondada 2013), the present study works with multimodal transcripts (Mondada 2018a) to study: ...
The paper argues for the importance and richness of gaze communication in two different musical settings: orchestra rehearsals and chamber music lessons. Of particular interest are the role and functions of gaze within orchestra rehearsals and chamber music lessons, focusing on the relationship between the institutional setting, participants' spatial orientation, the presence of the score, and gaze behaviour. The theoretical framework for this study is Multimodal Conversation Analysis. While previous research within this approach primarily considers gaze direction and shifts for turn-taking and participation roles, the present study extends its scope to include various eye movements and expressions, such as closing, widening, squinting, or squeezing eyes-especially used by conductors in orchestra rehearsals. Key questions addressed include the strength of gaze in both settings as a contextualization resource for participation roles and interactional purposes and its use to perform other diverse actions, such as conveying emotional states or structuring the interaction. Through these inquiries, the study seeks to deepen the understanding of the multifaceted role of gaze in musical-instructional interactions and its implications for conductor-musician(s) and professor-student(s) dynamics.
... The data were analyzed within the methodological framework of multimodal conversation analysis (Deppermann, 2013(Deppermann, , 2018Mondada, 2016Mondada, , 2019Goodwin, 2017). Multimodal conversation analysis focuses on the organization of different semiotic resources (speech, gestures, gaze, body movements, etc.) and their possible combinations and arrangements in complex multimodal gestalts. ...
Vocalizations are a central resource for instructing in orchestra rehearsals. Conductors use them to make the musicians understand what they want to hear, i.e., through singing and rhythmic vocalizations they imitate or depict the envisaged musical qualities. In this contribution, I examine what vocalizations of orchestra conductors look like and how they are embedded in the instructional interaction between conductor and musicians. Based on a corpus of orchestra rehearsals in France and Italy, the paper uses multimodal conversation analysis to describe the vocal resources used by the conductors for singing, the interactional and sequential organization of vocalizations, as well as their functional properties. Vocalizing appears to often be accompanied by other semiotic resources, such as gestures and gaze, in order to form vocal-gestural demonstrations that embody musical aspects, e.g., tempo, phrasing, articulation, etc. The analysis reveals that vocalizations are employed by conductors for various purposes that exceed the simple action of imitating the music. Data are in French, Italian and English.
... We discovered that the participants carefully and minutely attended to each other's gazing behaviors as a method for understanding their co-participant's actions at the micro-moments of conversations and deployed relevant embodied actions in response in fine-tuned ways. Deppermann (2013) argues that such a micro-sequential perspective, which requires closer attention to the multimodal actions and the coordination of verbal and embodied resources, is a challenge to the speaker-hearer dichotomy and to seeing the sequential organization only at the verbal turns. This aligns with the main impact of the IC assessment research on the overall language assessment literature in terms of the speaker-hearer dichotomy. ...
This paper deals with an end-of-semester interactional competence (IC) assessment of L2 oral communication skills for higher education students. Despite the growing research interest in L2 IC assessment using paired role-play tasks, the non-verbal actions of the learners in these contexts remain largely unexplored. With this in mind, this study uses multimodal Conversation Analysis to examine gazing at role-cards in orienting to, structuring, and navigating through the assessment situations, and displaying IC by attending to task initiation, asking/giving task-relevant details, eliciting/making/delaying offers, and ending the conversation. The findings bring new insights into multimodal aspects of L2 IC assessment.
... It also allows the study of how positions may be taken up, questioned, contested, or refused. Positioning Theory has sometimes been criticized of missing fine-grained empirical analysis of concrete recorded social interaction (e.g., Deppermann, 2013). However, as described by McVee and Van Langenhove, Chapter 1, and multiple other chapters in this volume, there exists in the meanwhile a large body of empirical research drawing on Positioning Theory, particularly in the field of educational research (see also McVee, 2017;McVee et al., 2019). ...
... Given the novelty of acoustemological research in organization studies, we propose an original analysis protocol inspired by two methods: (a) multimodal conversation analysis (Deppermann 2013) and (b) social semiotics of sound (Van Leeuwen 1999). Following these methods, data from observations and listening have been analyzed according to: (a) the temporalities of events; (b) the kind of events (speech, sound, gesture); and (c) the interactions or concurrent actions. ...
Drawing on ethnography, this study investigates the treatment of blind and visually impaired people (BVIP) in the workplace adopting a sociomaterial framework based on acoustemology. This approach concerns the process of knowing with and through sound. In line with interest in multimodality within organization studies, acoustemology recognizes the auditory as a way to access systems of meanings, negotiations, co-constructions, discrimination and culture within organizations. Considering visual impairment as a culture, this research explores the way in which sound and sonic technologies act as relational (both social and material) channels through which BVIP conduct themselves in the workplace, interact with sighted co-workers, gain recognition and produce and reproduce a system of meanings. Through acoustemology, this study contributes to the issue of organizational inclusion of people with disability proposing dis-continuity, a concept that helps explore inclusion as a practice that involves alternative epistemologies and brings about changes in organizational culture.
... gazes, mimicking, pointing, posturing, in combination with words, prosody, etc (e.g. Deppermann, 2013;Hazel et al., 2014;Richards, 2004). Multi-participants take each other's bodies into account when building relevant action in concert with each other. ...
... Episodes with child-initiated informings were transcribed and analyzed in terms of turn design, turn-taking procedures, actions, and multimodal aspects of interaction (cf. Deppermann, 2013;Mondada, 2018). Such informings occurred regularly during transitions between structured activities that were characterized by the children's and teachers' preoccupation with some manual, physical, or organizational tasks, for example, dressing, walking, or waiting for others, which allowed children to approach teachers outside of strictly organized educational interactions. ...
... Gaze directions are seen as crucial for the development of this stretch of talk and marked in the graphic transcript. Also, for that reason, the graphic transcript is made with a rather high granularity (Deppermann, 2013) i.e., many panels per time unit. In any face-to-face encounter, directions, frequencies, and duration of interlocutors' gaze are considered significant (Kaneko and Mesch, 2013;Kendon, 1967;Kleinke, 1986). ...
This paper describes practices for repair receipt in sequences of other-initiation of self-repair in informal Norwegian Signed Language multiperson conversation. Its main foci are how signers mark their (now-)understanding by employing upward or downward nods and withdrawal of mutual gaze and upper body. The multimodal analysis is illustrated with multilinear, glossed transcripts, comic-strip inspired graphic transcripts and video-clips. The analysis is conducted on a collection of 112 cases of other-initiation of self-repair. A simple quantitative breakdown shows the distribution of different embodied practices across different sequential positions, and some deviant cases are described. Among the findings is that even though these explicit repair receipts sometimes occur as responses to non-closing (failed) self-repairs, they are far more common in the closing cases. Whether the trouble-source turn is a first-pair part, a second-pair part or a telling also influences whether an explicit repair receipt is produced. Data are in Norwegian Sign Language with English translations.
... Traditionally, these recordings have been audio, but since the arrival of good and cheap video-recording material, CA has begun to investigate the multimodal achievement of social action and researchers have been paying increasing attention to the sequential importance of gesture, space, and material objects in interaction (for more on this see Deppermann, 2013). ...
Leadership research often uses interviews as a way of gaining access to organisational players’ lay theories of leadership, i.e. who they consider leaders to be and what they consider leadership is. Such an approach elicits post hoc reflection and thus runs the risk of recycling existing discourses of leadership. Conversely, in this chapter, we use membership categorisation analysis to investigate the way in which, as part of their everyday workplace activities, organisational players categorise some people as (non-)leaders and some events as the doing of (non-)leadership. The findings we present challenge classic, and widespread, theories of heroic leadership and reveal that, from an emic perspective, who leaders are and what they do is quite banal. We also point out that this category work is shot through with moral judgement. It is used to “do” criticism and to hold the leadership team accountable for not doing leadership. As a result of such action-oriented category work, the subordinates agree to take action to influence the leadership team to redress an organisational failing. Significantly, such proposed action from subordinates turns classic theories of heroic leadership, which often implicitly equate leadership with the hierarchical superior and top-down influence, on their head and invites critical reflection on the “just whatness” of leadership.KeywordsLeadershipMembership categorisation analysisLeader identityDiscursive leadership
... Traditionally, these recordings have been audio, but since the arrival of good and cheap video-recording material, CA has begun to investigate the multimodal achievement of social action and researchers have been paying increasing attention to the sequential importance of gesture, space, and material objects in interaction (for more on this see Deppermann, 2013). ...
In this concluding chapter, we revisit the main themes and insights from the book in the context of organisational healing. The chapter demonstrates why language and communication matter for organisational healing, a process that involves restoring interpersonal connections and renewing organisational practices, and which is essential for fostering positive change and growth within organisations. Exploring the extended metaphor of organisational healing, we showcase how discourse-linguistic approaches can help researchers and practitioners to reveal the causes and symptoms of organisational trauma, and how a social constructionist understanding of organisational problems can help better implement the healing process. We introduce the concept of diagnostic listening, concluding the chapter with thoughts about the importance of criticality and language awareness.KeywordsOrganisational healingSocial constructionismDiagnostic listeningDiscourse analysis
... Traditionally, these recordings have been audio, but since the arrival of good and cheap video-recording material, CA has begun to investigate the multimodal achievement of social action and researchers have been paying increasing attention to the sequential importance of gesture, space, and material objects in interaction (for more on this see Deppermann, 2013). ...
This chapter highlights the importance of combining approaches to metaphor analysis based on developments in the study of metaphors in organisational and discourse-focused disciplines. The chapter showcases a proposed method through the analysis of interviews with two employees and demonstrates why it is important to consider close linguistic and broader social context where metaphors are being used. The argument also underlines the importance of considering what metaphors can reveal about organisational realities and the role they play in shaping organisational culture and communication. By highlighting the methodological considerations involved in metaphor analysis in organisational contexts, this chapter contributes to the development of a more rigorous and nuanced approach to studying metaphors both in organisational and in discourse literature.KeywordsMetaphor analysisConceptual metaphorLinguistic metaphorOrganisational change
... Traditionally, these recordings have been audio, but since the arrival of good and cheap video-recording material, CA has begun to investigate the multimodal achievement of social action and researchers have been paying increasing attention to the sequential importance of gesture, space, and material objects in interaction (for more on this see Deppermann, 2013). ...
This book showcases various methodological approaches to the analysis of organizational talk and text. Arguing that organizations are discursive constructions that are communicatively constituted, the authors use the analysis of transcripts of audio-recordings of naturally-occurring workplace talk and authentic written texts to demonstrate what applied linguistics has to offer to scholarly research into organizations as well as management practice and training. The authors discuss the theoretical underpinnings of discursive approaches to the role language in the communicative constitution of organization, and then each chapter focuses on one particular analytical approach. The chapters cover conversation analysis; membership categorization analysis, positioning theory; ventriloquism; metaphor analysis; and metadiscourse analysis and computer-mediated discourse analysis. Consequently, this interdisciplinary work presents a number of methods that allow researchers unfamiliar with fine-grained linguistic analyses of naturally-occurring talk and text to explore ways of adding to their repertoire of research skills.
... However, this proposal is not new to EMCA. Since the analyst "inevitably trades on his members' knowledge in recognising the activities that participants to interaction are engaged in" (Turner, 1970, p. 177), membership knowledge may better be seen as a prerequisite for analysing interaction in any professional setting (Deppermann, 2013;ten Have, 2002; see also Garfinkel, 2002, for the "unique adequacy requirement"). ...
Error-correction in sport coaching consists of the following phases: (a) correction initiation, (b) error-identification, (c) solution proposal, and (d) practice resumption. Framed by multimodal conversation analysis, this article adopts a single-case analysis to examine an extended correction sequence in Muay Thai coaching. First, I illustrate the opening phase in which the participants negotiate the norms regarding the training procedure. I then examine how the normative organisation of correction and the coach's use of the previously-reported interactional practices are both fitted to the local contingencies of the setting. Finally, I demonstrate how the indexicality of Coach's correction can be remedied by members' practical knowledge about Muay Thai. I discuss that members' methods may have diversified in sedimented landscapes as instances of correction accumulate, refining the way in which Muay Thai practitioners teach embodied skills that constitute the work of their community.
... Multimodal interaktionsanalys används som metod och genom det sätts deltagarperspektivet i fokus, det vill säga vad deltagarna själva orienterar sig mot i interaktionen. En viktig del av multimodal interaktionsanalys är det etnografiska arbetet (Deppermann 2013). I denna studie spelar det etnografiska arbetet en viktig roll för den etnografiska kunskapen som behövs för att förstå den aktivitet som undersöks, särskilt då jag som forskare inte är en kompetent medlem av den aktivitet som undersöks (jfr Melander 2009, Norrthon 2020. ...
Hörrni ska vi göra: dada da boom» Epistemiska positioneringar under en skapandeprocess i en showdansklass JESSICA DOUGLAH 1. Inledning För att kunna uppträda med en dansföreställning måste dansare öva in den koreografi de ska uppträda med och denna process är både en skapandesituation och en lärandesituation. En förutsättning är att dansarna tillsammans med läraren uppnår en gemensam förståelse för hur dansen ska genomföras. Denna förståelse är främst lärarens ansvar att nå fram till då dansläraren har det yttersta ansvaret för aktiviteten (Prior 2000). Dansklasser har en tradition av att vara lärarstyrda, särskilt när det gäller teknikträningar för dansare (Råman 2009). Metoden att dansarna endast ska kopiera läraren har fått mycket kritik då dansarna inte får möjlighet att utveckla ett kritiskt tänkande på egen hand (Kimmerle och Côté-Laurence 2003) men även för att dansarna inte får utlopp för att vara kreativa och delaktiga i den skapande processen (Buckroyd 2000). Det finns emellertid forskning som visar på att den mer lärarstyrda dansklassen fungerar väl för dansarna som är tidigt i sin dansinlärning medan mer avancerade dansare gynnas av att få vara mer delaktiga i sitt lärande (Kimmerle och Côté-Laurence 2003). Aktiviteten dansklass är organiserad av en ansvarig deltagare, läraren, vilken har en kunskap om aktiviteten och är engagerad i att visa, förklara och kommentera. Eftersom dansklasserna leds av en lärare finns det förväntade roller, såsom att läraren förväntas ha mer kunskap än dansarna. Men som en del av en kreativ och skapande aktivitet innehåller dansklasser moment där läraren även vill inkludera dansarna i processen. Begreppet demonstration används för alla situationer där läraren till exempel ska kommentera, visa något, beskriva eller gör olika kombinationer av dessa resurser. Demonstrationer i dansklasser är multimodala (jfr Douglah 2020; Douglah 2021) och utgör en av grunderna för undervisning i dansklasser. De multimodala demonstrationerna är särskilt intressanta att studera då en dansklass är en förkroppsligad aktivitet och detta påverkar även hur demonstrationer förmedlas. Intresset ligger vid de vid olika verktyg som används i dessa lärandesituationer snarare än dansarnas faktiska lärande. Det övergripande syfte med studien är att visa på hur deltagarna i showdansklasser 1 kommer fram till vad som är framförbart. För att kunna svara mot syftet används multimodal 1 Mer om dansstilen under rubriken 2.1.
... Schließlich haben in den letzten Jahren Ansätze wie die Gesprächsforschung, die Konversationsanalyse und die Gestenstudien die differenzierten Analysen von Interaktion unter multimodaler Perspektive erforscht (vgl. Fricke 2012;Deppermann 2013;Mondada 2016;Deppermann & Streeck 2018;Norris 2019). Übersichten, Einführungen und kritische Diskussionen zu vieler dieser Arbeiten finden sich beispielsweise in Bateman (2014), Stöckl (2016), Klug (2016), Adami (2017) und Wildfeuer et al. (2020. ...
... In this approach, purposive interaction establishes actors' relationships. The conversation analysis approach (Deppermann, 2013;Mondada, 2019) explores how participants organize their actions sequentially. The base assumption of this approach is that social acts can be understood by the acts before and after them. ...
The emergence of immersive digital technologies, such as shared Augmented Reality (shAR), Virtual Reality (VR) and Motion Capture (MC) offers promising new opportunities to advance our understanding of human cognition and design innovative technology-enhanced learning experiences. Theoretical frameworks for embodied and extended cognition can guide novel ways in which learning in these environments can be understood and analyzed. This conceptual paper explores a research method in Educational Technology-multimodal analysis for embodied technologies-and provides examples from shAR, VR, and MC projects that use this approach. This analysis involves tracking learners' gestures, actions on physical and virtual objects, whole body movements and positions, and their talk moves, in addition to other relevant modalities (e.g., written inscriptions), over time and across space. We show how this analysis allows for new considerations to arise relating to the design of educational technology to promote collaboration, to more fully capture students' knowledge, and to understand and leverage the perspectives of learners.
... Using multimodal conversation that includes verbal and non-verbal cues could potentially enhance perceived human-like behaviors and social presence of the chatbot [71]. CommunityBots could be integrated with features and functionalities to process multimodal conversations to better understand user intentions and simulate natural conversations among humans [31] through identification of non-verbal cues [62], which constitute 93% of communication conveyed by humans [77]. However, previous studies found that there is a growing concern among a group of chatbot users due to the push towards automating conversation that focuses on being mechanically efficient with less emphasis on human touch -such as empathy and affability [27,93]. ...
In recent years, the popularity of AI-enabled conversational agents or chatbots has risen as an alternative to traditional online surveys to elicit information from people. However, there is a gap in using single-agent chatbots to converse and gather multi-faceted information across a wide variety of topics. Prior works suggest that single-agent chatbots struggle to understand user intentions and interpret human language during a multi-faceted conversation. In this work, we investigated how multi-agent chatbot systems can be utilized to conduct a multi-faceted conversation across multiple domains. To that end, we conducted a Wizard of Oz study to investigate the design of a multi-agent chatbot for gathering public input across multiple high-level domains and their associated topics. Next, we designed, developed, and evaluated CommunityBots - a multi-agent chatbot platform where each chatbot handles a different domain individually. To manage conversation across multiple topics and chatbots, we proposed a novel Conversation and Topic Management (CTM) mechanism that handles topic-switching and chatbot-switching based on user responses and intentions. We conducted a between-subject study comparing CommunityBots to a single-agent chatbot baseline with 96 crowd workers. The results from our evaluation demonstrate that CommunityBots participants were significantly more engaged, provided higher quality responses, and experienced fewer conversation interruptions while conversing with multiple different chatbots in the same session. We also found that the visual cues integrated with the interface helped the participants better understand the functionalities of the CTM mechanism, which enabled them to perceive changes in textual conversation, leading to better user satisfaction. Based on the empirical insights from our study, we discuss future research avenues for multi-agent chatbot design and its application for rich information elicitation.
... The study has been conducted using the methodological principles of Conversation Analysis (Drew 2004;Sacks 1992), which in terms of multimodal analysis describes how talk, gesture, gaze, body posture and the physical surroundings of the participants are jointly used in the performance of social action (Deppermann 2013;Deppermann/Streeck 2018;Goodwin 2017;Kärkkäinen/Thompson 2018;Keevallik 2018;Mondada 2014;Streeck et al. 2011). Data recordings were conducted with the use of mobile eye-tracking glasses (Tobii Pro Glasses 2) worn by the participants and an additional third camera to account for embodied conduct not visible through the eye-tracking. ...
This conversation-analytic paper examines the use of the German particle so in managing transitions between activities that form part of a joint project. Building on prior work on the transition relevant property of so by Barske/Golato (2010), the paper investigates the particle's use concurrently with other multimodal resources, such as the participants' bodily orientation and their gaze direction. I illustrate that, in addition to marking boundaries between activities and communicating those to co-participants, the examination of speaker and addressee's gaze behaviour reveals that the particle can have a retrospective as well as prospective orientation. What is more, the analysis shows that by alerting their co-interactants of a change-of-activity taking place and of their availability to initiate a new activity, participants manage to achieve coordination and cooperation within the larger project. All in all, the examination of this particle in transition spaces reveals that the organisation of a joint project is a collective and collaborative process that relies on participants' finely tuned coordination of their individual actions.
... In our study, we track "the process of emergence" : 21) of a given resource (in our case a gesture) in a community of practice drawing on both a multimodal extended EMCA-approach (Deppermann, 2013) and longitudinal CA . Deppermann & Pekarek Doehler (2021: 128-133) differentiates three kinds of approaches to longitudinal CA: developmental studies, studies of sociohistorical change and studies of joint interactional histories. ...
Theater rehearsals are (usually) confronted with the problem of having to transform a written text into an audio-visual, situated and temporal performance. Our contribution focuses on the emergence and stabilization of a gestural form as a solution for embodying a certain aesthetic concept which is derived from the script. This process involves instructions and negotiations, making the process of stabilization publicly and thus intersubjectively accessible. As scenes are repeatedly rehearsed, rehearsals are perspicuous settings for tracking interactional histories. Based on videotaped professional theatre interactions in Germany, we focus on consecutive instances of rehearsing the same scene and trace the interactional history of a particular gesture. This gesture is used by the director to instruct the actors to play a particular aspect of a scene adopting a certain aesthetic concept. Stabilization requires the emergence of shared knowledge. We will show the practices by which shared knowledge is established over time during the rehearsal process and, in turn, how the accumulation of knowledge contributes to a change in the interactional practices themselves. Specifically, we show how a gesture emerges in the process of developing and embodying an aesthetic concept, and how this gesture eventually becomes a sign that refers to and evokes accumulated knowledge. At the same time, we show how this accumulated knowledge changes the instructional activities in the rehearsal process. Our study contributes to the overall understanding of knowledge accumulation in interaction in general and in theater rehearsals in particular. At the same time, it is devoted to the central importance of gestures in theater, which are both a means and a product of theatrical staging.
... The standard model of turn-taking is thus primarily a vocalauditory one, working with units that have been defined in linguistic terms. However, face-to-face interaction is multimodal, involving diverse semiotic resources, including visible bodily actions [39][40][41][42][43][44]. Within the model, gaze and gesture have been shown to be major resources for turn-allocation (e.g. by addressing a turn with gaze or selecting a next speaker with a point; [10,11,45,46]), and some smaller scale studies have suggested that visual modalities may contribute to the recognition of turn completions [47][48][49][50][51]. ...
Human communicative interaction is characterized by rapid and precise turn-taking. This is achieved by an intricate system that has been elucidated in the field of conversation analysis, based largely on the study of the auditory signal. This model suggests that transitions occur at points of possible completion identified in terms of linguistic units. Despite this, considerable evidence exists that visible bodily actions including gaze and gestures also play a role. To reconcile disparate models and observations in the literature, we combine qualitative and quantitative methods to analyse turn-taking in a corpus of multimodal interaction using eye-trackers and multiple cameras. We show that transitions seem to be inhibited when a speaker averts their gaze at a point of possible turn completion, or when a speaker produces gestures which are beginning or unfinished at such points. We further show that while the direction of a speaker's gaze does not affect the speed of transitions, the production of manual gestures does: turns with gestures have faster transitions. Our findings suggest that the coordination of transitions involves not only linguistic resources but also visual gestural ones and that the transition-relevance places in turns are multimodal in nature.
This article is part of a discussion meeting issue ‘Face2face: advancing the science of social interaction’.
This paper is the first to address the impact of gendered, cultural and religious discourses on an under-researched subaltern group of infertile Muslim women bloggers. Taking a small story and case study approach, the analysis focuses on interactivity and positioning ( Bamberg & Georgakopoulou, 2008 , Georgakopoulou, 2008 ) in one woman’s stories as she works hard to address normative expectations and dominant discourses which abound in Muslim societies. The paper highlights the stigmatisation and isolation women face, not only in the physical world, but sometimes in the online world too. We argue that Weblogs provide a unique and unexplored space where discourses of gender, sexual, and other identities are resisted and challenged. Simultaneously Weblogs can serve as both supportive and exclusionary sites in which bloggers’ rights and duties become regulated. The study opens a window into the world of infertile Muslim women and has important implications for relevant healthcare and policy making.
The book explores the multifaceted nature of media and communication by challenging traditional views that consider media solely as technical infrastructures for transmitting information. Instead, it focuses on mediality as an empirically relevant concept and proposes to understand media as socially constituted semiotic procedures that shape and are shaped by communicative practices. The book is structured around this central idea, with four main sections. Part I examines digital environments, analyzing the interplay between multimodal approaches and mediality through case studies such as digital learning platforms and Zoom seminars. Part II focuses on journalistic procedures, investigating how media shapes political debates and news presentation on platforms like Instagram. Part III delves into embodied processes, particularly the role of the body movements and gestures in communication, illustrated through analyses of yoga tutorials and family dinner conversations. Part IV combines diverse semiotic and medial resources, with studies on historical data interpretation and virtual reality gaming practices. The book aims to provide a comprehensive understanding of the role of different media in constituting meaning and shaping social interactions.
This study investigates how teachers use language, body movements, and visual displays to seek students’ displays of linguistic knowledge in Chinese-as-a-second-language (CSL) classrooms. We focus on two types of multimodal practices used by CSL teachers. The first type of multimodal practice, [zhe(ge) shi ‘this is’ + image display], is used by teachers to elicit students to display lexico-semantic knowledge of lexical items representing concrete concepts. The second type of multimodal practice is in the form of [zhe(ge) shi ‘this is’ + character display]. It is utilized to elicit students to display their orthographic-phonological knowledge of lexical items representing abstract concepts. These two multimodal practices are employed in different sequential environments and pedagogical activities. Specifically, the first type of multimodal practice is used in the non-initial position of a vocabulary teaching activity, whereas the second type of multimodal practice is used at the beginning of a vocabulary teaching activity. This study shows how teachers use their language, body, and visual signs together to engage students in teaching vocabulary in CSL classrooms. The findings of the study have implications for abstract and concrete vocabulary teaching and multimodal classroom interactional research.
This collection of original papers illustrates recent trends and new perspectives for future research in Interactional Linguistics (IL). Since the research program was started around the turn of the century, it has prospered internationally. Recently, however, new developments have opened up new perspectives for interactional linguistic research. IL continues to study the details of talk in social interaction, with a focus on linguistic resources and structures of verbal and vocal interaction in bodily-visible interactional settings. Increasingly, though, it embraces methods supported by new technology and broadens its data and research questions to applications in teaching, therapy, etc. The volume comprises three parts with 14 contributions: (1) Studying linguistic resources in social interaction; (2) Studying linguistic resources in embodied social interaction; and (3) Studying social interaction in institutional contexts and involving speakers with specific proficiencies.
In this article, we ask how interlocutors proceed with their daily activities in the first months of the Covid-19 pandemic when faced with new ways of communication due to social distancing and the use of face masks. We carried out a fine-grained analysis of different micropractices from daily work in a healthcare center in Brazil and built our analysis on multimodal conversation analysis (MCA), interactional linguistics (IC), as well as gesture studies (GS). The analysis revealed that particularly the following recurrent patterns seem to be characteristic for communication during the pandemic in the given microcontexts: (a) a high use of deictic gestures, (b) an intensification of prosodic means, (c) verbal strategies such as reformulation and repetition, (d) the integration of object manipulation and (e) mitigation strategies in case of new formats that imply intrusion such as controls at travel checkpoints.
This study proposes a Multimodal Hermeneutic Model (for short, MHM) as a methodology that extends the analytic scope of Ricoeur’s (1973, 1976, 1981) classic hermeneutic theory of text interpretation towards examining multimodal texts. The model has been empirically tested by examining the advertising discourse order of ‘Saudization’ as channelled via the Saudi Uber Blog’s
multimodal text. A twofold social semiotic praxis has been theoretically incorporated into the distanciation-appropriation dialectics underlying the interpretation of multimodal texts in potentia. First, a multimodal cluster transcription (Baldry & Thibault 2006) has been utilized in enhancing the description of distanciated text sense as a holistic configuration of clusters across different communicative modalities with interacting semiotic modes (intra-textually). Second, an interpersonal-meaning analysis of multimodal participants (Kress & Van Leeuwen 2006a, 2006b, Halliday & Matthiessen 2004) has been employed in enriching the explanation of appropriated text reference (extra-textually). The empirical site used for validating the MHM is the Uber-Blogmediated multimodal text designed by Saudi Arabia-based Uber Company. The multimodal transcription of textual clusters has demonstrated how the text sense thematically revolves around the macro topic of ‘Saudization’ across verbal, visual, and pictorial modalities with material and semiotic modes, viz. linguistic, graphological, anthropic, sartorial, spatial, natural, and
technological. The multimodal participant analysis has explained how the referents of (i) a model Saudi Uber driver, (ii) the Saudi Public Transport Authority, (iii) Saudi driver-partners, and (iv)
Uber app collectively contribute to the recontextualization of ‘Saudization’ from a governmental discourse to an advertising discourse order realized in the multimodal text under analysis.
This innovative, timely text introduces the theory and research of critical approaches to language assessment, foregrounding ethical and socially contextualized concerns in language testing and language test validation in today’s globalized world.
The editors bring together diverse perspectives, qualitative and quantitative methodologies, and empirical work on this subject that speak to concerns about social justice and equity in language education, from languages and contexts around the world – offering an overview of key concepts and theoretical issues and field-advancing suggestions for research projects.
This book offers a fresh perspective on language testing that will be an invaluable resource for advanced students and researchers of applied linguistics, sociolinguistics, language policy, education, and related fields – as well as language program administrators.
DOI: https://doi.org/10.4324/9781003384922
Available: https://www.taylorfrancis.com/books/edit/10.4324/9781003384922/ethics-context-second-language-testing-rafael-salaberry-wei-li-hsu-albert-weideman
PREVIEW PDF available on this site.
Research indicates the importance of small talk in achieving professional goals and promoting social relationships in workplaces, and yet second-language (L2) users often face challenges in participating in small talk. As such, the need for teaching small talk has been emphasized; however, the understanding of how L2 interactional practices for small talk develop remains limited. This study aims to demonstrate the change in L2 English practices used for initiating and extending small talk that occurred during service encounters over a thirty-month period. The data were gathered from 74 h of audiovisual recordings of service encounters at a small convenience store in Hawai'i. The focal participant is the service provider, a Korean adult with limited L2 English proficiency, who had emigrated from South Korea a year before the start of the data collection. By employing multimodal conversation analysis (CA) and the Pearson's chi-square test, the results of the study demonstrated three aspects of change in the service provider's practices for small talk: increasing the frequency of initiations, emerging new methods for initiation, and developing expansion techniques for small-talk sequences. The study advances the understanding of the development of multimodal L2 interactional practices for small talk in service encounters.
Teams are ubiquitous in organisations; however, the in situ analysis of teamwork as it happens has rarely been the object of research. Despite claims that fostering team spirit is essential for success, trying to pin down such a notoriously ephemeral, slippery, and abstract notion has proved difficult. Using transcripts of naturally occurring meeting talk, this chapter showcases the use of conversation analysis as a method to make visible, and thus analysable, what can be said to be the process of achieving team spirit. Findings indicate that team spirit may be understood in terms of doing maximally pro-social actions, such as affiliation.KeywordsConversation analysisTeamsAffiliationLaughterTeasingTeam spirit
The widespread view that risk is highly relevant in late modern societies has also meant that the very study of risk has become central in many areas of social studies. The key aim of this book is to establish Risk Discourse as a field of research of its own in language studies. Risk Discourse is introduced as a field that not only targets elements of risk, safety and security, but crucially requires aspects of responsibility for in-depth analysis. Providing a rich illustration of ways in which risk and responsibility can serve as analytical tools, the volume brings together scholars from different disciplines within the study of language. An Introduction and an Epilogue highlight the intricate relationship between risk and responsibility. Part 1 deals with expert and lay perspectives on risk; Part 2 with emerging genres for risk discourse; Part 3 with risk and technology and Part 4 with ways of managing risk. The topics covered – such as COVID-19, nuclear energy, machine translation, terrorism – are socially pertinent and timely.
Tetsuta Komatsubara's chapter on "Framing risk metaphorically: Changes in metaphors of COVID-19 over time in Japanese", which is chapter 3 in the volume, won the Maenosono Young Researcher’s Award in 2024 as the best paper of each graduate school of Kobe University: https://www.kobe-u.ac.jp/ja/announcement/20240716-65819/
Stance-taking, the public act of positioning oneself toward objects, people or states of affairs, has been studied in many fields of research. Recently, its multimodal realization in interaction has received increasing attention. The current contribution aims to take stock of research on multimodal stance-taking so far, and to present possible avenues for future research. We systematically gathered and appraised 76 articles that investigate the involvement of bodily-visual resources in stance-taking in interaction. The critical appraisal focused on two dimensions of the stance act: form-function relations constituting it, and its dynamic organization in interaction. Regarding form-function relations, we found systematic involvement of specific bodily-visual resources in different stance acts, as well as patterns of multimodal intensification and mitigation of stances. As for its dynamic organization, the review discusses how stance-taking is organized temporally throughout an interaction, with all participants involved carefully negotiating and adapting their stances to one another. Finally, attention is paid to the broader context of stance-taking, including its role in different social and societal contexts. Based on this review, we were able to identify several gaps in the literature, and avenues for future research. We argue that much potential for broadening the scope of research lies in increasing the methodological diversity in approaching multimodal stance-taking, as well as in cross-linguistic studies and varying settings and participant constellations. In conclusion, research into multimodal stance-taking is vibrant, with ample opportunities for future work. This review can be considered as a call to action to move beyond the premise that stance-taking is multimodal, and further investigate this intriguing and fundamental human capacity.
Human interaction frequently includes decision-making processes during which interactants call on verbal and non-verbal resources to manage the flow of interaction. In 2017, Stevanovic et al. carried out pioneering work, analyzing the unfolding of moment-by-moment dynamics by investigating the behavioral matching during search and decision-making phases. By studying the similarities in the participant's body sway during a conversation task in Finnish, the authors showed higher behavioral matching during decision phases than during search phases. The purpose of this research was to investigate the whole-body sway and its coordination during joint search and decision-making phases as a replication of the study by Stevanovic et al. (2017) but based on a German population. Overall, 12 dyads participated in this study and were asked to decide on 8 adjectives, starting with a pre-defined letter, to describe a fictional character. During this joint-decision task (duration: 206.46 ± 116.08 s), body sway of both interactants was measured using a 3D motion capture system and center of mass (COM) accelerations were computed. Matching of body sway was calculated using a windowed cross correlation (WCC) of the COM accelerations. A total of 101 search and 101 decision phases were identified for the 12 dyads. Significant higher COM accelerations (5.4*10-3 vs. 3.7*10-3 mm/s2, p < 0.001) and WCC coefficients (0.47 vs. 0.45, p = 0.043) were found during decision-making phases than during search phases. The results suggest that body sway is one of the resources humans use to communicate the arrival at a joint decision. These findings contribute to a better understanding of interpersonal coordination from a human movement science perspective.
As a means of furthering their talk, co-present participants will on occasions orient to environmentally available text, such as that in a book or on a computer screen. This sort of action commonly relies on a combination of both embodied and spoken interactional practices to enable elements of the written language to become part of the ensuing talk. Such actions as pointing to part of a page or gazing at an illustration and then naming it can help establish a joint focus of attention, particularly in talk in which the textual object plays a role in future activities the participants are discussing. This study uses conversation analysis to suggest that textual objects therefore become an affordance for turn progressivity, since they contain language components that can serve as both potential prompts and turn-incorporable elements. The data are taken from Japanese/English bilingual interaction video-recorded between elementary and junior high educators who are preparing to team-teach English classes in Japan. We examine this phenomenon in two distinct sequential contexts: (1) devising a plan and (2) sharing a plan. The study provides insight into the ways inscribed objects can be used to facilitate interaction within the professional practice of team-teacher planning.
This paper brings together two different traditions in the study of reference and, more specifically, of some deictical terms: it combines the French tradition of the ›linguistique de l’énonciation‹ with a conversation analytic perspective insisting on the emergent character of the referential processes within the sequential unfolding of talk-in-interaction. These two approaches allow us to consider deixis as an interactional accomplishment by which participants elaborate not only their reference to the world but also their positioning within the socio-contextual space as well as within the discursive space. These ideas are applied to a corpus of sociological interviews, in which deictical reference and positioning are analysed as an interactional achievement of both the interviewee and the interviewer.
It is often assumed that the design of artefacts and environments shapes the ways in which people act and experience them. Designers and managers consider the material environment as defining people's behavioural responses. Museums and galleries provide us with an example par excellence in this regard. They display artefacts that have been carefully created to encourage particular kinds of behaviour and experience. They provide social scientists with a 'natural laboratory' in which we can study the relationship between behaviour and the physical environment. This paper draws on video fragments recorded at one particular exhibit displayed in the Science Museum London. It discusses how visitors produce their experience of the exhibit in and through their action and interaction. We explore individuals' action at the exhibit as well as the ways in which participants examine the artefact in concert with others, both companions and others who happen to be there. The paper concludes with a brief discussion about the implications of the observations for our understanding of the relationship between design and behaviour.
This article presents a revised version of GAT, a transcription system first devel-oped by a group of German conversation analysts and interactional linguists in 1998. GAT tries to follow as many principles and conventions as possible of the Jefferson-style transcription used in Conversation Analysis, yet proposes some conventions which are more compatible with linguistic and phonetic analyses of spoken language, especially for the representation of prosody in talk-in-interac-tion. After ten years of use by researchers in conversation and discourse analysis, the original GAT has been revised, against the background of past experience and in light of new necessities for the transcription of corpora arising from technologi-cal advances and methodological developments over recent years. The present text makes GAT accessible for the English-speaking community. It presents the GAT 2 transcription system with all its conventions and gives detailed instruc-tions on how to transcribe spoken interaction at three levels of delicacy: minimal, basic and fine. In addition, it briefly introduces some tools that may be helpful for the user: the German online tutorial GAT-TO and the transcription editing software FOLKER.
That human social interaction involves the intertwined cooperation of different modalities is uncontroversial. Researchers in several allied fields have, however, only recently begun to document the precise ways in which talk, gesture, gaze, and aspects of the material surround are brought together to form coherent courses of action. The papers in this volume are attempts to develop this line of inquiry. Although the authors draw on a range of analytic, theoretical, and methodological traditions (conversation analysis, ethnography, distributed cognition, and workplace studies), all are concerned to explore and illuminate the inherently multimodal character of social interaction. Recent studies, including those collected in this volume, suggest that different modalities work together not only to elaborate the semantic content of talk but also to constitute coherent courses of action. In this introduction we present evidence for this position. We begin by reviewing some select literature focusing primarily on communicative functions and interactive organizations of specific modalities before turning to consider the integration of distinct modalities in interaction.
In this commentary, I consider variability as an ordinary and irremediable feature related to the indexicality not only of transcripts but first of all of transcribing. In this sense, it is not just a characteristic of transcripts as texts, which can be assessed in a kind of philological comparison comparing formal features of autonomous and fixed textual objects, but a characteristic of transcribing as a situated practice. Practices are irremediably indexical, reflexively tied to the context of their production and to the practical purposes of their accomplishment. Thus, a transcript is an evolving flexible object; it changes as the transcriber engages in listening and looking again at the tape, endlessly checking, revising, reformatting it. Transcribing relies in a fundamental way not only on the possibility of fixing the relevant details in a complex multilayered representation but also on the possibility of manipulating them, playing them again and again, at different paces, positions, fragments, while transcribing their finely tuned coordination, their synchronization, the fine articulation between different projections and sequential implicativenesses. These manipulations are one of the ways in which transcribing is accomplished as a situated practice.
Over the last 50 years the process of producing transcripts of all kinds of interactions has become an important practice for researchers in a wide range of disciplines. Only rarely, however, has transcription been analyzed as a cultural practice. It is here argued that it is precisely the lack of understanding of what is involved in transcribing that has produced a number of epistemological problems, including the tendency to become either virtual-realists or hypercontextualists. By proposing a new interpretation of Plato's famous story of the prisoners in the cave who could only see the shadows of what was happening outside, this article examines the advantages of the selective nature of transcription, unveils some of the cognitive and affective implications of engaging in transcription, and proposes a complementary approach to transcription, in which transcripts are evaluated with respect to what they can (or cannot) reveal within a particular domain of inquiry.
Organizational features of ordinary conversation and other talk-in-interaction provide for the routine display of participants' understandings of one anothers' conduct and of the field of action, thereby building in a routine grounding for intersubjectivity. This same organization provides interactants the resources for recognizing breakdowns of intersubjectivity and for repairing them. This article sets the concern with intersubjectivity in theoretical context, sketches the organization by which it is grounded and defended in ordinary interaction, describes the practices by which trouble in understanding is dealt with, and illustrates what happens when this organization fails to function. Some consequences for contemporary theory and inquiry are suggested.
Despite technical advances over the past few years in the area of systems support for cooperative work there is still relatively little understanding of the organisation of collaborative activity in real world, technologically supported, work environments. Indeed, it has been suggested that the failure of various technological applications may derive from their relative insensitivity to ordinary work practice and situated conduct. In this paper we discuss the possibility of utilising recent developments within sociology, in particular the naturalistic analysis of organisational conduct and social interaction, as a basis for the design and development of tools and technologies to support collaborative work. Focussing on the Line Control Rooms in London Underground, a complex multimedia environment in transition, we begin to explicate the tacit work practices and procedures whereby personnel systematically communicate information to each other and coordinate a disparate collection of tasks and activities. The design implications of these empirical observations, both for Line Control Room and technologies to support cooperative work, are briefly discussed.
The organization of embodied participation frameworks, stance and affect is investigated using as data a sequence in which a father is helping his daughter do homework. Through the way in which they position their bodies toward both each other and the homework sheet that is the focus of their work the two contest the interactive and cognitive organization of the activity they are pursuing together. The father insisted that their work be organized in a way that would allow him to demonstrate the practices required to solve her problems. However the daughter refused to rearrange her body to organize the participation framework that would make this possible, and demanded instead that Father tell her the answers. When the daughter consistently refused to cooperate Father eventually walked out, but returned later, and they constructed a very different affective and cognitive alignment. Such phenomena shed light on range of different kinds of epistemic, moral and affective stances that are central to both the organization of cognition and action, and to how participants constitute themselves as particular kinds of social and moral actors in the midst of the mundane activities that constitute daily family life.
This paper describes a possible formal organizational device that serves to bound episodes of body movement such as gestures, fidgets, instrumental moves and the like. It involves a spate of movement — whether a single move or a series of moves — being completed by returning the moving body part to the position from which it departed at the outset. A series of specimens are examined which display this organizational device across a number of dimensions of variation — in the body part being moved, the characteristics of the mover, the amplitude of the move, etc., underscoring the formality and adaptability of the device.
The electronic edition of this article includes audio-visual data.
This paper discusses hand-gestures that serve as incipient and premonitory components of communicative actions and components of such actions. Gestures are examined with respect to the position in emerging turns and sequences of talk where they are made: before and at turn-beginning, in mid-turn, and during moments of turn-completion; and the projections that they make at these positions are delineated. Hand-gestures and their positioning relative to unfolding units of talk exemplify how the multimodality of the human body serves in the coordination of social action and the achievement of intersubjectivity.
Seeing is investigated as a socially situated, historically constituted body of practices through which the objects of knowledge that animate the discourse of a profession are constructed and shaped. Analysis of videotapes of archaeologists making maps and lawyers animating events visible on the Rodney King videotape focuses on practices that are articulated in a work-relevant way within sequences of human interaction, including coding schemes, highlighting, and graphic representations. Through the structure of talk in interaction, members of a profession hold each other accountable for, and contest the proper constitution and perception of, the objects that define their professional competence.
Workplace studies are of growing significance to people in a broad range of academic disciplines and professions, in particular those involved in the development of new technologies. This ground breaking book, first published in 2000, brings together key researchers in Europe and the US to discuss critical issues in the study of the workplace and to outline developments in the field. The collection is divided into two parts. Part I contains a number of detailed case studies that not only provide an insight into the issues central to workplace studies but also some of the problems involved in carrying out such research. Part II focuses on the interrelationship between workplace studies and the design of new technologies. This book provides a valuable, multidisciplinary synthesis of the key issues and theoretical developments in workplace studies and a guide to the implications of such research for new technology design and the workplace.
The organization of taking turns to talk is fundamental to conversation, as well as to other speech-exchange systems. A model for the turn-taking organization for conversation is proposed, and is examined for its compatibility with a list of grossly observable facts about conversation. The results of the examination suggest that, at least, a model for turn-taking in conversation will be characterized as locally managed, party-administered, interactionally controlled, and sensitive to recipient design. Several general consequences of the model are explicated, and contrasts are sketched with turn-taking organizations for other speech-exchange systems.
this paper. 25 However, one feature of it relevant to the construction of the utterance being examined in this paper will be briefly noted. A speaker can request the gaze of a recipient by producing a phrasal break, such as a restart or a pause, in his utterance. After such a phrasal break nongazing recipients regularly bring their gaze to the speaker
Furthermore, certain transcription conventions invite modification by others with expertise in the field. It was the purppose of this chapter to: I. Identify what constitute data for the developmental psycholinguist. 2. Expose theoretical and cultural underpinnings of the transcription process. 3. Provide a setof basic transcription conventions sensitive to psychological, linguistic, and cultural dimensions of young children's behavior. 4. Indicate the relevance and usefulness of these conventions to current theoretical concerns. A greater awareness of transcription form can move the field in productive directions. Not only will we able to read rrmch more off our own transcripts, we will be better equipped to read the transcriptions of others. This, in turn, should better equip us to .evaluate particular interpretations of data (i.e., transcribed behavior). Our data may have a future if we give them the attention they deserve.
The abstract for this document is available on CSA Illumina.To view the Abstract, click the Abstract button above the document title.
reprinted in Dijk, T. van (ed.). Discourse Studies, London : Sage, vol. IV, 126-157.
Car conversations constitute a perspicuous setting, characterized by multiactivity (i.e., by an engagement in multiple simultaneous activities, as talking and driving). Based on a corpus of videorecordings of various naturally occurring car journeys, the paper focuses on the way in which participants coordinate their multiactivity in either convergent or divergent ways. It shows how they mobilize various embodied multimodal resources, such as talk, gesture, gaze, head movements, and body postures in order to display their current engagement in one or more activities, in a way highly sensitive to the sequential organization of talk.
Studies of talk-and-bodily-conduct-in-interaction have inspired new insights into the way in which language, interaction and cognition might be articulated. More particularly, they have shown that participants mutually orient to the finely tuned multimodal details by which talk and action in interaction are sequentially organized. This article deals with this form of ‘participants’ multimodal online analysis’ by focusing on a particular phenomenon - the methodical practices and resources by which the end of a turn and of an activity phase is projected and collectively achieved - in a specific videorecorded setting - a meeting in an architect’s office. It aims at questioning both how these local orientations are systematically displayed and exploited by the participants for the sequential organization of their activity and how they can be demonstrably observed by the analyst.
The abstract for this document is available on CSA Illumina.To view the Abstract, click the Abstract button above the document title.
This paper deals with the multimodal and spatial arrangements of the participants within pre-beginning and opening sequences, i.e. sequences taking place before the actual opening of a social interaction and achieving the conditions for an imminent opening. In face-to-face conversations, these sequences are characterized by intense body activities in space, through which participants achieve their social and spatial convergence and conjunction, and initiate a coordinated common entry in the interaction. In this phase, even before beginning to speak, participants achieve the mutual orientation of their bodies and of their gaze. Pre-conditions for social interaction are visibly and publicly assembled in time, within the progressive establishment of a mutual focus of attention and a common interactional space. In public places and between unknown persons, this mutual arrangement is even more important, emerging progressively from the participants’ transition from moving to standing, and their transformation from unfocused pedestrians to focused would-be-imminent-co-participants. On the basis of a corpus of video recordings, the paper offers an analysis of a collection of pre-beginnings of itinerary descriptions in public space and systematically describes the identification of the emerging interactional partner, the organization of convergent trajectories in space, the exchange of first mutual glances, and the very first words produced in the encounter.
Gesture is one of the least charted terrains in human communication. This article presents one in a series of attempts to illuminate the forms, uses, meanings, and functions of hand gestures by using methods of microanalysis and naturalistic description. Reporting on findings from research on videotaped natural interaction in eight speech communities, the author describes speakers’ methods for making hand gestures relevant to the moment of symbolic communication. Gestures are “exposed” by means of indexical uses of gaze and language. This coordination becomes visible under “microscopic” viewing of videotapes. The analysis is embedded in a brief review of previous views of gesture and a discussion of the role of gesture for self‐awareness.
Preface The transcription system 1. Video analysis: interactional coordination in movement and speech 2. The display of recipiency and the beginning of the consultation 3. Maintaining involvement in the consultation 4. Forms of participation 5. The physical examination 6. Taking leave of the doctor 7. Postscript: the use of medical records and computers during the consultation Notes References Index.
Our perception of our everyday interactions is shaped by more than what is said. From coffee with friends to interviews, meetings with colleagues and conversations with strangers, we draw on both verbal and non-verbal behaviour to judge and consider our experiences.
Analyzing Multimodal Interaction is a practical guide to understanding and investigating the multiple modes of communication, and provides an essential guide for those undertaking field work in a range of disciplines, including linguistics, sociology, education, anthropology and psychology. The book offers a clear methodology to help the reader carry out their own integrative analysis, equipping them with the tools they need to analyze a situation from different points of view. Drawing on research into conversational analysis and non-verbal behaviour such as body movement and gaze, it also considers the role of the material world in our interactions, exploring how we use space and objects - such as our furniture and clothes - to express ourselves. Considering a range of real examples, such as traffic police officers at work, doctor-patient meetings, teachers and students, and friends reading magazines together, the book offers lively demonstrations of multimodal discourse at work.
Illustrated throughout and featuring a mini-glossary in each chapter, further reading, and advice on practical issues such as making transcriptions and video and audio recordings, this practical guide is an essential resource for anyone interested in the multiple modes of human interaction.
IntroductionParticipation as ActionStories as Participation FieldsParticipation in Linguistic AnthropologyConclusion
Note: Transcription symbols follow those developed by Gail Jefferson, and are explained in the Appendix to this special issue. The coding of gaze, head nods, and similar phenomena is explained in the text.
A theory of action must come to terms with both the details of language use and the way in which the social, cultural, material and sequential structure of the environment where action occurs figure into its organization. In this paper it will be suggested that a primordial site for the analysis of human language, cognition, and action consists of a situation in which multiple participants are attempting to carry out courses of action in concert with each other through talk while attending to both the larger activities that their current actions are ambedded within, and relevant phenomena in their surround. Using as data video recordings of young girls playing hopscotch and archaeologists classifying color, it will be argued that human action is built throught the simultaneous deployment of a range of quite different kinds of semiotic resources. Talk itself contains multiple sign systems with alternative properties. Strips of talk gain their power as social action via their placement within larger sequential structures, encompassing activities, and participation frameworks constituted through displays of mutual orientation made by the actors' bodies. The body is used in a quite different way to perform gesture, again a class of phenomena that encompasses structurally different types of sign systems. Both talk and gesture can index, construe or treat as irrelevant, entities in the participants' surround. Moreover, material structure in the surround, such as graphic fields of various types, can provide semiotic structure without which the constitution of particular kinds of action being invoked through talk would be impossible. In brief it will be argued that the construction of action through talk within situated interaction is accomplished through the temporally unfolding juxtaposition of quite different kinds of semiotic resources, and that moreover through this process the human body is made publicly visible as the site for a range of structurally different kinds of displays implicated in the constitution of the actions of the moment.
The concept of awareness has become ofincreasing importance to both social andtechnical research in CSCW. The concept remainshowever relatively unexplored, and we stillhave little understanding of the ways in whichpeople produce and sustain `awareness' in andthrough social interaction with others. In thispaper, we focus on a particular aspect ofawareness, the ways in which participantsdesign activities to have others unobtrusivelynotice and discover, actions and events, whichmight otherwise pass unnoticed. We consider forexample how participants render visibleselective aspects of their activities, how theyencourage others to notice features of thelocal milieu, and how they encourage others tobecome sensitive to particular events. We drawexamples from different workplaces, primarilycentres of coordination; organisationalenvironments which rest upon the participants'abilities to delicately interweave a complexarray of highly contingent, yet interdependentactivities.
Pointing as situated practice
- Charles Goodwin
Goodwin, Charles, 2003. Pointing as situated practice. In: Kita, Sotaro (Ed.), Pointing: Where Language, Culture and Cognition Meet. Lawrence
Erlbaum, Mahwah, NJ, pp. 217--241.
The Routledge Handbook of Multimodal Analysis
- London Routledge
- Kress
- Gunther
- Van Leeuwen
Jewitt, Carey (Ed.), 2009. The Routledge Handbook of Multimodal Analysis. Routledge, London.
Kress, Gunther, van Leeuwen, Theo, 2001. Multimodal Discourse. Hodder Arnold, London.