Figure 1 - uploaded by Tanya Stivers
Content may be subject to copyright.
Source publication
That human social interaction involves the intertwined cooperation of different modalities is uncontroversial. Researchers in several allied fields have, however, only recently begun to document the precise ways in which talk, gesture, gaze, and aspects of the material surround are brought together to form coherent courses of action. The papers in t...
Similar publications
The next generation of ambient assisted living services will be based on eco-systems or organizations of intelligent artificial agents embodied in companion robots and smart objects. To provide, anywhere and anytime, smart assistance services to people, these agents need to be endowed with advanced knowledge representation, reasoning and communicat...
The mechanisms by which aging and other processes can affect the structure and function of brain networks are important to understanding normal age-related cognitive decline. Advancing age is known to be associated with various disease processes, including clinically asymptomatic vascular and inflammation processes that contribute to white matter s...
The role of the internet in large-scale demonstrations, as witnessed in the Arab Spring, has been debated and reflects continued interest in the intermingling of social movements and digital technology. Yet behind these large photogenic events stand other less obvious social activities that may be equally profound, particularly in the form of alter...
In multilingual classrooms, subtitling can be used to address the language needs of students from different linguistic backgrounds. The way students distribute their visual and cognitive resources during a lecture is important in educational design. Students have to shift their attention between sources of information of varying density and relevan...
In the literature on the situated and distributed nature of cognition, the coordination of spatial organization and the structure of human practices and relations is accepted as a fact. To date, science educators have yet to build on such research. Drawing on an ethnographic study of high school students during an internship in a scientific researc...
Citations
... table below offers additional simple and clear examples from the data, accompanied by a brief description of each representational gesture.Table 1: Examples of the representational gestures and DIUs in my data.The analyses in this study use Conversation Analysis, Interactional Linguistics, and Multimodal Analysis(Stivers & Sidnell 2005;Couper-Kuhlen & Selting 2018). I study the design of the teachers' DIUs by analyzing their morphosyntactic and bodily-visual conducts, as well as teachers' and students' orientations. ...
Les onze articles qui composent ce numéro traitent de ces deux aspects du
langage – grammaire et pratiques multimodales – en tant que ressources
essentielles au fonctionnement de l'interaction. Les différentes contributions se
basent sur des données interactionnelles en L1 et L2 au sens large, notamment
en classe de langue, mais également sur des corpus de messages vocaux ou
des données élicitées lors de tâches de narration.
... Interested readers are well served by empirical studies, journal special issues and edited collections. Editorial introductions to journal issues on multimodal analysis tend to include accounts of the development of multimodal studies and trenchant summaries of significant articles (Deppermann, 2013;Hazel et al., 2014;Stivers and Sidnell, 2005), and an "epilogue" (Mondada 2014), are particularly recommended. Various edited volumes include collections of important contributions to multimodal interaction (e.g. ...
This chapter outlines affordances for the study of interaction presented by ’“multimodal conversation analysis’”. Multimodal analysis challenges ‘“classical’” Sacksian CA (Depperman 2013), becoming more sensitive to settings as recording technologies, and orthographies for transcribing actions captured by these, evolve. Refining the granularity of data arguably opens up more phenomena for inquiry than have been explored hitherto. The chapter reviews some of its central texts, suggesting key resources for its study and high-profile publication outlets in the field. Ethnomethodologically indifferent readings of multimodal CA studies produced a mix of both enthusiasm and caution for the authors. Although the range of phenomena within its purview is wide and the execution of analyses is technically impressive, the authors became less convinced by either its EM or CA credentials, developing a more critical approach accordingly. Based upon their engagement with textual presentations cited in this chapter, the authors ask a series of questions intended to stimulate debate regarding the ethnomethodological or constructive analytic status of multimodal analysis.
... Typically, research on children's iconic gestures production has focused on quantifying when gestures emerge and the representational strategies used (e.g., Cartmill, et al., 2017;Crais, et al., 2012;Ozçalişkan & Goldin Meadow, 2011) with relatively little attention given to the interactional contexts in which they are produced. However, research on adult gesture production has used approaches such as conversation analysis to explore the multimodal nature of social interactions (e.g., see Stivers & Sidnell, 2005 for an overview). Stivers and Sidnell (2005) argue that one needs to analyse the situated context of multimodal communication to understand the communicative significance of gesture. ...
... However, research on adult gesture production has used approaches such as conversation analysis to explore the multimodal nature of social interactions (e.g., see Stivers & Sidnell, 2005 for an overview). Stivers and Sidnell (2005) argue that one needs to analyse the situated context of multimodal communication to understand the communicative significance of gesture. ...
Adults across cultures produce iconic gestures, but little is known about the emergence of iconic gesturing in infants. This study aimed to identify plausible instances of infants’ earliest iconic gestures to learn more about their form and the interactional contexts in which they emerge. We identified the first 10 iconic gestures produced by five English-speaking children in a naturalistic longitudinal video corpus. In identifying gestures, we cast a wide net that included gestures produced during pretend play and those performed with objects in-hand. Analyses of gestures’ form and context show that children produced their first iconic gesture between 12 and 20 months, the great majority of which depicted actions. We found mixed evidence suggesting that children may produce conceptually less-challenging gestures earlier. Infants produced more object-in-hand gestures than empty-hand gestures to depict transitive actions but also more imagined-object than body-part-as-object gestures. Most gestures were produced independently of adult models, and many demonstrated innovation. Overall, within natural interactions, infants demonstrated impressive representational abilities and did not always rely on parental scaffolding. Our findings highlight the importance of considering the interactional context when conducting research on the development of gesture.
... In other words, interactive communication is essentially a multimodal display (Couper-Kuhlen & Selting, 2001:1). Informed of this idea, multimodal interaction primarily refers to the comprehensive investigation of various modal resources that constitute interaction, so as to understand the nature and panorama of human interaction (Stivers & Sidnell, 2005;Li, 2019: 45). ...
... In a conversation, each mode is a separate symbol system with equal status, which are able to express meaning in an appropriate context and accomplish communicative purposes. The realization of turn beginning includes three types of resources: lexis and syntax, prosody and body movement (Stivers & Sidnell, 2005). Lexis and syntax are the most fundamental resources for interpersonal interaction. ...
Turn beginning is a crucial location for establishing speakership and gaining recipiency. It is one of the most challenging tasks for an individual learner. This study used a corpus-based approach to investigate how Chinese EFL learners employ multimodal resources to successfully execute turn beginnings, with a specific focus on potential variances between intermediate and advanced proficiency levels. The findings revealed a hierarchy in the allocation of linguistic resources, with feedback being the most prevalent, followed by demonstrative, connective, and syntactic relevance. This suggested that learners prioritized feedback on the previous turn at the outset of a turn, often serving an interactional function to convey acknowledgement and recognition. Interestingly, intermediate learner group used more demonstrative to provide a cue for utterance-interpretation. However, the advanced learner group preferred to establish logical relevance between each turn through the use of connective. In terms of non-linguistic resources, learners predominantly employed gaze and head movement to project turn beginning intention, demonstrating their capacity to harness multimodal resources to a certain extent. Moreover, the advanced learner group’s increased use of hand gesture suggested that as language proficiency improved, learners were more inclined to engage the hearer’s attention at the commencement of turns. This study contributes to the scholarly understanding of multimodal turn-taking by elucidating the distinct characteristics that learners exhibit when initiating conversational turns. It offers insights into how learners navigate the complex interplay of linguistic and non-linguistic resources in the pursuit of effective communication.
... Within the ethnographically informed discourse analysis, I was also guided by the concept of multimodality, acknowledging the intricate ways in which verbal communication, gestures, and other elements synchronize to perform actions (Stivers and Sidnell, 2005). This perspective allows for eschewing prioritizing language use over practices with other semiotic resources. ...
Translanguaging theory highlights the dynamic use of multiple languages and communication modes by multilingual people in their daily experiences. Museums are informal family learning spaces where multilingual families use languages and other semiotic resources to create learning opportunities for their children. Using a microethnographic approach to discourse analysis and multimodal interaction analysis, I examined how a multilingual family uses translanguaging practices to organize their family learning in museums and the role of pointing gestures as part of their translanguaging repertoires in multilingual family learning. The analysis of two literacy events highlights that a child and his mother translanguaged with various semiotic resources to organize museum performances, joint attention, and telling, and that pointing gestures played a role in constructing a translanguaging space as they organized the two performances. Pointing involved the family in reading signage texts and allowed the mother to translate them for the child. Viewing translation as part of the translanguaging repertoire, this study recognizes the importance of the role of pointing gestures in constructing family learning at museums, enriching children’s schooling and literacy learning in classrooms. I argue that recognizing pointing as a critical component of translanguaging allows educators to develop strategies that leverage families’ unique repertoires to support multilingual students’ language and literacy learning.
... Multimodal analysis (MA) is a perspective towards the study of social interaction that can be combined with IL and CA methodologies. MA views social interaction as inherently multimodal, involving not only grammar and sound patterns, but also bodily-visual behaviors such as manual gestures, facial expressions, eye gaze, objects in the surroundings (Deppermann, 2013;Stivers & Sidnell, 2005;Streeck et al., 2011). People use multimodal resources in situated physical environments when they conduct social interaction (Goodwin, 2000). ...
This study investigates how teachers use language, body movements, and visual displays to seek students’ displays of linguistic knowledge in Chinese-as-a-second-language (CSL) classrooms. We focus on two types of multimodal practices used by CSL teachers. The first type of multimodal practice, [zhe(ge) shi ‘this is’ + image display], is used by teachers to elicit students to display lexico-semantic knowledge of lexical items representing concrete concepts. The second type of multimodal practice is in the form of [zhe(ge) shi ‘this is’ + character display]. It is utilized to elicit students to display their orthographic-phonological knowledge of lexical items representing abstract concepts. These two multimodal practices are employed in different sequential environments and pedagogical activities. Specifically, the first type of multimodal practice is used in the non-initial position of a vocabulary teaching activity, whereas the second type of multimodal practice is used at the beginning of a vocabulary teaching activity. This study shows how teachers use their language, body, and visual signs together to engage students in teaching vocabulary in CSL classrooms. The findings of the study have implications for abstract and concrete vocabulary teaching and multimodal classroom interactional research.
... Scholars advocate for multimodal task design to enhance language acquisition (Hampel & Hauck, 2006;Stein, 2004), yet research in this area remains underdeveloped (Grapin, 2019;Roth et al., 2023). Empirical studies have shown that integrating language with various semiotic forms like images and interactions results in the coordinated use of non-verbal resources alongside verbal and paraverbal elements (Kress, 2000;Stivers & Sidnell, 2005). Furthermore, Yaman Ntelioglou et al. (2014) highlights the positive effects of multimodal task design on improving language and content knowledge acquisition. ...
Content and Language Integrated Learning (CLIL) is increasingly adopted globally, including in Taiwan’s educational initiatives, yet challenges remain in implementing effective CLIL practices, such as pedagogy and curriculum design. This study investigated the effectiveness of multimodal task designs, combining hands-on learning with poster presentations, in enhancing oral communicative competence within CLIL contexts. Employing a mixed-methods, quasi-experimental design with a comparative case study framework, the study assessed English oral communicative competence in four intact fourth-grade Taiwanese CLIL Social Studies classes. The hands-on learning group (EG, n = 40) engaged in activities like Chinese Dumpling Making, Bird’s Nest Building, and Succulent Pot Designing, while the non-hands-on learning group (CG, n = 34) used traditional worksheets on the same topics. Both groups proceeded to poster presentations within their multimodal task design, where students’ oral communicative competence was assessed using rubrics developed based on Coyle’s 4Cs dimensions, focusing on Content, Communication, and Cognition. Additionally, students’ cultural knowledge related to the hands-on topics was evaluated through written tests. To complement the quantitative data, qualitative data from self-reported reflections and video recordings documenting interventions were collected for the assessment of oral communicative competence within a CLIL framework. Results demonstrate that integrating hands-on activities significantly enhanced procedural content, communication (i.e., sentence complexity, pronunciation accuracy for target vocabulary, presentation fluency), and cognitive abilities, confirming the efficacy of multimodal learning approaches in fostering linguistic and cognitive engagement. Post-test comparisons show the EG’s superiority in cultural knowledge acquisition across all three hands-on topics. Student reflections endorsed the enrichment of learning experiences through multimodal task design. Video analysis of both groups’ interventions revealed that despite significant engagement and autonomy, EG students commonly utilized general English rather than target vocabulary, a pattern similar to that observed in the CG. These findings highlight the potential of diverse modalities in CLIL to enhance English content learning and oral skills, shaping future pedagogy and language strategies in Taiwan. The study also emphasizes the role of embodied learning, the interplay between physical actions and cognitive processes, to facilitate deeper understanding and engagement with subject matter within CLIL settings.
... I draw on field observations [25] and multimodal conversation analysis [26], [27], [28], [29,] rooted in ethnomethodology / conversation analysis [30], [31], [32], [33]. This emic data-driven approach is 'situated, ongoing, patterned, contextualized and multimodal' [34, p. 125]. ...
... For this study, the principles of multimodal interaction analysis were used (Norris, 2004). Multimodal interaction analysis is a methodological approach to video investigation rooted in conversation analysis (Sacks et al., 1978), which focuses on multiple channels of communication simultaneously (Nassauer and Legewie, 2021;Stivers and Sidnell, 2005). Thus, the analysis does not consider one single aspect of the video only, such as speech, but takes into account as many features of the video as possible (including the photographs used and the creation context). ...
This study analyzes the self-representations of Russian-speaking women with anorexia on YouTube. Using multimodal interaction analysis, the research focuses on how the explanatory model of anorexia and the representation platform influence the narratives produced by vloggers who have experienced or are experiencing anorexia. It is concluded that anorexia is mainly represented in these videos as a weight-related mental health disorder. Although the explanatory model of anorexia shapes self-representations on YouTube, the platform plays an even greater role in the construction of such narratives. This study’s findings fill gaps in the existing literature by revealing the joint structural influences that shape storytelling regarding the anorexia experience. The theoretical perspective utilized in this article could be further applied in research examining media representations of other mental health conditions.
... CA and IL are employed to analyse the sequential positions of DIUs and the linguistic resources used by teachers related to DIUs. Finally, we adopt the multimodal perspective to interaction (Stivers and Sidnell 2005;Streeck, Goodwin, and LeBaron 2011) by investigating not only linguistic structures of DIUs, but also bodily-visual behaviours and the physical environment involved in the production of DIUs, as well as how they work together in accomplishing interactional pedagogical tasks in classroom interaction. ...
