Multimodal Theory and Methodology: For the Analysis of (Inter)action and Identity
Abstract
This concise guide outlines core theoretical and methodological developments of the growing field of Multimodal (Inter)action Analysis.
The volume unpacks the foundational relationship between multimodality and language and the key concepts which underpin the analysis of multimodal action and interaction and the study of multimodal identity. A focused overview of each concept charts its historical development, reviews the essential literature, and outlines its underlying theoretical frameworks and how it links to analytical tools. Norris illustrates the concept in practice via the inclusion of examples and an image-based transcript, table, or graph.
The book provides a succinct overview of the latest research developments in the field of Multimodal (Inter)action Analysis for early career scholars in the field as well as established researchers looking to stay up-to-date on core developments.
... This chapter examines miscommunication and the interactive nonalignment of an intercultural team working on a task via video conferencing technology using English. We utilize multimodal (inter)action analysis (Norris 2004(Norris , 2011(Norris , 2019(Norris , 2020 as our theoretical and methodological framework to shed new light on how participants appear to co-construct common ground, while they in fact do not achieve conceptual convergence. ...
... We utilize multimodal (inter)action analysis (Norris 2004(Norris , 2011(Norris , 2019(Norris , 2020 as our theoretical and methodological framework to shed new light on how participants appear to co-construct common ground, while they in fact do not achieve conceptual convergence because they each produce their own higher-level actions, drawing on different social practices as they go about the task in different ways. Multimodal (inter)action analysis is a theoretical and methodological framework that allows the analysis of human actions and interactions in their vast complexity. ...
... Thus, in multimodal (inter)action analysis, we in fact do not use modes as analytical units, but the actions that social actors perform. For a detailed account of the theoretical and methodological background, see Norris (2019) and for a detailed explanation of the analytical tools and how they relate, please refer to Norris (2020). The framework indeed is quite vast with multiple explanatory layers that allow for micro analyses to be connected to meso and macro levels. ...
This chapter examines miscommunication and the interactive nonalignment of an intercultural team working on a task via videoconferencing technology using English. We utilize Multimodal (Inter)action Analysis (Norris, 2004, 2011, 2019, 2020) as our theoretical and methodological framework to shed new light on how participants appear to co-construct common ground, while they in fact do not achieve conceptual convergence.
... MIA (BernadMechó, this volume; Norris, 2004Norris, , 2020) is a framework for the analysis of social interaction, wherein the mediated action (Vygotsky, 1934(Vygotsky, /1987Wertsch, 1991) is the central unit of analysis. In MIA, the mediated action is analyzed in terms of lower and higherlevel actions. ...
... High modal density can be achieved with high modal complexity, inten sity, or both. The more complex or intense the modes of communication are (i.e., the higher the modal density), the more attention a social ac tor pays to the action(s) being produced (Norris, 2020). Essentially, high modal density becomes a locus of attention in interaction. ...
... We de fine modal intensity as the phenomenon of heightened pragmatic salience occurring as a result of an action being made prominent relative to sur rounding or cooccurring actions that have become recognized as regular, expected, or standard due to frequency of occurrence within a particular context. Through MIA (Norris, 2004(Norris, , 2020 of video recordings of an instructor teaching university mathematics in a lecture hall and a tutor in a onetoone tutoring session of high school English, we sought answers to the first research question: what teaching strategies do instructors use when attempting to focus students' attention and involve them in subject matter in facetoface teaching contexts? And, what role does modal den sity play in these teaching strategies? ...
... Our work is innovative and makes contributions to the fi eld in several ways. Firstly, our choice of qualitative analysis method to study online multimodal interactions, Multimodal (Inter)action Analysis (MIA) ( Norris, 2004( Norris, , 2019( Norris, , 2020, is novel particularly in Applied Linguistics. More specifi cally, in online language learning and teaching literature, several researchers (e.g., Balaman & Sert, 2017 ;Cappellini & Azaoui, 2017 ) have utilised Multimodal Conversation Analysis for a micro-analysis of the sequential organisation of talkin-interaction in multimodal computer-mediated pedagogical interactions. ...
... We proposed that it is likely that the same HLAs were also foregrounded in the learners' attention/awareness since the teacher's expressive embodied modes were also available to the learners through the teacher's webcam image. This is in line with Norris (2020 ) in which she suggested the potential to explore the HLAs foregrounded in other interlocutors' attention/awareness through studying one social actor's HLAs and LLAs since the notion of attention/awareness is studied in (inter)action and participants' (inter)actions build on and are impacted by each other. Likewise, when two learners were present, the teacher actively (inter)acted with the learners during SENDING, ALLOCATING , and DESCRIBING THE CONTENT OF THE RESOURCE in the modes of spoken language, gaze, gesture, and facial expressions while simultaneously being engaged in object handling and print modes. ...
... However, the teachers' LLAs had higher multimodal density in the convergent task compared to the divergent tasks. Norris (2004Norris ( , 2020 considers a HLA to be multimodally dense when either (a) a mode plays a central role (modal intensity), or (b) an action can only be performed by multiple modes Contributions, pedagogical refl ections, and future perspectives 151 being inextricably linked (modal complexity). Our analysis suggests that modal density can also be achieved through brisk modal shifts that foreground the LLA in the teacher's attention/awareness. ...
... Multimodal (Inter)action Analysis (MIA) established and described by Norris (2004Norris ( , 2019Norris ( , 2020 constitutes the methodological framework of this study. MIA enables researchers to focus on a single actor's actions (in our case, the teacher) as well as the mediational qualities of these actions through multiple modes. ...
... MIA (Norris, 2004(Norris, , 2019(Norris, , 2020 is "a holistic analytical framework that understands the multiple modes in (inter)action as all together building one system of communication" ( Norris & Pirini, 2016 , p. 24). It considers interactions between social actors and other social actors, objects, or the environment as actions. ...
... Modal density is also a measure of social actors' phenomenological attention/awareness ( Norris & Pirini, 2016 ). Norris (2020 ) describes that attention and awareness, considered as two different sides of the same concept, can be analysed as a continuum: foreground -mid-ground -background. The foreground presents focused attention whereas the background refers to the level to which a performer pays least attention and/or is least aware. ...
... There are many new developments and perspectives. The one that we work in at the AUT Multimodal Research Centre is multimodal (inter)action analysis (MIA) (Norris 2004a(Norris , b, 2011a(Norris , 2029(Norris , 2020Geenen et al. 2015), which has grown from theory and methodology to a fully-fledged framework. I will take a moment to set out the state of this framework and discuss the role that audio-visual technology plays in MIA. ...
... On the overarching philosophical strata, MIA posits that human action, interaction and identity come about through a primacy of perception and a primacy of embodiment (Norris, 2019). On the theoretical strata, MIA follows MDA (Scollon, 1998(Scollon, , 2001, insisting on the principles of social action (including communication) and history (Norris, 2020). On this theoretical level, MIA further follows MDA by arguing that all human actions are mediated, and that mediated actions with a history are practices. ...
... Then, MIA moves on from MDA to multimodal mediated theory, noting that mediated actions appear on different levels (lower-level, higher-level and frozen) (Norris, 2004a). Multimodal mediated theory further suggests that modes are systems of mediated actions (Norris, 2013) and discourses are practices (mediated actions with a history) with an institutional and/or ideological dimension (Norris, 2020). When looking at Figure 1, we see how the philosophical and the theoretical strata seemingly sit above method and methodology. ...
This paper presents a concise introduction to Multimodal (inter)action analysis (MIA), which began to be developed in the early 2000s in tandem with technological advances for visual qualitative research. By now, MIA has grown into a fully-fledged research framework, including multimodal philosophy, theory, method and methodology for the study of human action, interaction and identity. With systematic phases from data collection to transcription (including transcription conventions) and data analysis, this framework allows researchers to work in a data-driven and replicable manner moving past common interpretive paradigms (Norris 2019, 2020).
... Multimodal theory not only believes that language can produce meaning but also believes that nonverbal symbols can also produce meaning, and these non-verbal symbols interact with each other and are independent of each other and work together with verbal symbols to complete the generation of meaning. Ref. [11] discusses the core theories and methodologies in the field of multimodal (interactive) action analysis through examples and image-based transcripts, tables, or diagrams so that readers can gain rich and useful knowledge from them and play an active role in promoting the further development of this field. [12] proposes a theory that treats patterns as symbolic systems, explores how to analyze multimodal texts from a preliminary perspective, and presents a list of problems that may be involved in the new field and need further research To provide inspiration for researchers in related fields and to promote greater breakthroughs and progress in the future development direction of this field. ...
... 1 t + obtained by summing all the blocks in the image. Then the difference between the motion vectors of frames t Eq. (10) is summed over all the blocks, and the smoothness of the t ndon motion can be calculated by Eq.(11). ...
Based on multimodal theory, this paper constructs a multimodal content analysis system for college English with the digital model of teaching content, teaching interactive technology, and virtual reality technology as the main functional modules. The digital transformation of teaching content is done through the use of the particle swarm optimization algorithm and Wiki collaboration, and the learning objects in the learning content library are processed and transformed. The main technical aspects of shot detection, key frame determination, feature extraction, and feature matching are improved to establish the interactive function of English teaching videos. The quadratic error measurement algorithm enhances the edge folding algorithm and streamlines the 3D model to realize the virtual teaching simulation function. Students majoring in business English at a university in Kunming, Yunnan Province, China, are used as research objects to carry out the teaching application practice of this paper’s system. The total average English unit scores of the experimental class increased from 68.2 to 72.1 in the pre-test, and the recognition, comprehension, and utilization dimensions of the learning effect were higher than those of the control class by 0.46, 0.66, and 0.51, respectively, which showed a significant difference (P<0.05). The experimental group of subject students also had a better experience using the system in this paper.
... Importantly for teachers, analysis moreover aligned with past research findings in suggesting the potential functional role of gratitude in the development of positive affect and binding relationships between students (Algoe, 2012;Froh et al., 2008;Seligman et al., 2005). Extending research chiefly involving explicit instantiation of gratitude (e.g., Emmons & McCullough, 2003;Howells, 2004;Seligman et al., 2005), multimodal analysis (Norris, 2020) and interpretation then described the natural emergence of gratitude for learners in one discussion group, as recognition of the supportive actions of peers built up. Moreover, considering Sampson's (2023) dissatisfaction with textual transcripts, this small-lens analysis combined the perspectives and behaviours of group members in a pictorial fashion to provide beginning insights into the subtle, verbal and embodied turns via which gratitude naturally evolves and has specific meaning for individuals. ...
... Indeed, adding historical context is another angle proposed by Ushioda's (2016) small-lens approachthe potential of which has been illustrated admirably by the studies of Consoli (2024) and Sampson (2023). Given the propensity for L2 learning to engender anxiety (MacIntyre, 2017), understanding more about the natural emergence of gratitudesuch as through multimodal analysis (Norris, 2020) like that utilized in the current studyand its impact on student relationships and positive affect would seem particularly useful for pedagogical practice. ...
Link to free download for 50 days: https://authors.elsevier.com/a/1iuZj,7ttADsGu
In light of MacIntyre and Mercer's (2014) demarcation of key areas in positive psychology for second language (L2) learning, this paper concentrates on gratitude. As part of wider practitioner research with Japanese undergraduates in an L2 discussion course, data were collected via introspective journals and videorecording of classroom action. Through initial interactions with this data, the author became aware of the significance of gratitude for students. Adopting Ushioda's (2016) "small-lens" approach to classroom research, the study explored the wider extent, foci, and functions of gratitude referenced by learners over a semester, as well as integrating a finer focus on the ways in which gratitude emerged in one group. The paper utilizes qualitative multiple threading (Davis & Sumara, 2006; Sampson, 2021) to display the widespread degree of gratitude (noted in 72% of journal entries), explicit and implicit foci, and the ways in which participants recognized their own considerate actions. The potential functional role of gratitude in the development of binding relationships between students over the semester is also highlighted. Finally, multimodal analysis of video data (Norris, 2020) visually illuminates the subtle turns via which gratitude evolved and had significant and specific meaning for individuals in one discussion group.
... Since analyzing multimodal interaction means broadening the scope beyond writing and speech (Jewitt, 2009), a comprehensive framework offering researchers a tool to probe into a vast array of meaning-making resources is needed. Regarding this, the framework of multimodal interaction analysis proposed by Norris (2004Norris ( , 2020 is applicable in many educational research contexts. Norris (2004) explained that among the communicative modes observable through multimodal interaction analysis are "proxemics, posture, head movement, gesture, gaze, spoken language, layout, print, (and) musicwhich are essentially systems of representation." ...
... The framework is also applicable in science (Wilmes & Siry, 2021) and higher education (Bernad-Mechó, 2021). These studies are evidence that the framework established by Norris (2004Norris ( , 2020 is relevant and applicable in research across disciplines, classroom settings, and educational levels. Some scholars, however, adopted different frameworks in investigating semiotic resource orchestration in language teaching. ...
Teaching is an activity involving semiotic resources beyond verbal language. Therefore, more studies investigating what semiotic resources are employed and how they are utilized within EFL classroom settings are needed. This study sought to explore the types of semiotic resources employed by two EFL teachers in Indonesia and the roles of these semiotic resources in teaching activities. Through a qualitative case study, two EFL teachers in Bandung and Medan, Indonesia, were purposively selected. Data collection was through a semi-structured interview and document analysis. A six-phase of thematic analysis was employed to analyze the interview results, while a three-step of skimming, reading, and interpretation was used to analyze the documents. This study found that spoken language, pictures, videos, proxemics, gestures, head movements, gaze, and music were the semiotic resources used by both teachers in their teaching. The findings also demonstrated that these semiotic resources were used in the opening, delivering, and closing of the class, explaining learning content, performing interactions with students, checking the students' progress, managing classroom order, showing appreciation, and conducting evaluations. This study implies the crucial roles of all stakeholders in education to help EFL teachers in multimodal knowledge and practical strategy to create better teaching outcomes.
... MDA. Higher-level actions are the sum of chains of different single actions (Norris, 2004(Norris, , 2020, and higher-level actions make the actions interdependent (Zhang and Wang, 2016). Zhang (2009) proposes an outline of the multimodal discourse analysis model (MDAM; see Figure 1), which includes four levels-culture level, context level, content level, and expression level. ...
... Our findings corroborate the previous point of view that classroom teaching is a multimodal experience that happens through orchestration of spoken language and an array of other communicative modes, such as gesture, gaze, and facial expression (Kress et al., 2005;Jewitt, 2006;Peng, 2019;Lim, 2021;Qin and Wang, 2021). The ensemble of the actions, or the sum of chains of different single actions, belong to the higher-level actions (Norris, 2004(Norris, , 2020, which make the actions interdependent (Zhang and Wang, 2016). Our findings also reveal that AR is a useful method to improve teachers' professional development, just as Wallace (1998) mentioned an "empowering procedure, " or quality enhancement (Kember, 2000). ...
This study investigates the influences of action research on primary school English instruction from five dimensions in the classroom, viz., types of questions, language errors, gestures, facial expressions, and interpersonal distance. Four English teachers’ 9 real classroom teaching videos before and after action research are collected and annotated by using ELAN software. The results show that primary school English teachers in Chinese rural areas prefer closed questions to open questions; They make some language errors; Deictic gestures are the most common gestures used, while metaphoric gestures, beat gestures and iconic gestures are rare; Teachers have the same preference for three types of facial expressions, and teachers’ serious expressions accounts for most of the time; They seldom keep an intimate distance or personal distance from their students. Second, Action research is effective to motivate teachers in rural areas, who make great progress in all five dimensions after AR: more open questions are asked; pragmatic errors and grammatical errors are reduced; deictic gestures increase; apathetic decrease, and more intimate distance is exhibited in the interpersonal distance dimension. Third, teacher’s English teaching competence is partly transferrable to her future professional development, and this is also the long-lasting effect of AR.
... For example, the objects present in a classroom construct the mode of layout, which gives off messages about the social actor and structure the interaction somehow. According to Norris (2004Norris ( , 2019Norris ( , 2020, the higher-level actions are also fluid and develop in real-time, and each higher-level action is bracketed by social openings and closings that are at least in part ritualized. Jewitt (2014, p. 36) holds that multimodal interaction focuses on the mediate interaction in a given context, that is, how a variety of modes are brought into and constitutive of social interaction. ...
... The researchers follow the procedures suggested by Norris (2004Norris ( , 2019Norris ( , 2020 to analyze two EFL teachers' multimodal pedagogic discourse during classroom lead-ins. According to Norris (2004, 2019, 2020), the first step to a multimodal interaction analysis is to understand an array of communicative modes. ...
Classroom lead-in is the initial stage for motivating students to become engaged in-class interaction. However, little research, to our knowledge, has analyzed the role of teachers’ multimodal competence reflected through their multimodal pedagogic discourse in the realization of the ultimate goals of classroom lead-ins. Based on the data collected from a teaching contest in China, this paper explores how two-winner teachers utilize their multimodal ensembles of communicative modes to engage students during classroom lead-ins. The analysis shows that different communicative modes construct the higher-level action of lead-in, and they are orchestrated into multimodal ensembles for the specific function of each lead-in move. The findings indicate that EFL teachers’ high multimodal competence plays a decisive role in performing classroom lead-ins, and different lead-ins strategies influence the different orchestration of communicative modes. In constructing multimodal pedagogic discourse, teachers build up their professional image and display their personal charm as well. Future research for multimodal discourse analysis and pedagogic research is suggested in the paper.
... The methodological approach chosen for the analysis in this research is Multimodal (Inter)action Analysis (Norris 2004(Norris , 2019(Norris , 2020. This methodological framework gathers concepts from different theoretical approaches such as mediated discourse (Scollon 2001), studies of social interaction and social semiotics (Kress andvan Leeuwen 2001, 2020). ...
... Various aspects and issues of multimodality have been in the focus of scholars' attention for over forty years so far (Norris, 2016) whereas linguistic approach and interpretation of multimodality have been exercised for a relatively short period of time (Norris, 2020). Currently, one of the most scrutinized linguistic domains of multimodal texts is the YouTube discourse which demonstrates a clear-cut multimodal unity 'constituted by the video content, oral and multiple written elements, numerous semiotic devices' (Zaluzhna, 2021, p. 40). ...
Color terms has multiple times been investigated covering a scope of linguistics domains, such as psycholinguistics, semiotics, semasiology, phraseology, cognitive linguistics, etc. The topic has been thoroughly scrutinized and substantial conclusions concerning cultural, psychological, conceptual and many other aspects of their functioning have been arrived at.
The present paper brings in focus multimodal approach to the study of the color domain which has not yet been addressed in linguistics, color nominations in modern makeup discourse in particular, which defines the novelty of the paper.
The purpose of the research is to outline the mechanisms of color nominations interaction with visual mode (including imagery on the product, overall visual representation of the product, advertising campaign images, etc.) to create a multimodal unity of the makeup advertising discourse.
The object of the research are color nominations in their close connection with the visual mode within the makeup advertising discourse whereas the subject is how the above correlation is created working for the multimodal unity of the products advertised.
The empirical database comprises 37 color nomination units (represented by both single words and words combinations) including their authentic extended descriptions, selected by continuous sampling from Jeffree Star (a provocative American makeup brand) Blood Money collection (released in 2020 and still available for purchasing): an eyeshadow palette (18 color names), liquid lipsticks (8 color names), lip glosses (4 color names), automatic eyeliners (5 color names), highlighters (2 color names).
... This is evidenced by special collections, such as the one edited by Geenen (2023) in this journal positioning MIA as an analytical method for the future. MIA is a multimodal discourse approach and is complementary to systemic functional (Halliday 1978) and social semiotic (Bezemer and Jewitt 2009;Jewitt 2014) frameworks (Norris 2020). While the former explores the ways in which language is organised and used to accomplish a number of social functions (ideational, textual, and interpersonal meanings), the latter aims to understand the agency of social actors as well as social and power relations between them (Jewitt et al. 2016). ...
Multimodal (inter)action analysis offers a powerful and robust methodology for the study of action and interaction between social actors, their environment, and the objects and tools within. Yet its implementation in the analysis of synchronous multimodal online data sets, e.g. (inter)actions via videoconferencing, is limited. Drawing on our research in understanding teacher-learner (inter)actions in instruction-giving fragments in synchronous multimodal online language lessons, we describe and illustrate the ways in which we adapted and extended some of the methodological and analytical tools. These include (1) the use of a grounded theory approach in delineating and identifying higher-level actions, (2) the embodiment and disembodiment of frozen actions, (3) electronic print mode, (4) semiotic lag, (5) semiotic (mis)alignment, (6) modal density (mis)alignment, and (7) how modal density can be achieved by brisk modal shifts in addition to through modal intensity and complexity. We conclude by a call for further educational research in online teaching platforms using the framework to have richer understandings of the (inter)actions between social actors with particular roles and identities (teachers-learners), their environment, and the objects and tools within, which bring their “own material properties, feel and techniques of use, affordances and limitations” (Chun, Dorothy, Richard Kern & Bryan Smith. 2016. Technology in language use, language teaching, and language learning. The Modern Language Journal 100. 64–80: 65).
... Overall, Frames 8-10 which conclude communicating key task information are high in modal complexity, which assists in foregrounding the importance of delivering essential information for task completion within the teacher's attention/awareness continuum. Given all these modes are also available to the learners through the teacher's webcam image, it is likely that the same HLAs are also foregrounded in their attention/awareness ( Norris, 2020 ). The second HLA, focusing on task accomplishment (Frames 11-15) demonstrates similar modal complexity as Craig delivers a modally dense summary of how learners should accomplish the task (Frame 15). ...
... Multimodal theory (e.g., Kress & van Leeuwen, 2006;Norris, 2020), social semiotics (e.g., Metz, 1974;van Leeuwen, 2004), and systemic functional linguistics (e.g., Bartlett & O'Grady, 2017;Halliday, 1978) provide useful conceptual guidelines for "reading" multimodal texts (including film) and are often utilized in cinema studies. The theories all address such topics as shared notions of the social dimension of communication, the grammatical (or structural, compositional) relations of semiotic elements (linguistic and visual grammar) in a text, the metafunctions of linguistic and non-linguistic texts (i.e., ideational, interpersonal, and textual), and contextually specific narratives, ideas, and meanings expressed in the interplay of different modes. ...
This paper describes one 4-week long online language learning module which utilizes the animated documentary film "My Favorite War" (Burkovska-Jakobsen, 2020) and analyzes four students’ final essays. The learning module was created considering ethnographic views of cultural inferences (Spradley, 2016), the framework of social space (Lefebvre, 1991), multimodal theory (Kress & van Leeuwen, 2006), and the model of learning activities, or knowledge processes (Cope & Kalantzis, 2015). It was developed for a Latvian language course at the University of Washington (Seattle, USA) in 2021. Its goals are to help students learn more about the target language and locals’ place-based experiences, and to promote students’ semiotic consciousness while developing their multiliteracies. However, the assignment’s assessment criteria related to the interaction of semiotic resources and of linguistic and non-linguistic information were not shared in order to find out if students would include these issues in their essays. Analysis of students’ essays show that after the learning module, students were able to comprehend (a) some individual compositional elements from the film; (b) symbolic and functional meanings of differently marked spaces; and (c) the main character’s feelings in various social spaces, internal conflicts, relationship models, and thoughts about family, work, war, historical truth, and independence. The described learning model can inspire educators worldwide to develop similar teaching practices for other less commonly taught languages.
... An adaptation of the multimodal (inter)action analysis (MIA) framework (Norris, 2004(Norris, , 2020) was used to explore how one EMI lecturer fostered and managed interaction during an online class. ...
This study analysed the multimodal interactive discourse of one English-medium instruction (EMI) lecturer to engage students in a digital environment. It examined the first live online class given to a group of international students living in different countries. A methodology based on the multimodal (inter)action analysis approach was followed to study how interaction unfolded and was promoted and managed. Results showed the complexity of classroom interaction in this digital environment, the importance of lecturer waiting time, the high modal density and functional diversity of the follow-up/ feedback stage and the most frequent discourse functions expressed during the interaction. The results will be of interest to designers of EMI training courses concerned with student engagement in virtual settings. Some suggestions are given regarding the need to know how to foster EMI lecturers’ awareness of multimodal interactive discourse.
... Another potential avenue to extend both research understandings and methods of representation that might do more justice to the multidimensional nature of emotions is multimodal analysis (e.g., Goodwin, 2000Goodwin, , 2010Goodwin, , 2018Norris, 2004Norris, , 2011Norris, , 2020. Like any area of inquiry, the breadth of this field is truly impressive, and I must admit that I am only just beginning to make my own way into exploring the landscape. ...
This short-form research monograph offers a socially situated view of the emergence of emotionality for additional language (L2) learners in classroom interaction in Japan.
Grounded in a complexity perspective, I argue that emotions need to be studied as they are dynamically experienced and understood in all of their multidimensional colors by individuals (in interaction). Via practitioner research, I apply a small-lens focus (Ushioda, 2016), interweaving experiential and discursive data, offering possibilities for exploring, interpreting and representing the lived experience of L2 study emotions in a more holistic yet detailed, social yet individual fashion. Amidst the currently expanding interest in L2 study emotions, the book presents a strong case for the benefits of locating interpretations of the emergence of L2 study emotions back into situated, dynamic, social context.
The electronic version of this book is available open access - free to view or download - from the following URL (Routledge):
https://www.taylorfrancis.com/books/oa-mono/10.4324/9781003306955/complexity-second-language-study-emotions-richard-sampson
... W ostatnich dwóch dekadach w szeroko pojętej lingwistyce można zauważyć rosnące zainteresowanie komunikacją niewerbalną -zjawisko określane w naukach humanistycznych mianem zwrotu wizualnego (Mitchell 1994) To zainteresowanie przekłada się na popularność takich nurtów, jak multimodalna analiza dyskursu (multimodal discourse analysis (np Kress, van Leeuwen 2001;O'Halloran 2004;Norris 2020)) czy gramatyka systemów wizualnych (np Kress, van Leeuwen 1996;Machin 2016) Ta ostatnia jest przykładem adaptacji modeli systemu językowego do opisu innych systemów semiotycznych na podstawie analogii między leksykonem i regułami gramatyki z jednej strony a zasobami niewerbalnymi danego systemu i zasadami ich użycia z drugiej W podobny sposób poszukiwanie funkcjonalnych analogii między językowymi środkami wyrazu a środkami niewerbalnymi 1 motywowało opracowanie kognitywnych modeli metafory wizualnej 1 Chodzi tu o analogie funkcjonalne na najbardziej ogólnym poziomie relacji, wskazujące na potencjał wyrażania metafor pojęciowych zarówno przez środki językowe, jak i wizualne bądź realizację struktury kognitywnej opartej na domenie źródłowej i domenie docelowej Specyfika obu systemów semiotycznych oraz właściwych im zasobów utrudnia, czy wręcz uniemożliwia, Celem niniejszego artykułu jest przedstawienie wybranych teorii, metod i analiz w dziedzinie kognitywnego ujęcia metafory wizualnej Na początku omawiam dwa źródła tego obszaru badań mieszczące się w tradycyjnym ujęciu metafory wizualnej jako środka artystycznej ekspresji i w kognitywnej teorii metafory werbalnej Następnie przedstawiam aspekt teoretyczny tej dziedziny, opisując wybrany model analizy (Forceville 1996) oraz identyfikacji (VISMIP) metafor wizualnych Przegląd analiz empirycznych pokazuje z kolei dalsze zróżnicowanie metod stosowanych przez autorów oraz badanych przez nich gatunków W ostatniej części artykułu omawiam aktualny stan badań nad metaforą wizualną w Polsce, zwracając uwagę na konieczność popularyzacji i systematyzacji tego obszaru ...
COGNITIVE MODELS OF VISUAL METAPHOR: THEORY, METHODOLOGY, ANALYSES
The aim of the article is to present cognitive studies on visual metaphor: selected theories, methods and analyses. An overview of literature shows a growing interest in this area of research worldwide, especially in the light of the “visual turn”. Compared to the scholarly output of foreign researchers, publications of Polish authors are scarce and their results are not systematized. The discussion of the current state of research on visual metaphor is therefore motivated, on the one hand, by knowledge gap, and on the other, by the need to popularize and systematize this area of study in Poland.
... The (inter)action multimodal framework (Norris 2004(Norris , 2019(Norris , 2020) was adopted to undertake the research. One of the main concerns of this approach is the study of social interaction. ...
This paper examines the evolving genre of university lectures. It focuses on synchronous online lectures. The aim of the study is to shed some light on how interaction between teacher and students unfolds in large English-medium instruction (EMI) lectures in the digital context. A qualitative multimodal microanalysis of an episode of interaction was performed from an (inter)action multimodal analysis framework. This preliminary exploratory study reveals the structural and multimodal complexity of interaction in live online lectures. EMI teacher's semiotic resources combine to make meaning comprehensible in a lingua franca and to engage learners in a virtual context where there is not eye-contact with them. Suggestions are made to undertake contractive studies on interaction in online and face-to-face lectures that may respond to the need of EMI teacher training. This paper aims to contribute to the literature of this still unexplored academic instructional digital genre.
... When investigating entries in the lab book, we have drawn on the notion of modal density, which is used to analyze when and how modes structure attention toward and away from higher level actions, including those "beyond the focus" (Norris, 2016(Norris, , p. 153, 2019(Norris, , 2020, for instance, how medical physicists engage in acceptable research practices of their community. ...
Writing and genre scholarship has become increasingly attuned to how various nontextual features of written genres contribute to the kinds of social actions that the genres perform and to the activities that they mediate. Even though scholars have proposed different ways to account for nontextual features of genres, such attempts often remain undertheorized. By bringing together Writing, Activity, and Genre Research, and Multimodal Interaction Analysis, the authors propose a conceptual framework for multimodal activity-based analysis of genres, or Multimodal Writing, Activity, and Genre (MWAG) analysis. Furthermore, by drawing on previous studies of the laboratory notebook (lab book) genre, the article discusses the rhetorical action the genre performs and its role in mediating knowledge construction activities in science. The authors provide an illustrative example of the MWAG analysis of an emergent scientist’s lab book and discuss its contributions to his increasing participation in medical physics. The study contributes to the development of a theoretically informed analytical framework for integrative multimodal and rhetorical genre analysis, while illustrating how the proposed framework can lead to the insights into the sociorhetorical roles multimodal genres play in mediating such activities as knowledge construction and disciplinary enculturation.
... Multimodal (inter)action analysis (Norris, 2004(Norris, , 2019(Norris, , 2020 is "a holistic analytical framework that understands the multiple modes in (inter)action as all together building one system of communication" (Norris & Pirini, 2016: 24). Within this framework, all actions are considered interactions between social actors and other social actors, objects, or the environment. ...
Online language teaching is gaining momentum worldwide and an expanding body of research analyses online pedagogical interactions. However, few studies have explored experienced online teachers' practices in videoconferencing particularly while giving instructions, which are key to success in task-based language teaching (Markee, 2015). Adopting multimodal (inter)action analysis (Norris, 2004, 2019) to investigate the multimodal construction of instructions in a single case study, we examine instruction-giving as a social practice demonstrated in a specific site of engagement (a synchronous online lesson recorded for research purposes). Drawing on the higher-level actions (instruction-giving fragments) we have identified elsewhere (Satar & Wigham, 2020), in this paper we analyse the lower-level actions (modes) that comprise these higher-level actions, specifically focusing on the print mode (task resource sheets, URLs, text chat, and online collaborative writing spaces) wherein certain higher-level actions become frozen. Our findings are unique in depicting the modal complexity of sharing task resources in synchronous online teaching due to semiotic misalignment and semiotic lag that precludes the establishment of a completely shared interactional space. We observe gaze shifts as the sole indicator for learners that the teacher is multitasking between different higher-level actions. Further research is needed to fully understand the interactional features of online language teaching via videoconferencing to inform teacher training policy and practice.
Despite recent calls to more fully incorporate multimodal perspectives into literacies research, there is still limited scholarship examining how students critically engage in reading activities by drawing on embodied practices. Racially and linguistically minoritized students are particularly disadvantaged by dominant logocentric and developmentalist approaches, which privilege oral and written discourse and often position these students as less capable of performing complex literacy practices. Drawing from three independent ethnographic studies, our multimodal interactional analysis examines how students of a range of ages and raciolinguistic backgrounds use embodied actions and other semiotic resources to agentively navigate text, task, and ideological constraints in activities involving reading and analyzing texts. Our analysis demonstrates the crucial role of students' embodied practices in expanding upon and challenging the constraints of literacy activities, focusing particularly on how students leveraged epistemic stance‐taking, embodied affective responses, and embodied forms of argumentation to negotiate and co‐construct meaning. Through a focus on embodied agency, this paper presents and applies an interactional perspective on the embodied nature of literacy activities; shows how students' creative mobilizations of embodied and other semiotic resources contribute to their critical readings of texts; and offers pedagogical and methodological implications for ways educators and researchers can attend to the intricacies of students' embodied sense‐making in literacy activities.
The journey from a novice to an established educator is fraught with challenges that significantly impact the development of a professional identity. This study examines the experiences of novice university English language teachers in China, focusing on Caroline, an early career teacher who navigates the challenge of navigating a high power culture and integrating into established teaching communities. Employing Multimodal (Inter)action Analysis (MIA), this research offers a nuanced examination of the interplay between Caroline’s imagined and practiced identities. The findings reveal the complexities of her identity formation, highlighting struggles with exclusion, resistance from senior colleagues, and the reconciliation of her aspirations with professional realities. The study advocates for a comprehensive approach to teacher training and support, emphasizing the need for emotional resilience, personal growth, and the integration of theory with practice. It suggests targeted mentorship, the creation of supportive ecosystems, and the adaptation of educational policies to better prepare novice educators for the multifaceted challenges of the teaching profession. This case study contributes to the global discourse on teacher identity formation, offering insights that can inform the development of more effective support structures for novice teachers, thus enhancing the quality of education.
The present paper mainly aims to investigate and explore how English is presented and represented in comparison to Arabic and other non-Arabic languages (if there is any) in different selected regions of Iraq. The data in this work consists of (221) public sector Linguistic Landscape (LL hereafter) markers and have been collected from different Iraqi cities, namely An Nasiriya, Karbala and Baghdad. With its four strategies: duplicating, fragmentary, overlapping, and complementary, Reh's (2004) model has been found suitable to be applied to the data in hand. Not to mention that the concept of multimodality proposed by Norris (2020) also has been found relevant to respond to the nature of the collected data in the analysis. That is to say, going beyond word-based analysis of the collected datasets and further consider other non-verbal (visual) elements of the LL markers give the hint that Iraqi culture might be considered as a bilingual society. However, this study hypothesizes that languages could be verbally or non-verbally displayed on the Linguistic Landscape signs across different Iraqi geographical regions, specifically in the center and south of Iraq. It also hypothesizes that the vast majority of visitors, who share different linguistic and cultural knowledge, enhances their nature with regard to their history and economy. Hence, this might show that Iraqi society is a bilingual one though its people hold one or at the most two different languages: Arabic and Kurdish. As for the record, Kurdish is specifically used by people who live in the north of Iraq. Putting forward a third hypothesis, the presence of different multimodal signs with different languages is socially symbolic where it implicitly reflects the sense of globalization, language policy and ecology.
Este capítulo presenta los principios básicos de los estudios sobre multimodalidad, incluye una definición de los conceptos clave y de las metodologías de investigación más importantes, y analiza cómo contribuye la multimodal en la enseñanza de segundas lenguas. El estudio describe el concepto básico de no centralización lingüística de la lengua y de su enseñanza basada en los diferentes modos comunicativos y la contribución de estos en la construcción de significado. El capítulo está estructurado en tres secciones principales. En primer lugar, una revisión de la literatura previa estudia el concepto de multimodalidad. Se analizan los marcos teóricos y metodológicos más utilizados en el estudio de la multimodalidad, destacando de este modo los conceptos y los objetivos más importantes de enfoques tales como el análisis del discurso multimodal desde la sistémica funcional, la semiótica social, el análisis de la (inter)acción multimodal y el análisis conversacional. A continuación, se examina la contribución que la investigación multimodal ha hecho al campo de la enseñanza de segundas lenguas, identificando tendencias y necesidades investigadoras. La sección se cierra con una propuesta de nuevas líneas de investigación y enfoques para explorar la multimodalidad desde una perspectiva interdisciplinar. Finalmente, el capítulo se centra en la integración de la multimodalidad en la enseñanza del español, y su contribución en la mejora del proceso de enseñanza-aprendizaje a través del desarrollo de la competencia comunicativa multimodal. A modo de ejemplo, se propone la pedagogía basada en el género como un enfoque centrado en la multimodalidad para desarrollar la multialfabetización de los alumnos, así como su creatividad y pensamiento crítico.
The chapter focuses on picturebook read-aloud sessions as multimodal communicative contexts that promote children’s approach to environmental education in a foreign/second language. The initiative TALES (Telling And Listening to Ecosustainable Stories) is the context of observation of the present study. TALES is a series of read-aloud sessions about nature in English L2 for children organized in informal contexts. The aim of the initiative is to promote ecoliteracy through stories that can influence children’s interpretation of events, situations and people and their relationship with the environment (Eisler 2000; Nanson 2021; Stibbe 2021). By adopting a multimodal perspective on communication and interaction (Bezemer and Kress 2016; Jewitt 2017; Norris 2020), the chapter analizes the process of picturebook mediation including storyteller’s picturebook selection and the read-aloud performance with children. The analysis focuses on multimodal aspects that allow storytellers to give salience to natural elements during read-aloud sessions.
In the present text the “edge” metaphor used by Aronin and Politis (in Theory and Practice of Second Language Acquisition 1:27–49, 2015) in their article on multilingualism is taken as a starting point to examine the concept of borders and the specificity of multilingualism in the spacetime of internal and external borderlands on the example of Central and Eastern Europe. Factors influencing identity construction and language choices, such as language policy of the states, political pressures, mobility, gender, religion and social class are identified in the context of moving administrative borders and changing sovereignty. Conflict prevention and management is also discussed as well as the potential of borderlands to function as laboratories of good educational practice promoting individual and social multilingualism.
This paper discusses the linguistic features of the production of football commentary within its macrosocial contexts. It contrasts the communication of the TV production team during a football game with the live football commentary aired on TV. Applying a mixed methods analysis, it reveals how football commentary is prepared and produced in the technical setting of the commentator’s booth in a stadium. This study reveals how a medial reality of the same events concerning a goal are broadcasted in the 11 min highlight video football commentary. The author and her co-principal investigator video recorded a total of 36 h in four work settings. Finally, retrospective interviews with the commentator and head of sports of the TV station place the study in the broader social context of football as a media production. In this paper, I show how live football reporting is prepared and conducted using two different theoretical and methodological frameworks: Conversation Analysis (CA) and Multimodal (Inter)action Analysis (MMIA). Both provide insight into the synthetic verbal, non-verbal and parallel actions for the successful production of the football commentary and they also reveal the sequential structure of naturally occurring talk (CA).
This visual essay examines the non-verbal actions and the production of identity of Serbian and English native speakers during a videoconferencing task in New Zealand. This cross-cultural study examines the modes of facial expression, posture, and hand/arm movements to show how these are telling of the identity of the Serbian and English native speakers. Although gesture and language have received significant attention in research, very few studies have addressed the cross-cultural non-verbal actions and no study has compared Serbian native speakers with New Zealand English speakers. This visual essay addresses this gap.
The present study centers on exploring how different modes such as speech, gesture, gaze, body posture, and head movement are employed by an EFL teacher and his students during a Critical Learning Episode (CLE) (Davis et al.,2009). CLEs are brief instances of classroom interaction where the instructor and the researcher believe that learning is being fostered or inhibited. This article is part of a larger qualitative multiple-case study that took place at a private Colombian University. The lesson was videotaped and then analyzed within a Multimodal (Inter)action Analysis framework (Norris, 2020). Findings indicate that CLEs are created in a highly embodied, multimodal, and ecological manner through different modal configurations. Besides speech and writing, modes such as gestures, posture, gaze, and head movement played not a marginal, but prominent role, in performing various pedagogical classroom activities such as enhancing shared/focused attention, strengthening alignment, helping teachers and learners to visually make meaning of morphological and syntactical units, and serving as devices to check for understanding.
The construction of Samoan identity in New Zealand is shaped by complex historical and contemporary social, economic, cultural and political factors. In addition, New Zealand-born Samoans are negotiating an ethnic and identity that incorporates their experience with the intergenerational stories and cultural knowledge of their ancestors. Such cultural and identity negotiations are occurring through the practice of Siva Samoa, Samoan dance. Using Multimodal (Inter)action Analysis we conduct a micro analysis of two excerpts of video data involving female Samoan dancers rehearsing for a dance showcase. Vertical identity production (Norris, S. (2011). Identity in (inter)action: introducing multimodal (inter)action analysis. Berlin and Boston: Mouton: 179, 2020:85) is used as a framework to analyze the multiple layers of discourse within each site of engagement that shape the construction of Samoan Identity for the participants involved.
In this paper, we explore how a group of 10 and 11-year-old primary school children engage with a picture book about a refugee boy from Somalia. As we examine in some detail a video-recording of the children’s discussion, we suggest that the children’s emotional engagement with the story was pivotal to not only their making sense of the book, but to their critical discussion of the issues the story raised. The discussion we report on here was part of a wider project to examine the use of picture books for critical literacy in schools. Critical literacy is often discussed as a rational endeavour, where children are invited to ask analytical questions about the message a text seeks to communicate and the means by which this is achieved. Following others, for example Anwarrudin (2016), who have challenged this focus on rationalism, we explore the role of emotions in the session. Data show that the children’s critical-analytical discussions of the story were closely connected with their emotional engagement. We use Norris’ (2004) multimodal interaction analysis to examine the children’s emotional and embodied engagement with the book and its story. This analysis of the children’s words, gestures, posture, gaze and voice quality reveals the complexity of their reactions to the book and specifically the role of ‘emotional collisions’ (Kuby, 2012, p. 35) in provoking embodied and affective reactions but also intellectual curiosity and ‘critical engagement’ (Johnson & Vasudevan, 2012, p. 35). With regards to the role of picture books in critical literacy pedagogy, our paper offers teachers new insights into what processes of thinking, feeling and communicating they can expect to be part of critical literacy lessons.
This chapter examines the miscommunication of an intercultural team working on a task via videoconferencing technology using English. We utilize multimodal (inter)action analysis (Norris, 2004, 2011, 2019, 2020) as our theoretical and methodological framework to shed new light on how participants appear to negotiate and co-construct common ground, while they in fact do not achieve conceptual convergence but, instead, produce their own actions. The data for this chapter comes from a corpus of twelve dyads working on tasks via videoconferencing technology in New Zealand. Data was collected from various English monolingual and Serbian multilinguals in various interactive constellations. In this chapter, we focus on a dyadic team with a Serbian native speaker and a monolingual New Zealand English speaker. Rajic and Norris (2018) show that Serbian native speakers’ nonverbal actions vastly differ from New Zealand English speakers’ nonverbal actions. While the difference in production of nonverbal actions is relevant in all interactions, they do not necessarily lead to intercultural miscommunications. However, as noted earlier, interactive alignment is not just a linguistic accomplishment (Pirini & Geenen 2018). Our results conflict somewhat with those of other scholars (House, 1999; Mauranen, 2006), who claim that few miscommunications occur in English as a Lingua Franca interactions. Owing to our analysis, we would like to claim that miscommunications in linguistically and culturally diverse communicative situations are more frequent than previously thought. However, many of the miscommunications that occur cannot be said to come about because of cultural differences.
Sports announcer talk is characteristic for its intonation and grammar. The language of sports commentary is unique in its syntactic style in that it uses grammatical omissions and dislocations (Callies and Levin 2019 Gerhard, 2008, Trouvain 2011). Research has shown that the dramatic phases in incidents in sports announcer talk are an pivotal aspect of speech (Trouvain 2011). The dramatic phases are created with para-verbal language such as prosody, pitch and loudness and they are typical of sports commentary and sports announcer talk. In this presentation, I compare multimodal communication, prosody and pitch during the real event of the goal and the goal as it is narrated on TV.
The purpose of the proposed paper – which places itself within the field of Postcolonial Critical Discourse Studies (see Esposito, E. (2021). Politics, ethnicity and the postcolonial nation – a critical analysis of political discourse in the Caribbean. John Benjamins Publishing Company) – is to analyse the multi-semiotic practices contributing to the characterisation of the protagonist of Trinidadian short-film “Doubles with Slight Pepper” as a cinematic portrayal of the hybrid Indo-Trinidadian identity. By way of a Multimodal Critical Discourse Analysis (Kress, G. and van Leeuwen, T. (1996). Reading images: the grammar of visual design. Routledge, London, Kress, G. and van Leeuwen, T. (2001). Multimodal discourse. The modes and media of contemporary communications discourse. Hodder Educations, London, Kress, G. and van Leeuwen, T. (2006). Reading images: the grammar of visual design, 2nd ed. London: Routledge; O’Halloran, K.L. (2004). Multimodal discourse analysis. Continuum, London), the study aims to cast light on the strategies exploited by the filmmaker (1) to depict the protagonist (Dhani) as an in-betweener whose ambitions are inhibited by his social status stemming from generations of subjugation and misuse by colonialists; and (2) to promote Indo-Trinidadian cultural specificities. Following an introduction to the key concept of individual and collective identity with a focus on Trinidad and Tobago, and an outline of diasporic cinema as applied to the Indo-Trinidadian community, the Multimodal Critical Discourse Analysis is carried out on the verbal, non-verbal, and visual sub-corpora gathered up in sequences according to the main three identitarian traits exhibited in the short-film: ‘religion/folklore’, ‘food’, and ‘lineage’.
(German:)
Sprache tritt immer in multimodalen Kontexten auf. Doch durch zunehmende gesellschaftliche Komplexität und dank technischer Erfindungen wird das Zusammenspiel von Sprache und anderen Modi, insbesondere technisch erzeugten stehenden und bewegten Bildern, immer enger und wirksamer. Eine Linguistik, die Sprache, Äußerungen und Texte nicht nur (wie früher meistens) an und für sich selbst, sondern in ihrem multimodalen Gebrauchskontext untersuchen möchte, steht vor erheblichen methodischen Herausforderungen. Angesichts der Fülle und Vielfalt multimodaler Daten sollte Sprachwissenschaft sich auf die Frage konzentrieren, ob und in welcher Weise Sprache eine kommunikative Arbeitsteilung mit anderen Modalitäten eingeht und wie sie dadurch in ihrer Funktion und Form beeinflusst wird. In diesem Aufsatz wird an zwei unterschiedlichen Beispielen (einem Musikvideo und einem wissenschaftlichen Erklärvideo) untersucht, in welcher Weise Sprache bei dynamischen Sehflächen andere Formen annimmt und andere Funk-tionen erfüllt als sonst üblich. Beide Beispiele sind (wie vermutlich alle dynamischen Sehflächen) mehr oder weniger intensiv ästhetisch komponiert. Dabei zeigen beide Bei-spiele, dass der Wortlaut sprachlicher Texte (Lexik und Grammatik) in Videos sehr stark auf ihre multimodale Umgebung abgestimmt ist.
(English:)
Language always occurs in multimodal contexts. However, due to increasing social complexity and thanks to technical inventions, the interplay between language and other modes, especially technically generated still and moving images, is becoming ever closer and more effective. A linguistics that wants to study language, utterances and texts not only (as was mostly the case in the past) in and of themselves, but in their multimodal context of use, faces considerable methodological challenges. In view of the abundance and diversity of multimodal data, linguistics should focus on the question of whether and how language enters into a communicative division of labor with other modalities and how this influences its function and form. This paper uses two different examples (a music video and a science explainer video) to examine the way in which language in dynamic visual surfaces takes on different forms and functions than usual elsewhere. Both examples (like presumably all dynamic visual surfaces) are more or less intensely aesthetically composed.
In the last decade, Virtual Reality (VR) has been increasingly studied and used for educational purposes (Mellet-d’Huart & Michel, 2006; Gobin-Mignot & Wolff, 2019). Whilst this technology is highly adapted for technical skills training such as flying an aircraft or performing surgery, simulations through VR with language learners seem less relevant at first sight. However, previous work has highlighted numerous potentials, especially for the development of language skills which requires a rich contextualisation to ensure the necessary input and intake processes for aural/oral skills acquisition (Pekarek Doehler & Berger, 2018). We have sought to identify professional social places virtualized by VR applications, the results of which are presented as a taxonomy for teachers in English Professional Purposes (EPP). As mainstream VR applications are mostly designed for leisure purposes, we will discuss their integration into formal learning through the predominant role of educational scenarios that some applications require if they are used in a language teaching/learning context. We will show the value of VR in supporting situated learning specific to professional situations, as well as their potential for soft skills development. The results open the floor for questions that discuss whether VR applications are indeed a complementary resource among other authentic resources used in the classroom and whether they lead to a renewal of language learning practices.
Universities in English speaking countries have been witnessing an increasing number of international faculty who speak English as an international language. However, the universities are mostly guided by the dominant monolingual ideologies and language policies that favor verbal repertoires in standard English even though instructional communication is characteristically multimodal. Against this backdrop, this article focuses on two faculty members in STEM and investigates their instructional interaction. Using multimodal interaction as a theoretical and methodological heuristic, the analysis pays attention to how modes, codes and material objects are ensembled in specific configurations as instructors and students engage in the process of meaning negotiation. The instructors’ instructional practices and perspectives demonstrate the importance of the entanglement of language with diverge semiotic, social and material elements of communication. The findings suggest a need for a broader definition and scope of instructional interaction in STEM.
In this paper, we analyse a group of 6 and 7 year olds’ interactions during a literacy event. We explore the complexities of their meaning-making following a read aloud of Where the Wild Things Are (Sendak 1963). Our focus is on discourses of gender/sex/uality, a term that acknowledges the complex relationship between gender, sex and sexuality, and how these discourses are enacted. Our guiding question was: How did discourses of gender/sex/uality circulate in this group of young children’s multimodal and playful responses to a literacy event? By considering the relationship between reader response, play and gender/sex/uality, we gained insight into how children’s responses to texts are connected to their own identities and lived experiences. We used critical multimodal discourse analysis to understand the children’s meaning-making processes. This revealed how the children were drawing from varying scripts to inform their play and creative processes. The children referenced gender/sex/uality to collaborate, to compete and to seek inclusion or status in the group. We discuss four children who drove this collective dialogue and who guided the group’s interactions. Another child’s responses pushed against and evolved in tandem with the emerging consensus. This study deepened and expanded our consciousness of children’s enactments of gender/sex/uality and how such enactments reinforced heteronormativity. The children’s artefacts, actions and talk are testimony of dominant discourses that guided and ultimately led them to adopt storylines that aligned with heteronormative scripts. Our analysis of how the children’s responses unfolded revealed how power asymmetries were reinforced and hegemonic ideologies persisted. Understanding the influences of social norms during interactive literacy events may help educators create opportunities for all learners to write themselves into these events and classroom interactions more broadly.
In this paper, I apply a metapragmatics-based approach to visual communication, combined with adapted concepts of Social Semiotics (“visual modality”) and CDA-oriented visual analysis (“canons of use”), to reconstruct two visual registers of authenticity which are prevailing within a social media photo sample of recent Austrian chancellor Sebastian Kurz (Facebook, Instagram; total of 84 photos), posted during the parliamentary elections in 2017. Triangulated with the discourse analysis of the marketing manager’s metapragmatic reflections on this social media campaign, the study shows how the partly intertwined registers of (1) “professional sensorism” (as a blended register comprising emblems of sensory modality and balanced composition, thereby drawing on the conceptualization of authenticity as sensory and affective experience of “now”) and (2) “voyeuristic fictionalization” (comprising indexicals associated with fiction genre, and based on the notion of authenticity as arising via “unnoticed observing”) are conceptualized and implemented as a—superior—visual stylization, acting as a social positioning, in mediatized political communication.
Speakers in research dissemination talks are challenged with the need to connect with an audience that does not necessarily share their knowledge and expertise. This communicative situation can be particularly challenging for speakers using English both as a foreign language and for academic purposes. This study combines multimodal and ethnographic methods to explore how speakers of dissemination talks engage with their public. It focuses on four presenters’ use and combination of language, paralanguage, kinesics, proxemics and gaze during intensive moments of engagement. The results show that these interpersonal rich points consist of dense multimodal ensembles that serve to shorten the distance between presenters and their audiences. The findings suggest that a skilful orchestration of modes can be greatly beneficial to achieve the desired level of audience engagement. Therefore, developing speakers' multimodal communicative competence should be a priority in English for specific and academic purposes (ESP/ EAP) training.
This article problematizes the notion of selecting micro-data pieces to shed light upon the focus of participants. The issue presented is two-fold: 1. The article shows that selecting micro-analytical data pieces does not allow a researcher to determine the focus of a participant; and 2. The article demonstrates that language use of a participant does not necessarily mean that the participant is focused upon a conversation. Both, purely working with micro-analytical data pieces and the presumption that language use indicates focus of the speaker, are problematized and it is shown with an example from a relatively large study of family Skype conversation that includes 82 participants that:
1. Focused attention can only be analyzed correctly when crossing micro-analytical boundaries; and
2. A participant can utilize language without paying focused attention to an interaction.
"Il y a eu une telle fertilisation réciproque des idées de la sémiotique de Saussure, centrée sur le code et le langage, et de la sémiotique inspirée par Peirce, qui est pragmatique et interprétative, qu'il est difficile de trouver aujourd'hui un sémioticien qui ne croit pas à la nécessité de développer une socio-sémiotique, interprétative et pragmatique ».
S'il fallait donner l'illustration de cette conception ouverte des avancées en sémiotique, l'ouvrage de Gunther Kress et Théo van Leeuwen : Reading Images - The Grammar of Visual Design, en serait
la meilleure preuve.
By testing a model for analysing identity in interaction, the present article explores how a history student teacher produces social identity in relation to his future profession as a teacher, with an important point of departure being the relationship between the academic and professional aspects of teacher education. This is addressed through an empirical analysis of a student teacher’s identity production in a specific academic setting: a bachelor thesis course. The main body of data consists of audio recordings and video recordings from a group of three student teachers giving feedback on each other’s theses. With respect to methodology, the article employs a model from multimodal (inter)action analysis that focuses on the concept of vertical identity – the notion that identity in interaction is produced in three layers of discourse simultaneously. The results show that the main participant produces the identity of history teacher in an academic setting where such identity production is not encouraged, e.g. by resemiotisising curricula: thus, policy documents can work as a tool when producing teacher identity. This production of identity is done by employing strong agency, which consequently points to the need of a more elaborated discussion on agency in the tested model.
Multimodality is a fast growing area of inquiry as the interest in modes beyond, but always also including language, such as layout, gesture, gaze, or body posture is increasing. A conglomeration of research from various fields interested in examining language as it is embedded in a vast array of other modes, rather than a coherent field of inquiry, this collection showcases the major achievements in this multidisciplinary field, through the inclusion of theoretically, methodologically and practically important works Topics range from language studies to education and classroom discourse, from sociolinguistic inquiries into work place practice to communication, from psycholinguistics and gesture studies to systemic functional grammatical inquiries.
article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/). Selection and peer-review under responsibility of the Organizing Committee of WCLTA 2013. Abstract This article examines tacit participation in an adult art class. Drawing on video excerpts from an extensive 4 month video ethnographic study of an art school, I elucidate how a new student tacitly learns to participate in the group dynamics of the art school. Through video analysis, and using a mediated discourse theoretical (Scollon, 1998, 2011) and multimodal (inter)action analytical lens (Norris, 2004, 2011), I illustrate how the learning of tacit practices is accomplished. I show how successful participation for a novice depends on the following three tenets: 1. the ability to gain focused attention (by the novice); 2. the ability to grant the novice access to shared focused attention (by expert participants); and 3. the ability and willingness of expert participants to relinquish their own focused interaction at times in order to allow the novice to learn successful participation. When these three abilities are present, a new student integrates successfully into a new classroom setting, even if the student is mediocre at art. While, if these abilities are missing, a new student will drop out of the class (in this art school), even if they are very good at art.
According to Nishida (1958) and Bourdieu (1977) social identity is embedded in cultural and social currents, constructed through social and societal histories, and internalized by the individual as habitus. Certainly, identity is constantly constructed on a micro level, where an individual’s identity is claimed, contested, and re-constructed in interaction and in relation to other participants (Scollon 1997). Such constructions are often, if not always expressed as elements through lower-level, higher-level and frozen actions (Norris, 2002, 2004). But social identity is also constructed in relation to others who may not be present at a given moment (Goffman 1959) and it is always constructed through schemes of perception.
In this chapter, I elucidate the interconnections between habitus, social identity construction, and two women’s construction of male domination. In order to demonstrate this complexity, I draw on nexus analysis and multimodal interaction analysis as my methodological frameworks (Scollon & Scollon 2004; Norris 2004); I take into consideration the performance of sequential and simultaneous actions and the (not necessarily intentional) re-production of practices on different timescales as constructed in the histories and cultures of the individuals and the societies that they belong to; and I discuss the two women’s identity and power construction, and thereby investigate what role – if any – agency plays in the process.
Visual attention can increase spatial resolution even when it leads to a decrease in performance. Whether this effect is mediated by reduction of external noise or by signal enhancement is an unsettled question. Although we previously demonstrated that attention can improve speed and accuracy in an acuity task, those experiments made use of a local postmask, which could be considered a source of external noise. In this work, a peripheral cue improved observers' abilities to indicate which side of a Landolt-square target had a gap whether or not a local postmask was used and with both central- and spread-neutral cues. In addition, we documented the presence of visual field inhomogeneities in a resolution task. Given that these experiments presented the target alone with no external noise added (i.e., without distracters or masks), our results indicate that transient attention enhanced the quality of the stimulus representation. Furthermore, because performance in the Landolt-square task indexes resolution, this attentional benefit indicates that transient attention can produce signal enhancement through finer spatial resolution.
Does attention alter appearance? This critical issue, debated for over a century, remains unsettled. From psychophysical evidence that covert attention affects early vision-it enhances contrast sensitivity and spatial resolution-and from neurophysiological evidence that attention increases the neuronal contrast sensitivity (contrast gain), one could infer that attention changes stimulus appearance. Surprisingly, few studies have directly investigated this issue. Here we developed a psychophysical method to directly assess the phenomenological correlates of attention in humans. We show that attention alters appearance; it boosts the apparent stimulus contrast. These behavioral results are consistent with neurophysiological findings suggesting that attention changes the strength of a stimulus by increasing its 'effective contrast' or salience.
Dieser Aufsatz ist in erster Linie ein theoretischer Beitrag, der mein Verständnis
von Rhythmus anhand einer internationalen Videokonferenz über eine sequenzielle
Rekonstruktion des von den Beteiligten im zeitlichen Verlauf konstituierten
Moments hinweg erlaubt. Basierend auf den soziokulturellen Theorien der mediatisierten
Diskursanalyse (Scollon 1998, 2001; Wertsch 1998; Norris/Jones
2005a) und der multimodalen (Inter-)Aktionsanalyse (Norris 2004, 2011a,
2013b; Pirini et al. 2014) beleuchtet dieser Aufsatz: (1) wie eine internationale
Videokonferenz durch rhythmische Zeitgeber (Scollon 2005) beeinflusst wird
und (2) wie die Videokonferenz durch große (Inter-)Aktionsrhythmen koordiniert
wird (Lemke 2000). Insbesondere befasst sich dieser Beitrag mit Rhythmen, die
den emergenten Rhythmen im zeitlichen Verlauf des konstituierten Moments
übergeordnet sind.
Identity construction is a widely covered topic in studies of discourse and a
topic that has interested me for some time (Norris 2002, 2004, forthcoming).
As in my other chapters, my focus in this chapter is a methodological
one that allows the investigation of identity construction from a slightly
new perspective. In this chapter, I take up the topic of personal identity
construction and illustrate what a multimodal approach can offer to grasp
such a complex, fluid and ever-changing notion. While these pages centre
around one social actor in particular, I would like to emphasize that the
reader needs to keep in mind the quote above, which alludes to the fact that
one social actor can never act alone or have a personal identity without the
collective.
My work is grounded in the methodological framework of multimodal
interaction analysis (Norris, 2004) and with this, my writing is first of all an
extension of Scollon’s (1998, 2001) mediated discourse analysis. Second,
this framework is strongly influenced by the work of Kress and van Leeuwen
in multimodality (1998, 2001; and van Leeuwen 1998). Besides these two
merging directions, the framework of multimodal interaction analysis draws
on and builds upon the micro analytical aspects found in interactional
sociolinguistics of Goffman (1959, 1961, 1974), Gumperz (1982) and
Tannen (1984); discourse analysis as in Hamilton (1996, 1998) or Schiffrin
(1994, 2005); and the macro analytical aspects of a historical approach of
Wodak et al. (2001).
This article develops a new methodological tool, called scales of action, which allows the empirical investigation of ubiquitous actions such as driving on the one hand, and the highly complex relationships between (for example) drives and other actions in everyday life on the other hand. Through empirical analysis of ethnographic data of drives performed by a German artist and an American IT specialist, the article illustrates how talk and driving are embedded differently in different cultural contexts. Examining the actions of the two drivers before, during, and after a drive further demonstrates that chronologically performed actions are not necessarily sequential in nature. Using a mediated discourse theoretical approach and building upon multimodal (inter)action analysis, the article provides analysts with a tool that captures the inherent complexities of everyday actions. Through the notion of scales of action and their composition, this article sheds new light upon the complexity and cultural differences of drives and car talk in middle class Germany and North America.
Researchers seeking to analyse how intersubjectivity is established and maintained face significant challenges. The purpose of this article is to provide theoretical/methodological tools that begin to address these challenges. I develop these tools by applying several concepts from multimodal (inter)action analysis to an excerpt taken from the beginning of a tutoring session, drawn from a wider data set of nine one-to-one tutoring sessions. Focusing on co-produced higher-level actions as an analytic site of intersubjectivity, I show that lower-level actions that co-constitute a higher-level action can be delineated into tiers of materiality. I identify three tiers of materiality: durable, adjustable and fleeting. I introduce the theoretical/methodological tool tiers of material intersubjectivity to delineate these tiers analytically from empirical data, and show how these tiers identify a multimodal basis of material intersubjectivity. Building on this analysis I argue that the durable and adjustable tiers of material intersubjectivity produce the interactive substrate, which must be established in order for actions that display fleeting materiality to produce intersubjectivity. These theoretical/methodological tools extend the framework of multimodal (inter)action analysis, and I consider some potential applications beyond the example used here.
Multimodal discourse analysis is an emergent field that began around the verge of the millennium with books such as Reading Images: A Grammar of Visual Design (Kress & van Leeuwen, 1996/2006), Mediated Discourse as Social Interaction (Scollon, 1998), Multimodal Discourse (Kress & van Leeuwen, 2001), and Analyzing Multimodal Interaction: A Methodological Framework (Norris, 2004). Initially, multimodal discourse analysis was primarily the domain of mediated discourse analysts, social semioticians, and systemic functional linguists. While early developments were somewhat overlapping in time, these works resulted from, and aligned with, two separate major paradigm shifts stemming from previous work in discourse analysis...
The philosophical problem of an history of objects, and not only of the history of the « discovery” of an object is tackled in this theoretical article that uses an empirical example -an article by Pasteur- and Whithead’s philosophy. It explores on which conditions, according to Whithead, it would be possible to overcome the limits of « social” explanations of realism without falling back on the realism of the past.
In this article, we take a multimodal (inter)action analytical approach, showing how objects in everyday life are identity telling. As social actors surround themselves with objects, multiple actions from producing the objects to acquiring and placing them in the environment are embedded within. Here, we investigate examples from two different ethnographic studies, using the notion of frozen actions. One of our examples comes from a five-month long ethnographic study on identity production of three vegetarians in Thailand (Makboon, forthcoming); and the other example comes from a four-month long ethnographic study of three working parents on the East coast of North America (Norris, 2006, 2008). We illustrate the frozen actions embedded in particular objects and argue that the analysis of frozen actions allows us to partially understand how identity is produced and experienced by social actors in everyday life.
Pronoun comprehension is facilitated for referents that are focused in the discourse context. Discourse focus has been described as a function of attention, especially shared attention, but few studies have explicitly tested this idea. Two experiments used an exogenous capture cue paradigm to demonstrate that listeners' visual attention at the onset of a story influences their preferences during pronoun resolution later in the story. In both experiments trial-initial attention modulated listeners' transitory biases while considering referents for the pronoun, whether it was in response to the capture cue or not. These biases even had a small influence on listeners' final interpretation of the pronoun. These results provide independently motivated evidence that the listener's attention influences the online processes of pronoun comprehension. Trial-initial attentional shifts were made on the basis of non-shared, private information, demonstrating that attentional effects on pronoun comprehension are not restricted to shared attention among interlocutors.
Noam Chomsky, more than any other researcher, has radically restructured the study of human language over the past several decades. While the study of government and binding is an outgrowth of Chomsky's earlier work in transformational grammar, it represents a significant shift in focus and a new direction of investigation into the fundamentals of linguistic theory.This monograph consolidates and extends this new approach. It serves as a concise introduction to government-binding theory, applies it to several new domains of empirical data, and proposes some revisions to the principles of the theory that lead to greater unification, descriptive scope, and explanatory depth.Earlier work in the theory of grammar was concerned primarily with rule systems. The accent in government-binding theory, however, is on systems of principles of universal grammar. In the course of this book, Chomsky proposes and evaluates various general principles that limit and constrain the types of rules that are possible, and the ways they interact and function. In particular, he proposes that rule systems are in fact highly restricted in variety: only a finite number of grammars are attainable in principle, and these fall into a limited set of types.Another consequence of this shift in focus is the change of emphasis from derivations to representations. The major topic in the study of syntactic representations is the analysis of empty categories, which is a central theme of the book. After his introductory comments and a chapter on the variety of rule system, Chomsky takes up, in turn, the general properties of empty categories, the functional determination of empty categories, parasitic gaps, and binding theory and the typology of empty categories.Noam Chomsky is Institute Professor at MIT. The book is the sixth in the series Linguistic Inquiry Monographs, edited by Samuel Jay Keyser.
Earlier work suggested that "frame," "schema" and related terms could be understood as "structures of expectation." Suggesting now that there are two distinct types of structures of expectation to which such terms have been applied, we use "frames" to refer to the anthropological/sociological notion of interactive frames of interpretation, and "schema" to refer to the cognitive psychological/artificial intelligence notion of knowledge schemas. Drawing on and expanding earlier analysis of talk in a pediatric interaction, we show how frames and schemas interact. Balancing and shifting examination, consultation and management frames accounts for the burden on the pediatrician who examines a child in the mother's presence. Mismatches in the pediatrician's and mother's schemas for health and cerebral palsy account for the mother's discomfort and recalcitrant concerns, and consequently for her frequent questions which trigger the frame switches.
Building on the argument that practices between teacher and learners in classrooms may differ (Scollon and Scollon, 1981; Brice Heath, 1983 [1996]; Street, 1984; Gee, 1996; Barton and Hamilton, 1998), I look at how literacy focused school classroom teaching/learning practices instilled into an individual have a long-term effect. Using a multimodal (inter)action analytical approach (Norris, 2004, 2014) and the site of engagement as my analytical tool that brings together concrete actions, practices and discourses as a coherent whole, I examine actions, practices and discourses produced and reproduced by an art teacher and a new art student in a small private art school in Germany. While the art teacher draws on and re-produces the practice of painting, the new art student draws on and reproduces the practices and discourses that she learned in formal schooling, forcing her to produce and understand modal configurations that do not align with the creative practice that she is learning. This paper has potential educational and social ramifications as it illustrates that formal schooling may have a negative effect upon creativity by focusing the schooled individual upon results and on language/listening. These foci directly translate into modal behaviour which disadvantages the individual when trying to learn a creative practice, where the process and showing/seeing are emphasised. As the world becomes more multimodal and creative, we may want to engage in more research to rethink what and how children are taught.
Preface 2004 1. Introduction 2. Conversational Style: Theoretical Background 3. The Participants in Thanksgiving Dinner 4. Linguistics Devices in Conversational Style 5. Narrative Strategies 6. Irony and Joking 7. Summary of Style Features 8. The Study of Coherence in Discourse 9. Coda: Taking the Concepts into the Present Appendix 1: Key to Transcription Conventions Appendix 2: Steps in Analyzing Conversation Appendix 3: Participants in Thanksgiving Dinner Appendix 4: Flow of Topics in Thanksgiving Conversation REFERENCES AUTHOR INDEX SUBJECT INDEX
In this series of three lectures -shortened in one long paper- is presented the philosophical, sociological and mythical account of the links between humans and non-humans; the first section explores the notion of translation in order to give activity back to objects, the second follows empirical examples of technical systems and the third offers a mythical account of how humans and techniques co-evolved. The general purpose of the article is to show that there are many ways to escape the dualist paradigm separating humans and non-humans.
This paper describes a number of objective experiments on recognition, concerning particularly the relation between the messages received by the two ears. Rather than use steady tones or clicks (frequency or time‐point signals) continuous speech is used, and the results interpreted in the main statistically.
Two types of test are reported: (a) the behavior of a listener when presented with two speech signals simultaneously (statistical filtering problem) and (b) behavior when different speech signals are presented to his two ears.
Centering is a set of processes by which people can coordinate their attention during conversation. Speakers signal where their attention is via particular lexical, syntactic and prosodic choices. These choices make discourse entities salient to addressees and may also signal whether speakers expect addressees' attention to be already centered on the same entities. Two questions arise: What linguistic devices do speakers use to make an entity salient in a discourse? And how do speakers re-refer to discourse entities that move in and out of the center of attention? I manipulated speakers' center of attention using a videotaped basketball game. Speakers tended to refer to prominent entities as sentence subjects. When they referred to entities as sentence objects, they were more likely to re-refer next by repeating the full noun phrase verbatim rather than pronominalising. This happened even though both a recency-of-mention strategy and world knowledge would have provided enough information to uniquely identify the referent of a pronoun. When speakers pronominalised entities that had not been centered syntactically, they tended to make them salient prosodically by lengthening the duration of those pronouns.
Our perception of our everyday interactions is shaped by more than what is said. From coffee with friends to interviews, meetings with colleagues and conversations with strangers, we draw on both verbal and non-verbal behaviour to judge and consider our experiences.
Analyzing Multimodal Interaction is a practical guide to understanding and investigating the multiple modes of communication, and provides an essential guide for those undertaking field work in a range of disciplines, including linguistics, sociology, education, anthropology and psychology. The book offers a clear methodology to help the reader carry out their own integrative analysis, equipping them with the tools they need to analyze a situation from different points of view. Drawing on research into conversational analysis and non-verbal behaviour such as body movement and gaze, it also considers the role of the material world in our interactions, exploring how we use space and objects - such as our furniture and clothes - to express ourselves. Considering a range of real examples, such as traffic police officers at work, doctor-patient meetings, teachers and students, and friends reading magazines together, the book offers lively demonstrations of multimodal discourse at work.
Illustrated throughout and featuring a mini-glossary in each chapter, further reading, and advice on practical issues such as making transcriptions and video and audio recordings, this practical guide is an essential resource for anyone interested in the multiple modes of human interaction.
This article uncovers explicit simultaneous identity construction by applying Scollon and Scollon's (2001) notion of Discourse System and Multimodal Interaction Analysis (Norris, 2004a, 2004b). As a contribution to the theoretical discussion, this article investigates the micropolitics of personal national and ethnicity identity construction of Hispanic/Latino Americans in the Greater Washington DC area as a way of explicating a multimodal framework. This framework allows for the incorporation of multiple modes of communication into a discourse study, explicating how personal national and ethnicity identity can be misunderstood with far-reaching consequences. Turning towards a practical use of the theoretical knowledge, this article is relevant to societal discourses in which members from different cultural backgrounds interact, i.e. to any kind of intercultural scenario in the broadest sense. As such, the article suggests that educating diverse communities about simultaneous identity construction would result in a positive change and a possible solution to the discrepancies that can be found in communities, small groups and families.
This article identifies some limitations of discourse analysis by analyzing interactions between five boys in which the TV and the computer are featured as mediational means. The incorporation of several modalities into transcripts and a shift in focus from primarily language to human action facilitate a better understanding of the multi-modal interaction involved. The use of conventional transcripts with a focus on language demonstrates that movie- and computer-mediated interactions appear fragmented; by contrast, an inclusion of images into the transcripts, representing central interactions and/or images of a movie or computer screen, demonstrates the significant visual modes that are imperative to the ongoing talk. Just as written words correspond to the oral language, images can exemplify the global interaction among the participants, or they can represent the images on the screen. In addition, viewing an image is much faster than reading a description, so that these images also display the fast pace of the movie- and/or computer-mediated interaction.
This study develops an analysis of one site of engagement of public discourse in which identity is socially constructed. Through an analysis of the handing out of handbills in public places, the study argues that such sites of engagement are socially constructed through activities such as handing, a social situation frame in which there are expectations on appropriate behaviors, a regulatory frame of civic responsibility, and a generic frame in which the text itself implies a reader or receiver. The study argues that in such sites of engagement identities are imputed, claimed, ratified or contested and that the ascription of identity is, therefore, inherent in the activities at the sites of engagement in which this discourse takes place. Thus public discourse is argued to be inherently constitutive of social identity.
The dynamic emergence of new levels of organization in complex systems is related to the semiotic reorganization of discrete/continuous variety at the level below as continuous/discrete meaning for the level above. In this view both the semiotic and the dynamic closure of system levels is reopened to allow the development and evolution of greater complexity.
This book explores what speech, music and other sounds have in common. It gives a detailed description of the way perspective, rhythm, textual quality and other aspects of sound are used to communicate emotion and meaning. It draws on a wealth of examples from radio (disk jockey and newsreading speech, radio plays, advertising jingles, news signature tunes), film soundtracks (The Piano, The X-files, Disney animation films), music ranging from medieval plain chant to drum 'n' bass and everyday soundscapes.
Thesis (Ph. D.)--Georgetown University, 2002. Includes bibliographical references (leaves 340-349).
A new class of brightness illusions is introduced that cannot be entirely accounted for by bottom-up models of neuronal processing. In these new illusions, brightness can be modulated by the location of voluntary attention in the absence of eye movements. These effects may arise from top-down or mid-level mechanisms that determine how 3D surfaces and transparent layers are constructed, which in turn influence perceived brightness. Attention is not the only factor that influences perceived brightness in overlapping transparent surfaces. For example, grouping procedures may favor the minimal number of transparent layers necessary to account for the geometry of the stimulus, causing surfaces on a common layer to change brightness together. Attentional modulation of brightness places constraints on possible future models of filling-in, transparent surface formation, brightness perception, and attentional processing.
This chapter focuses on the effect of covert spatial attention on contrast sensitivity, a basic visual dimension where the best mechanistic understanding of attention has been achieved. I discuss how models of contrast sensitivity, as well as the confluence of psychophysical, single-unit recording, and neuroimaging studies, suggest that attention increases contrast sensitivity via contrast gain, an effect akin to a change in the physical contrast stimulus. I suggest possible research directions and ways to strengthen the interaction among different levels of analysis to further our understanding of visual attention.
If speakers articulate clearly enough to meet the perceptual needs of their listeners, clarity should depend on what listeners know about (listener-Given) rather than on what speakers know about (speaker-Given). For words excerpted from spontaneous speech, however, intelligibility to naive adult listeners showed only effects of the speaker's knowledge. Words introducing labeled map landmarks to two successive listeners were less clear on repetition even though the second listener had not heard the original mention (Experiment 1). Repeated mentions became less clear even after the listener reported inability to see the landmark (Experiment 2). Speakers were affected by what they had heard listeners mention: Intelligibility fell equally in coreferential repetitions across and within speakers (Experiment 3), whether or not the repeater could see the referent (Experiment 4). The results are explained via fast priming processes dependent on the speaker's knowledge and slow, optional processes drawing inferences about the listener's.
Chomsky's universal grammar: An introduction
- V J Cook
- M Newson
Cook, V.J. and Newson, M. 1996. Chomsky's universal grammar: An introduction. Cambridge, MA: Blackwell.
- W J M Levelt
Levelt, W.J.M. 1989. Speaking. Cambridge, MA: MIT Press.
Language, power and ideology
- R Wodak
Wodak, R. 1989. Language, power and ideology. Amsterdam: Benjamins.
- W Wu
Wu, W. 2014. Attention. London and New York: Routledge.
Timing and context in children's everyday discourse: Im plications for the study of referential and social meaning
- F Erickson
Erickson, F. 1980. Timing and context in children's everyday discourse: Im
plications for the study of referential and social meaning. Sociolinguistic
Working Paper 67(1-43). Austin, TX: Southwestern Educational Develop
ment Laboratory.