No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Multimodal Discourse Analysis: A Conceptual Framework
Abstract
The overarching theme of Discourse and Technology is cutting-edge in the field of linguistics: multimodal discourse. This volume opens up a discussion among discourse analysts and others in linguistics and related fields about the two-fold impact of new communication technologies: The impact on how discourse data is collected, transcribed, and analyzed—and the impact that these technologies are having on social interaction and discourse.
As inexpensive tape recorders allowed the field to move beyond text, written or printed language, to capture talk—discourse as spoken language—the information explosion (including cell phones, video recorders, Internet chat rooms, online journals, and the like) has moved those in the field to recognize that all discourse is, in various ways, "multimodal," constructed through speech and gesture, as well as through typography, layout, and the materials employed in the making of texts.
The contributors have responded to the expanding scope of discourse analysis by asking five key questions: Why should we study discourse and technology and multimodal discourse analysis? What is the role of the World Wide Web in discourse analysis? How does one analyze multimodal discourse in studies of social actions and interactions? How does one analyze multimodal discourse in educational social interactions? and, How does one use multimodal discourse analyses in the workplace? The vitality of these explorations opens windows onto even newer horizons of discourse and discourse analysis.
... In her discreet work, Martha captures two aspects that are particularly useful for our research. The first refers to the multimodal nature of human communication (Kress and Van Leeuwen 2001;Iedema 2003;Norris 2004;Van Leeuwen 2005;Kress 2010). Recording the actions and conversations of the characters we were talking about, Martha is interested in how the different semiotic modes, from verbal speech to facial expressions, body language and other behaviors, contribute to meaning-making processes (Danesi [1999] 2018, 1). ...
... The fact that "language cannot be isolated from other kinds of semiotic modelling" (Cobley 2016, 27) foreshadows multimodality as the new reality of human communication (Kress and Van Leeuwen 2001;Norris 2004;Kress 2010). Recognition of the multimodal nature of meaning-making processes has become a key premise of the sociosemiotic theory of multimodality (Bezemer and Jewitt 2018). ...
... Iedema (2003, 39) also emphasized the "multimodal nature of all human meaning-making". All our social interactions with otherness are multimodal (Norris 2004). All these observations will soon be supported by research coming from cognitive sciences and psycholinguistics. ...
This paper presents an analytical framework for analyzing how multimodal resources
of emotion expression are semiotically materialized in discursive interactions specific
to political discourse. Interested in how political personae are emotionally constructed
through multimodal meaning-making practices, our analysis model assumes an
interdisciplinary perspective, which integrates facial expression analysis – using
FaceReader™ software – (Ekman, Friesen, and Hager 2002), the theory of emotional
arcs (Dodds et al. 2011; Reagan et al. 2016) and bodily actions (hand gestures)
analysis that express emotions (D’Errico and Poggi 2012; Mittelberg 2013), in the
analytical framework of multimodality. The results show how the multimodal choices
that political actors make during discursive interactions allow them to build their
political brand and make connections with the audience on an emotional level.
... Thus, different modes can be positioned on a foreground-background continuum. Generally, the higher the modal density, the higher a text recipient's awareness/attention (Iedema 2003;Norris 2004). ...
... When creating retail texts, especially intermodality transductions are of paramount importance, since they disproportionally increase modal density, i.e. the level of co-occurrence of sensory modes and the intensity of their interplay (Norris 2004). Therefore, since sensory modes have an impact on customer emotions (Krishna 2012), intermodality transductions are likely to increase customer emotions in retail texts. ...
... The concept of transduction (Kress 2010) refers to the use of different sensory modes from either the same sensory modality (intramodality transduction) or different sensory modalities (intermodality transduction). Intermodality transductions are especially important, since they raise modal density, i.e. the degree of co-occurrence of sensory modes and the intensity of their interplay (Norris 2004), and thus, increase customer awareness/attention (Iedema 2003;Norris 2004). When producing transductions, retailers need to ensure the coherence and consistency of the sensory modes they apply due to the Gestalt principle, according to which customers perceive first the ensemble before paying attention to any details (Alderson 1952;Hultén 2015). ...
The objective of this article is to show how sensory marketing can benefit from taking a multimodal and social semiotic perspective. For this purpose, the framework of ‘Multimodal Sensory Marketing’ is suggested. Important pillars for the scaffolding of this framework are intra- and intermodality transductions. Based on the understanding of retail environments as spatial texts, this article distinguishes between store exterior texts, store interior texts, and customer texts (movements and interactions). The originality of this article resides in the demonstration of how retailers may choose and combine different sensory modes in their meaning making of theme-based retail texts. Intermodality transductions are of particular importance, since they increase the co-occurrence of sensory modes, the intensity of their interplay, and ultimately, may enhance favorable consumer behavior. Transductive links support such transduction processes. Managerial implications and directions for future research are provided.
... Spoken language is only one of the many available mediational means with which people take actions (Jones and Norris 2005: 4). Hence anything and everything that is significant in the situationsuch as space, physical layout, written or multimodal texts, actions and body positionsis taken into account (Norris 2004). Therefore, it is necessity not only to gather data in many modes (audio, visual, textual), but also to do that with different people at different stages to cover different points of view of subjectivities (generalizations, experiences, observations and comparisons) (Norris and Jones 2005c: 202). ...
... When we were finally in the tower, Mira demonstrated and explained the task: Kifibin was supposed to lift the floor tiles and vacuum the floor under them. This excerpt shows the opening of a higher-level actiongiving instructionswhich is composed of many multimodal lower-level actions (Norris 2004). The supervisor utilises the materiality of the particular task in many ways: she demonstrates how to lift up the floor tiles (lines 3 and 5) and points to the significant artefacts (cords and a table leg, line 6). ...
... In Finland, people expect their personal space to be quite large and independence is much appreciated at work, and so Maarit might feel a bit unsure about writing the message on Mae Noi's behalf even though she is used to providing her with scaffolding in many of the tasks she has to do at work. The filled electronic form with the message attached to it becomes a frozen action, a material manifestation of the scaffolding situation (Norris 2004), but reveals only a small part of it. The manager will only see the end product, in which Maarit's contribution remains invisible. ...
Linguistic diversity is growing in labour markets throughout Europe, including Finland, where cleaning is the most common job for immigrants. This paper explores material scaffolding provided for second language users in tasks involved in cleaning work. The notion of ‘scaffolding’ refers to temporary and adaptive support, and here the emphasis is especially on ‘material scaffolding’, that is, material artefacts and body movements employed in mentoring. The theoretical framework of the study is van Lier’s (2004) ecological perspective on language learning, and a discourse-ethnographic perspective of nexus analysis (Scollon and Scollon 2004) is adopted to analyse the ethnographic data collected at two work sites. Scaffolding is regarded as a nexus of social practices in which participants, interaction orders and mediational means intersect. The social actions in work encounters are explored to show how space, body movements and material artefacts are employed in work-related communication. The observations on these two work sites suggest that cleaning work can be successfully performed with a beginner’s level in the work language as long as guided support is available, and especially if the multiple material mediational means in the immediate surroundings are utilized in scaffolding.
... Finally, we draw on another collection of theories and constructs relating to the multiple modes of human interconnectedness, themselves strongly influenced by ethnomethodology. These include Scollon and Scollon's (2004) nexus analysis (NA) and Norris's (2003) multimodal interactional analysis (MIA). ...
... (Jewitt, 2009b, p. 22) The conventions used for representing multimodal aspects of interaction in this study are adapted from different conventions found in ethnomethodological (and its sister discipline, conversation analysis) literature, NA, and MIA (e.g. Jefferson, 1984;Mondada, 2008;Norris, 2003;Scollon & Scollon, 2004), with screen shots from the video data being prioritised over symbolic descriptions of actions for ease of transcription and interpretation by the analyst and by the reader. ...
We introduce the complex social space of basketball training sessions in inner-city Leeds, contextualising the site in relation to stigmatising discourses that suggest social disorderliness. The microanalysis of video data from the training sessions counteracts these stories of unruliness by showing how social orderliness, cooperation and creativity unfold in sophisticated ways in the details of interaction. The paper is rooted in a linguistic ethnographic study of translanguaging in a superdiverse contexts. The significance of its contribution lies in its analysis of communication that bridges across communicative modes to contribute to constituting a culture, as it extends the concept of translanguaging to encompass embodied practice. The paper highlights deeply rooted webs of practices that appear to transcend linguistic and cultural diversity in the basketball training sessions as well as how new practices or shared schemata are taught, learned and creatively deployed.
... Since persuasion is more effective when it is unexpected (O'Keefe, 2002;Perloff, 2003), it also prompts genre dynamism and the integration of new elements (i.e., new semiotic modes) that consequently add to the description of the genre. A multimodal approach to genre analysis is based on the assumption that the genre-creation process is multimodal because users select among different modes to express meaning (Kress & van Leeuwen, 2001;Kress, 2003;Norris, 2004;Kress & van Leeuwen, 2006). The concepts of mode and media, which are central to multimodal analysis, can help to shed light on the production and interpretation processes involved in genres.Jewitt (2004)clarifi es the difference between these concepts, stating that modes enable representation of meaning, while media allow for the dissemination of meaning.Kress (2003)further claims that the production process of a text is closely related to design. ...
... From this analysis, integrated with the information obtained from the interviews with speakers, a series of potentially persuasive points were identifi ed. These are moments in the presentation which are particularly rich in terms of persuasive efforts from the speakers, and are also especially rich in modal density (Norris, 2004). This approach was adopted to avoid prioritizing any semiotic mode in particular, which has proved useful to keep the focus on the multimodal ensemble as a whole and the way different modes interact to encode a persuasive message. ...
... As an analytic methodology, multimodality allows us to understand how people use their meaning making resources in context (Eggins, 1994;Kress & van Leeuwen, 2006;LeVine & Scollon, 2004). A multimodal analysis incorporates all the communicative modes that can be identified in the scope of recorded human interaction (Norris, 2004) allowing researchers to answer both the question of how people use their linguistic resources and how these resources are structured for use (Eggins, 1994). In working with multimodal discourse analysis as methodology, " it becomes apparent that[the various modes]are intricately interwoven, they are not easily separable, and they are interlinked and often interdependent " (Norris, 2004, p. 102). ...
... Youth are engaged in the production of a wide variety of multimodal, new media artifacts including digital stories, video games, video " mash ups " , and spoken word digital poetry, each of which requires the use of a different set of semiotic resources for meaning making. Just as interpreting everyday interactions requires an understanding of how communicative resources work in combination (Norris, 2004), Lemke (2002) argues that the production of multimodal artifacts requires more than understanding how the producer works with each individual mode. Rather, " the meaning potential, the meaningresource capacity of multi-modal constructs is the logical product, in a multiplicative sense, of the capacities of the constituent resources systems " (p. ...
In the 21st century, meaning making is a multimodal act; we communicate what we know and how we know it using much more than printed text on a blank page. As a result, qualitative researchers need new methodologies, methods, and tools for working with the complex artifacts that our research subjects produce. In this article we describe the co-development of an analytic methodology and a tool for working with youth produced films as multimodal artifacts of youth engagement with identity. Specifically, we describe how to employ this multimodal framework in data analysis, with an emphasis on how different modes interact with one another, and how new meanings are made possible through multimodal interactions. © 2012: Erica Rosenfeld Halversonand, Michelle Bass, David Woods, and Nova Southeastern University.
... With the development of discourse analysis transferring to multimodal discourse analysis, multimodality has aroused the researchers' attention and interests, such as in the multimodal interactional analysis from interactive sociology (Norris, 2004;Scollon & Scollon, 2004), in the multimodal discourse analysis from social semiotics (Kress & van Leeuwen, 1996, 2006O'Halloran, 2004O'Halloran, , 2011O'Toole, 1994;van Leeuwen, 2005), in the multimodal discourse analysis from cognitive linguistics (Forceville, 1996;Forceville & Urios-Aparisi, 2009). Although multimodality has aroused the researchers' attention and interests, research on PCs in the multimodal discourse analysis are relatively scarce. ...
... I used purposeful sampling in choosing videos that incorporated a first person perspective (positioning themselves behind the camera) and videos that incorporated a third person perspective (positioning themselves in front of the camera). I conducted a multimodal discourse analysis (Norris, 2004), comparing youth video production choices with their post-production reflection. In particular, I noted the moments in the videos where youth positioned themselves in front of the camera, taking a third person perspective or chose to record from a first person perspective and located the part of their reflection that discussed their production choices in this moment to analyze what factors motivated youth to intentional positioning. ...
... In communication, different modes are used such as languages, signs, symbols, color, audio, video, gestural marks, pictures, images etc. According to Norris (2004), the conceptual framework of MDA allows for the integrations of all noticeable modes of communication performed by social actors. Norris further writes, "modes of communication interact heuristically for meaning-making process" (p. ...
Graffiti is prevalent in modern cities across the world. It represents a range of issues and ideas, and its meaning can be interpreted socially, culturally, and politically. It features distinct forms of language. Using Multimodal Discourse Analysis (MDA) approach, this study analyses the contents, language and linguistic features of 44 graffiti arts found in the Kathmandu Valley to interpret the meanings of the graffiti. The results of this study reveal multiple issues such as culture, politics, gender discrimination and violence against women. The use of language and linguistics in the graffiti seems distinct in terms of word choice, syntactic structure, and rhetorical devices. It was found that the graffitists used multiple modes such as sign, symbol, color, words with images and stylish writing structures. The study will be of great importance to the researchers who want to analyze the language of graffiti and interpret the meanings they denote and to contribute to the body of existing literature on linguistic studies of graffiti. It is also useful for course designers and educators as they can incorporate graffiti in the courses and use them as resource materials in the classroom.
... From these episodes, we chose representative excerpts for micro-level analysis. Young children's ability to put their thoughts into language is still developing, and so in the analysis we underscored the notion that verbal language is just one mode of communication that is presented among other communicational modes (Norris, 2004). ...
... From these episodes, we chose representative excerpts for micro-level analysis. Young children's ability to put their thoughts into language is still developing, and therefore, in the analysis, we underscored the notion that verbal language is just one mode of communication that is presented among other communicational modes (Norris, 2004). ...
In this chapter, we draw on Green's (1988) three-dimensional model of literacy and propose a framework for researching and enhancing children's engagement and learning opportunities in science from a dynamic literacy perspective. The chapter shows how early science education that draws on multiliteracies pedagogy can provide children with rich opportunities to engage in operative, cultural and critical dimensions of scientific literacy embedded in children's life-worlds. The chapter demonstrates how young children benefit from understanding how they can actively participate in the existing scientific culture as it occurs in children's life-worlds. Regarding scientific literacy in the framework of dynamic literacy has the potential to offer cross-disciplinary viewpoints on science education.
... Following on from the heterogeneity in visual studies as to scope, objectives, and methods, there is currently also no consensus as to terms and definitions in multimodal studies. Terms such as mul- timodality, multimodal analysis, semiotic resources, modes, modalities are used rather loosely, have no clear delimitations and often overlap (Kress, 2010;Norris, 2004;O'Halloran, 2008). The experi- ence in general linguistics would suggest that the lack of shared terminology is prone to hinder the development of the field. ...
Multimodal studies posit that meaning is not only communicated through spoken and written words, but also through other modes such as images, gesture, gaze, proximity etc. The widespread availability of high-quality, miniaturised audio and video recording and storing technology has made multimodal data collection cheap and easy. However, the transcription and analysis of the resulting avalanche of recorded data is complex, time-consuming, labour-intensive and expensive. To date there is no established practice or consensus as to scope, methods, objectives or definitions. In fact, concern has been voiced that the field risks expanding to the point of incoherence, sometimes building theory from intuition and generalising from single case studies. Lessons from the 200-year-old discipline of modern linguistics can provide one way forward for the vibrant emerging field of multimodal studies by introducing methods that generate results and hypotheses which can be critically evaluated and empirically tested.
... Like every transcription method, the one presented here reflects the purpose of the study for which it has been developed (d. Baldry, 2005; Baldry & Thibault, 2006; Norris, 2002 Norris, , 2004a Norris, , 2004b Thibault, 2000). As the transcription relies both on images and on language to capture the interaction between images and sound in each CD-ROM and the children's non-verbal and verbal behaviour, it is a multimodal transcription. ...
... new semiotic modes) that consequently add to the description of the genre. A multimodal approach to genre is based on the assumption that the genre-creation process is multimodal, because users select among available modes to express their meaning (Kress & Van Leeuwen, 2001;Kress, 2003;Norris, 2004;Kress & Van Leeuwen, 2006). ...
Multimodality approaches discourse focusing on the analysis of the semiotic modes that help fulfil the communicative purpose of a particular genre, which in the case of product presentations is to persuade of the excellence of the product. We argue that a Multimodal Discourse Analysis approach is especially suitable for the study of product presentations since persuasion in this genre implies the use of different semiotic modes (Chaiken & Eagly, 1976; Poggi & Pelachaud, 2008). The aim of this paper is to present a methodological approach to the study of product presentations from a MDA perspective along with potential pedagogical implications.
... It was a time-consuming process and team discussions and sharing of analytic insights were essential. Although new technologies have enabled new forms of discourse and discourse analysis (Scollon & LeVine, 2004), we developed our own conceptualizations for analyzing multimodal discourse in this context based on previous research on the complexity of Web pages, Web sites, Web users, and Web genres (Baldry & Thibault, 2006; Kress, 2010) and the variety of interwoven interactions that people engage in simultaneously (Norris, 2004). We continued to focus on what participants said about their online activity and Facebook content, and thus emphasized analysis of the transcripts in combination with the Web-based data. ...
Understandings of health behaviors can be enriched by using innovative qualitative research designs. We illustrate this with a project that used multiple qualitative methods to explore the confluence of young adults' drinking behaviors and social networking practices in Aotearoa, New Zealand.
Participants were 18-25 year old males and females from diverse ethnic, class, and occupational backgrounds. In Stage 1, 34 friendship focus group discussions were video-recorded with 141 young adults who talked about their drinking and social networking practices. In Stage 2, 23 individual interviews were conducted using screen-capture software and video to record participants showing and discussing their Facebook pages. In Stage 3, a database of Web-based material regarding drinking and alcohol was developed and analyzed.
In friendship group data, young adults co-constructed accounts of drinking practices and networking about drinking via Facebook as intensely social and pleasurable. However, this pleasure was less prominent in individual interviews, where there was greater explication of unpleasant or problematic experiences and practices. The pleasure derived from drinking and social networking practices was also differentiated by ethnicity, gender, and social class. Juxtaposing the Web-based data with participants' talk about their drinking and social media use showed the deep penetration of online alcohol marketing into young people's social worlds.
Multiple qualitative methods, generating multimodal datasets, allowed valuable nuanced insights into young adults' drinking practices and social networking behaviors. This knowledge can usefully inform health policy, health promotion strategies, and targeted health interventions. (PsycINFO Database Record
(c) 2015 APA, all rights reserved).
... thematic analysis (Braun & Clarke, 2006) and discursive analyses (Willig, 2001) were undertaken with the transcripts of discussions and interviews. We developed our own analytic techniques to examine the multimodal data, based on previous work examining the complexity of web pages, websites, web users and web genres and simultaneous interactions across sites (Baldry & thibault, 2006; Kress, 2010; Norris, 2004). We engaged in regular team discussions (face-to-face and online using scopia software) with named team members, PhD students, and other postgraduate students involved. ...
Young adults in Aotearoa/New Zealand (NZ) regularly engage in heavy drinking episodes with groups of friends within a collective culture of intoxication to ‘have fun’ and ‘be sociable’. This population has also rapidly increased their use of new social networking technologies (e.g. mobile camera/ video phones; Facebook and YouTube) and are said to be obsessed with identity, image and celebrity. This research
project explored the ways in which new technologies are being used by a range of young people (and others, including marketers) in drinking practices and drinking cultures in Aotearoa/NZ. It also explored how these technologies impact on young adults’ behaviours and identities, and how this varies across young adults of diverse ethnicities (Maori [indigenous people of NZ], Pasifika [people descended from the Pacific Islands] and Pa¯ keha¯ [people of European descent]), social classes and genders.
We collected data from a large and diverse sample of young adults aged 18-25 years employing novel and innovative methodologies across three data collection stages. In total 141 participants took part in 34 riendship focus group discussions (12 Pa¯ keha¯ , 12 Ma¯ ori and 10 Pasifika groups) while 23 young adults showed and discussed their Facebook pages during an individual interview that involved screencapture software and video recordings. Popular online material regarding drinking alcohol was also collected (via groups, interviews, and web searches), providing a database of 487 links to relevant material (including websites, apps, and games). Critical and in-depth qualitative analyses across these multimodal datasets were undertaken.
Key findings demonstrated that social technologies play a crucial role in young adults’ drinking cultures and processes of identity construction. Consuming alcohol to a point of intoxication was a commonplace leisure-time activity for most of the young adult participants, and social network technologies were fully integrated into their drinking cultures. Facebook was employed by all participants and was used before, during and following drinking episodes. Uploading and sharing photos on Facebook was particularly central to young people’s drinking cultures and the ongoing creation of their identities. This involved a great deal of Facebook ‘work’ to ensure appropriate identity displays such as tagging (the addition of explanatory or identifying labels) and untagging photos. Being visible online was crucial for many young adults,
and they put significant amounts of time and energy into updating and maintaining Facebook pages, particularly with material regarding drinking practices and events. However this was not consistent across the sample, and our findings revealed nuanced and complex ways in which people from different ethnicities, genders and social classes engaged with drinking cultures and new technologies in different ways, reflecting their positioning within the social structure. Pa¯ keha¯ shared their drinking practices online with relatively little reflection, while Pasifika and Ma¯ ori participants were more likely to discuss avoiding online displays of drinking and demonstrated greater reflexive self-surveillance. Females spoke of being more aware of normative expectations around gender than males, and described particular forms of online identity displays (e.g. moderated intake, controlled selfdetermination). Participants from upper socioeconomic groups expressed less concern than others about both drinking and posting material online.
Celebrity culture was actively engaged with, in part at least, as a means of expressing what it is to be a young adult in contemporary society, and reinforcing the need for young people to engage in their own everyday practices of ‘celebritising’ themselves through drinking cultures online. Alcohol companies employed social media to market their products to young people in sophisticated ways that meant the campaigns and actions were rarely perceived as marketing. Online alcohol marketing initiatives were actively appropriated by young people and reproduced within their Facebook pages to present tastes and preferences, facilitate social interaction, construct identities, and more generally develop cultural capital. These commercial activities within the commercial platforms that constitute social networking systems contribute heavily to a general ‘culture of intoxication’ while simultaneously allowing young people to ‘create’ and ‘produce’ themselves online via the sharing of consumption ‘choices’, online interactions and activities.
... Kucirkova et al. (2013) looked in detail at a parent-child interaction when reading a self-made book on iPad. A detailed multimodal interactional analysis approach allowed the authors to analyse the interaction along verbal and non-verbal interaction indices (Norris, 2004 ) and to distinguish between verbal and non-verbal communication and " embodied (gesture, gaze and language) and disembodied (e.g., a book, iPad, and picture) resources used for meaningmaking " (Kucirkova et al., 2013, p. 117). In addition to the analysis of communicative modes of the parent and child, Kucirkova et al. considered the communication modes of the audio-visual book, i.e., the sounds and images which were part of the story the mother and child viewed and interacted with during story-sharing. ...
Children's interactive e-books are novel literacy tools with interactive and multimodal representations of story contents and increased customizable features. The learning opportunities represented by these new affordances demand a thorough consideration of children's engagement, including the contextual and socio-cultural factors which influence the books' deployment in home and classroom settings. Currently, there is inconclusive evidence about how the affordances of interactive e-books support children's learning, with studies mostly limited to comparison studies with non-digital books and observational studies of children's immediate engagement. In both lines of research, the content of the stories, the overall context of interaction and the background of the interactants are neglected. This article makes recommendations for future research and highlights the value of iPads as a new medium enriching children's experiences but also challenging traditional research assumptions.
When disseminating science to a general audience it is necessary for scientists to refashion the knowledge to meet the audience's expectation that science benefits everyday life rather than just the scientific community. One way to realize this is to transcend the description of scientific inquiry to discuss its implications for the society and humanity. Limited research has been conducted to determine whether and how such transcendence is discursively and multimodally achieved. To fill this gap, I adopted a three-phase analytical approach combining genre analysis and multimodal analysis to investigate 28 TED talks on biology and relevant secondary data. The study revealed that some TED presenters in recent years have adopted the Expanding the horizon move to surpass the intellectual dimension of science and help the audience understand the significance of science. Three multimodal strategies were identified: portraying technological innovations cinematically to create a futuristic sense; presenting quotations from non-scientists multimodally; and strategic use of shots of presenters. The findings suggest a tendency to incorporate “ultra-scientific” perspectives in scientific communication and that multimodal semiotic resources can be useful in such practices.
Qualitative researchers often work with texts transcribed from social interactions such as interviews, meetings, and presentations. However, how we make sense of such data to generate promising cues for further analysis is rarely discussed. This article proposes mode-enhanced transcription as a tool for sensitizing social interaction data, defined as a process in which researchers attune their attention to the dynamic interplay of verbal and nonverbal features, expressions, and acts when transcribing and proofreading professional transcripts. Two scenarios for using mode-enhanced transcription are introduced: sensitizing previously collected data and engaging with modes purposefully. Their implications for research focus, data collection, and data analysis are discussed based on a demonstration of the process with a previously collected dataset and an illustrative review of published articles that display mode-enhanced excerpts. The article outlines the benefits and further considerations of using mode-enhanced transcription as a sensitizing tool.
Liberal studies (LS) textbooks used in Hong Kong public secondary schools have incurred controversy regarding perceived biased contents; however, what appears uncertain are what techniques the textbooks used to incite hostility towards the mainland of China and encourage students to participate in violent acts in Hong Kong. The present study used Michel Foucault’s order of discourse theory, particularly the “division and rejection” framework regarding the power of discourse to analyze five popular LS textbooks. The findings reveal that the textbooks depict a division between the fabricated irrational image of the mainland and the rational image of Hong Kong. Specifically, the textbooks paint Chinese patriotism as irrational while encouraging the student readers to reject their Chinese identity. The textbooks also create a division between altruistic and capable civilians and passive, silent ones, with the morality of the former being connected to violence and radicalism. In this way, the textbooks produce a discourse of hostility towards the mainland and the people in the mainland while encouraging violence.
Discourse is often defined as social meaning-making practice, including but not limited to the linguistic. Even if language is seen as dominant, discourse analysts have found it necessary to analyze non-linguistic data such as images, video, music, architecture, monuments, spatial configurations, or toys. This chapter covers arguments for analyzing multiple types of data: first, within the linguistic domain, e.g. different genres, or spoken and written text; second, different modes as defined in Multimodal (Critical) Discourse Analysis. The analysis of such varied data must address the specifics of each type and offers the potential benefits of triangulation. Critics have argued that such work may require an interdisciplinary team and risks incompatibilities between methods and underlying epistemologies, impressionistic analyses, reinventing the wheel, or propagating linguistic imperialism.
The expression ‘Fake News’ inside Internet memes engenders significant online virulence, possibly heralding an iconoclastic emergence of weaponized propaganda for assaulting agencies reared on public trust. Internet memes are multimodal artifacts featuring ideological singularities designed for ‘flash’ consumption, often composed by numerous voices echoing popular, online culture. This study proposes that ‘Fake News’ Internet memes are weaponized iconoclastic multimodal propaganda (WIMP) discourse and attempts to delineate them as such by asking: What power relations and ideologies do Internet memes featuring the expression ‘fake news’ harbor? How might those manifestations qualify as WIMP discourse? A multimodal critical discourse analysis of a small pool of ‘fake news’ Internet memes drawn from four popular social media websites revealed what agencies were often targeted and from what political canons they likely emerged. Findings indicate that many Internet memes featuring ‘fake news’ are specifically directed, revealing an underlying hazard that WIMP discourse could diminish democratic processes while influencing online trajectories of public discourse.
This article reports on part of an ethnographic research project undertaken over a period of 20 months in Leeds, UK, with a youth spoken word (YSW) poetry organisation. The research focused on the fluid practices in which the youth engage that span spoken, written, visual, gestural, digital, musical and spatial modes, and across times and places. Given its inherent fluidities, YSW is a particularly interesting practice for studying semiosis. Among other aspects, the research focused on the trajectories of poems written and performed by youth and the semiotic transformations they undergo across time and space. The article explores how resemiotisation sheds light on the complexities of the transformations that one particular poem undergoes as it travels. It focuses on a poem titled ‘To Him’, written and performed by a 17-year old poet. The main arguments put forward are that: 1) translanguaging is a more comprehensive term than others for describing how the poet engages her repertoire; 2) resemiotisation is both a lens for conceptualising translanguaging, and an indispensable analytical process in the case of the data studied for understanding the complexities of the poet’s meaning-making practices. Looking beyond our own discipline and making use of musical annotation, we contend that focusing solely on spoken and written language, as is the case in much research on plurilingualism and translanguaging, would be insufficient to gauge the complexity of the meaning-making process undertaken by the young poet (e.g. Pennycook, 2017).
Photos of demonstrations in Egypt have been overwhelmingly used in online news since the revolution of January 25, 2011. This article explores the visual aspects of demonstration photo albums during the political conflicts in Egypt that led to two Egyptian revolutions.¹¹ This article adopts the term “revolution” because the new Egyptian Constitution issued in 2014 described events of January 25, 2011, and June 30, 2013, as revolutions.View all notes It scrutinizes the different types of photo albums, considers the differences and similarities between photo album frames and still photograph frames, and implements syntagmatic analysis of demonstration albums. The article concludes that visual framing occurs through nonchronological and sequential syntagmas.
The chapter discusses the concept of ‘multimodal layering’ as both a theoretical perspective and a methodological framework for analysis. It helps the researcher explore points of potential meaning making coherence created through digital platforms such as iPads. Multimodal layering refers to the multiplicative effect (Lemke in Visual Communication 1(3): 299–325, 2002) when semiotic modes closely associated with a text are reframed in new contexts as users make meaning in digital spaces. The effect occurs in the constant shift between the material and immaterial layers of screens, modes and texts through which individual students need to navigate when reading and writing in learning events (Walsh and Simpson in Australian Journal of Language and Literacy 37(2): 96–106, 2014). This layering is made even more complex when meaning is constructed in the social context of the classroom in the dynamic operationalization of iPads/tablets in private and public spaces. We propose that the multiplicative effects existing within the semiotic boundaries of a text alter when multimodal layering occurs as the text is repurposed in a new interaction and the learner is repositioned to respond to the reconfiguring of semiosis.
This article develops a new methodological tool, called scales of action, which allows the empirical investigation of ubiquitous actions such as driving on the one hand, and the highly complex relationships between (for example) drives and other actions in everyday life on the other hand. Through empirical analysis of ethnographic data of drives performed by a German artist and an American IT specialist, the article illustrates how talk and driving are embedded differently in different cultural contexts. Examining the actions of the two drivers before, during, and after a drive further demonstrates that chronologically performed actions are not necessarily sequential in nature. Using a mediated discourse theoretical approach and building upon multimodal (inter)action analysis, the article provides analysts with a tool that captures the inherent complexities of everyday actions. Through the notion of scales of action and their composition, this article sheds new light upon the complexity and cultural differences of drives and car talk in middle class Germany and North America.
This chapter reflects on two ethnographic studies that both necessitated the foregrounding of non-linguistic frameworks, drawing on visual research methods, in order to best understand children’s meaning making. The cultural contexts of these two studies, and the age ranges of the participants, did not fit easily with Western linguistic theory, but tied more naturally with visual research theories and techniques, and required us as researchers to consider the research environments as visual and sensory places. This chapter describes how we focused on the children’s embodied, non-verbal communicative practices and meaning making in these places and captured them with visual data collection techniques.
Researchers seeking to analyse how intersubjectivity is established and maintained face significant challenges. The purpose of this article is to provide theoretical/methodological tools that begin to address these challenges. I develop these tools by applying several concepts from multimodal (inter)action analysis to an excerpt taken from the beginning of a tutoring session, drawn from a wider data set of nine one-to-one tutoring sessions. Focusing on co-produced higher-level actions as an analytic site of intersubjectivity, I show that lower-level actions that co-constitute a higher-level action can be delineated into tiers of materiality. I identify three tiers of materiality: durable, adjustable and fleeting. I introduce the theoretical/methodological tool tiers of material intersubjectivity to delineate these tiers analytically from empirical data, and show how these tiers identify a multimodal basis of material intersubjectivity. Building on this analysis I argue that the durable and adjustable tiers of material intersubjectivity produce the interactive substrate, which must be established in order for actions that display fleeting materiality to produce intersubjectivity. These theoretical/methodological tools extend the framework of multimodal (inter)action analysis, and I consider some potential applications beyond the example used here.
Mit seinen Darstellungen der Geschichte des Publikums hat Richard Butsch gezeigt, dass im Verlauf der Kommunikationsgeschichte die jeweiligen zeitspezifischen Formen und Formate der öffentlichen Kommunikation auch das entsprechende Publikum geschaffen haben (Butsch 2008).
Analyzing Public Discourse demonstrates the use of discourse analysis to provide testimony in public policy consultations: from environmental impact statements to changes in laws and policies.
Nexus Analysis presents an exciting theory by two of the leading names in discourse analysis and provides a practical guide to its application. The authors argue that discourse analysis can itself be a form of social action. If the discourse analyst is part of the nexus of practice under study, then the analysis can itself transform that nexus of practice. Focussing on their own involvement with and analysis of pioneering communication technologies in Alaska they identify moments of social importance in order to examine the links between social practice, culture and technology. Media are identified not only as means of expressing change but also as catalysts for change itself, with the power to transform the socio-cultural landscape. In this intellectually exciting yet accessible book, Ron Scollon and Suzie Wong Scollon present a working example of their theory in action and provide a personal snapshot of a key moment in the history of communication technology, as the Internet transformed Alaskan life. © 2004 Ron Scollon and Suzie Wong Scollon. All rights reserved.
Today’s emerging technological achievements seem to be moving towards the realization of ubiquitous learning as described by Weiser (1991). Nevertheless, ubiquitous learning is not preconceived or a priori; the number of possibilities offered by such learning can only happen through strategies and practices that re-conceptualize the content, processes and human relationships of teaching and learning. Bearing the previous in mind, the aim of this paper is to report on the findings from a doctoral thesis on students’ multimodal experiences resulting from engagement in the creation of a student-generated virtual museum. Drawing from the literature, analysis and evaluation of the qualitative research methodology this paper addresses how engagement with an online educational tool such as WebQuests impacts upon pupils’ multimodal awareness, and its potentials to support ubiquitous learning. To facilitate understanding a learning framework based on multimodal literacies, the theory of communities of practice and museum based pedagogy is presented in brief. Following this, the findings of the intervention are discussed in the context of the case study undertaken with a group of primary aged students to determine the extent to which the intrinsic characteristics of ubiquitous learning could be served by the affordances of the WebQuest method within the learning framework implemented. © Common Ground, Stefania Savva, Nicos Souleles, All Rights Reserved
In this article, we take a multimodal (inter)action analytical approach, showing how objects in everyday life are identity telling. As social actors surround themselves with objects, multiple actions from producing the objects to acquiring and placing them in the environment are embedded within. Here, we investigate examples from two different ethnographic studies, using the notion of frozen actions. One of our examples comes from a five-month long ethnographic study on identity production of three vegetarians in Thailand (Makboon, forthcoming); and the other example comes from a four-month long ethnographic study of three working parents on the East coast of North America (Norris, 2006, 2008). We illustrate the frozen actions embedded in particular objects and argue that the analysis of frozen actions allows us to partially understand how identity is produced and experienced by social actors in everyday life.
This paper investigates audience design in monologues. The study uses video blogs, a spoken, asynchronous form of computer-mediated communication, to illustrate how talk reflects the lack of an immediately present audience. It is based on a corpus of English language vlogs collected from the video hosting site YouTube. The study demonstrates how speakers adapt to a mediated speech situation where there is not even minimal feedback and the speaker has to address absent viewers. Clark and Carlson's audience design (1992) and Bell's audience design (1984), introducing the notion of participant roles, are central constructs in the present study. It is argued that when vloggers (re)assign participant roles, the audience is actively involved, as they have to recognize their new status. The phenomena examined in this paper include multimodal, syntactical, and lexical features. The particular context of the medium and the monologic nature of the data will be given special consideration in the analysis of genre specific features. These include terms of address (e.g. YouTubers, YouTube, vlog fans), directives/directed language (e.g. comment, rate, and subscribe, leave me a comment). Other features under discussion include questions (how are you guys doing), voicing the audience, constructed dialogue, whispering, gestures, categorization etc.
Goffman's dramaturgical model of interaction is often used to examine how communicative events are framed differently in public and private settings. However, this idea has restrictively been conflated with specific spatial locations. This paper contends that it is not the place where discourse is performed that styles it as front or backstage, but rather the stances taken in interactions, and the participation framework involved. Furthermore, backstage can be used to negotiate stances later performed frontstage, where they may index wider, ideological aspects of identity. This is demonstrated through the analysis of a sequence of linked backstage and frontstage interactions that involve two rugby coaches. The extracts presented and analysed take place in a New Zealand rugby team, a type of organisation in which space is particularly fluid. The dramaturgical model is demonstrated to be of use in examining leadership discourse within organisations, particularly in organisations in which multiple leaders exist.
Little is known about how specific iPad applications affect parent–child story-sharing interactions. This study utilises a case-study approach to provide an insight into the patterns of interaction, which emerge when a mother and her 33-month-old daughter share a self-created, audio-visual ‘iPad story’. Multimodal analysis allowed us to gain insights into the complex interaction patterns orchestrated in this new, personalised story-sharing medium. We found that the app-mediated story-sharing context produced a harmonious and smooth interaction, achieving a coherence that is typical of ‘happy’ oral stories. We suggest that the observed interaction resembles that of experiencing a piece of art, and we highlight the need for a holistic approach to understanding the implications for research and practice of children's interactions during multimedia story sharing.
Maguru panggul pedagogy is a traditional form of instruction used to teach Balinese Gamelan that literally means the teacher is the mallet. This pedagogy is presented as a semiotic apprenticeship where students become active participants by not only learning the music, but by also contributing their knowledge about it during negotiated classroom interactions. Three examples from research conducted in two Balinese Gamelan classrooms in the United States are analyzed utilizing the theory of mediated discourse analysis, and discussing these examples in terms of distributed cognition. Maguru panggul is revealed as a semiotic apprenticeship that draws upon several semiotic modes.
This article explores the way users of an online gay chat room negotiate the exchange of photographs and the conduct of video conferencing sessions and how this negotiation changes the way participants manage their interactions and claim and impute social identities. Different modes of communication provide users with different resources for the control of information, affecting not just what users are able to reveal, but also what they are able to conceal. Thus, the shift from a purely textual mode for interacting to one involving visual images fundamentally changes the kinds of identities and relationships available to users. At the same time, the strategies users employ to negotiate these shifts of mode can alter the resources available in different modes. The kinds of social actions made possible through different modes, it is argued, are not just a matter of the modes themselves but also of how modes are introduced into the ongoing flow of interaction.
What is the role and meaning of emotions in the second language learning process? To respond to this question, this article focuses on how learners' emotions manifest in their verbal communication over the course of a semester-long joint task. Recognizing interpersonal, functional, and developmental aspects of emotions, I illustrate how a group of English-as-aforeign-language learners discursively constructed and shared their emotional attitudes toward their group work and how such emotional intersubjectivity pushed the group, in their knowledge co-construction, to challenge assigned tasks and material. I argue that emotions do not merely facilitate, filter, or hinder an individual's inner cognitive functioning; rather, they can in any forms mediate development, especially when learning is embedded in an interpersonal transaction. I end by considering implications of the study and its limitations.
Recent reforms to curriculum standards in China have highlighted for the first time ‘emotion and attitude education’. This new focus is the pedagogic backdrop to the research reported in this article–an exploration of how evaluative stance is construed through the co-deployment of linguistic and visual resources in primary and secondary textbooks for teaching English as a foreign language in China. In particular, this study considers language-image complementarity and co-instantiation. It is found that linguistic and visual appraisal resources play essential roles in realizing various attitudinal curriculum goals, guiding students to the putative reading and in the joint construction of texts. Working towards an ontogenetic view on the attitudinal accumulation, it identifies an attitudinal shift from an emotional release to a more institutionalized type of evaluation as students advance through the school years.
Currently, several million people around the globe have accounts in the virtual worlds of massively multiplayer online video games. The overall population of these virtual worlds has grown rapidly since 1996. This thesis offers an insight into interaction forms available in massively multiplayer online games by analyzing their communicative and social aspects. The thesis aims to provide a deeper understanding of how virtual environments may be used in the future to overcome the limitations of current text-based communication, which are classified by a reduced set of intuitive non-verbal cues. The work is conducted using conceptual analysis, by applying a theoretical model that translates the perceivable interaction forms that human beings use in face-to-face interaction in the real world to virtual worlds. This thesis will argue that that the richness of interaction within computer- mediated environments (CMEs) varies depending on the available medium, and that CMEs structure the processes of identity presentation and communication in significant ways. Additionally, it argues that the number of channels (e.g. speech, gesture or posture) and their dynamics (e.g. the simultaneous use of multiple channels) affect the overall level of richness of communication that takes place between users within computer-mediated environments that support communication. Finally, it will argue that the increased range of communicative channels that support interaction within massively multiplayer online video games can be used to overcome the limitations of current text-based communication in cyberspace, and that, as virtual worlds become an increasingly common phenomenon, they will influence the processes by which internet users communicate and present their identities online in the future.
From emails relating to adoption over the Internet to discussions in the airline cockpit, the spoken or written texts we produce can have significant social consequences. The area of mediated discourse analysis considers the actions individuals take with texts – and the consequences of those actions.
Discourse in Action brings together leading scholars from around the world in the area of mediated discourse analysis and reveals ways in which its theory and methodology can be used in research into contemporary social situations. Each chapter explores real situations and draws on real data to show how the analysis of concrete social actions broadens our understanding of discourse. Taken together, the chapters provide a comprehensive overview to the field and offer a range of current studies that address some of the most important questions facing students and researchers in linguistics, education, communication studies and other fields.
Our perception of our everyday interactions is shaped by more than what is said. From coffee with friends to interviews, meetings with colleagues and conversations with strangers, we draw on both verbal and non-verbal behaviour to judge and consider our experiences.
Analyzing Multimodal Interaction is a practical guide to understanding and investigating the multiple modes of communication, and provides an essential guide for those undertaking field work in a range of disciplines, including linguistics, sociology, education, anthropology and psychology. The book offers a clear methodology to help the reader carry out their own integrative analysis, equipping them with the tools they need to analyze a situation from different points of view. Drawing on research into conversational analysis and non-verbal behaviour such as body movement and gaze, it also considers the role of the material world in our interactions, exploring how we use space and objects - such as our furniture and clothes - to express ourselves. Considering a range of real examples, such as traffic police officers at work, doctor-patient meetings, teachers and students, and friends reading magazines together, the book offers lively demonstrations of multimodal discourse at work.
Illustrated throughout and featuring a mini-glossary in each chapter, further reading, and advice on practical issues such as making transcriptions and video and audio recordings, this practical guide is an essential resource for anyone interested in the multiple modes of human interaction.
This article uncovers explicit simultaneous identity construction by applying Scollon and Scollon's (2001) notion of Discourse System and Multimodal Interaction Analysis (Norris, 2004a, 2004b). As a contribution to the theoretical discussion, this article investigates the micropolitics of personal national and ethnicity identity construction of Hispanic/Latino Americans in the Greater Washington DC area as a way of explicating a multimodal framework. This framework allows for the incorporation of multiple modes of communication into a discourse study, explicating how personal national and ethnicity identity can be misunderstood with far-reaching consequences. Turning towards a practical use of the theoretical knowledge, this article is relevant to societal discourses in which members from different cultural backgrounds interact, i.e. to any kind of intercultural scenario in the broadest sense. As such, the article suggests that educating diverse communities about simultaneous identity construction would result in a positive change and a possible solution to the discrepancies that can be found in communities, small groups and families.
ResearchGate has not been able to resolve any references for this publication.