Article

Multimodal acquisition of interactive aptitudes: A microgenetic case study

Authors:
To read the full-text of this research, you can request a copy directly from the author.

Abstract

In this article, I detail incremental microgenetic alterations in the development of one particular socio-interactive aptitude: making a relevant interactive contribution. Taking heed of Clark’s (2014) call for the need to reorient our attention to investigate the pragmatics of interaction by accounting for the multiple communicative modes through which this is acccomplished I detail the ways in which parental facilitation and a flexible participatory configuration, made possible by video-conferencing technology, create conditions enabling the agentive re-introduction of a psycho-socially relevant topic. Paramount are the ways in which residual interactive specificities in introduction, co-production and multimodal configurations re-manifest suggesting a more symbiotic relationship between traditional notions of ‘message’ and ‘production’. During the microgenesis of interactive aptitudes, children are not just learning what constitutes psycho-socially relevant topoi, they also acquire an understanding of exactly how to make the contribution through multimodal ensembles.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

... Specifically, contributions from other interlocutors who may not be directly visible are not detailed. Geenen (2018) takes this up elsewhere arguing that constantly changing participation frameworks and parental facilitation off-screen may contribute to certain forms of pragmatic development for younger interlocutors. Multiparty interactions mediated by videoconferencing technologies create challenges for budding conversationalists who are in the early stages of acquiring communicative competence. ...
... There are unique attentional demands, certain ambiguities about who is being addressed, who should take the lead and who is responsible for various topics, comments and closures. Geenen (2018) argues however, that the fluidity with which interlocutors enter and exit the interaction creates opportunities for younger interlocutors to introduce 'new' topics. ...
... While topic introduction in social interaction has been traditionally approached by looking at the use of spoken language, Geenen (2018) argues that topic introduction can occur through the presentation of physical objects. Additionally, topic introduction is a collaborative phenomenon and one recurrence of a topic introduction suggests that children are learning not only what topics are considered appropriate to introduce with certain social actors, but also, how to introduce said topics. ...
Article
Full-text available
FULL TEXT HERE: https://rfmv.fr/numeros/5/articles/11-visual-transcription/ The communication and language sciences have decidedly taken a multimodal turn and a proliferation of work in previously language-dominated fields is focusing on the contribution of non-verbal communicative modes in social interaction. While this has proven empirically and theoretically fruitful, it throws an additional kink in an already complicated issue: transcription practices and the interpretive and representational process accompanying said practices. In this article, we review and champion a methodological framework that provides analytical tools and a transcription protocol for the generation of visual transcripts. Visual transcription, we contest, is far more congruent with the multimodal ethos and we both detail the method while applying it to a data segment from a larger project investigation teamwork and collaborative problem-solving in dyadic videoconferencing interactions. The analytical focus on action coupled with the accompanying transcription method reveals complex and fluctuating distributions of attention and interactional awareness. Non-verbal actions reveal contributions to the task at hand which are not realized in talk and the analysis details an intricate ebb and flow of attentional orientations which are realized through non-verbal means.
... The inclusion of all communicative modes in research has revealed that very young children (2 years) are sensitive to subtle alterations in participation frameworks (Geenen, 2018). They appear adept at tracking conversational topio and re-introducing said topics to newcomers who may have 'missed something'. ...
... They appear adept at tracking conversational topio and re-introducing said topics to newcomers who may have 'missed something'. The actions themselves, however, are more nuanced and do not always occur through the mode of spoken language as children draw upon a range of non-verbal modes like gesture and object handling to introduce topics (Geenen, 2017(Geenen, , 2018. Only a methodology which prioritises the pragmatics of interaction (Clark, 2014) can help accurately deduce when and to what extent communicative aptitudes emerge. ...
Chapter
Multimodal (Inter)action Analysis was developed to study social interaction based upon the theoretical notion of mediated action. Building on this core concept, Multimodal (Inter)action Analysis includes several theoretical/methodological tools. These tools facilitate analysis which moves flexibly between micro-level moments of interaction and macro-level practices and discourses. In this chapter, the application of mediated action to multimodal analysis is discussed, before the central theoretical/methodological tools are introduced. Tight links are made between the tools used in Multimodal (Inter)action Analysis and the core theoretical tenets, to support robust multimodal interaction research.
... Multimodal (Inter)action analysis has already proved quite valuable in unravelling some of the complexities which emerge through new technologically-mediated forms of social interaction. In the fractured interactional ecologies which emerge when communication is complexly mediated by new technologies, non-verbal modes take on a new importance and appear to be pivotal for exemplifying divergence in stance or disagreement (Norris and Pirini 2017), can be vital for the production of identity (Geenen 2017) and can be exploited to help facilitate smoother interactions with pre-verbal children and toddlers (Geenen 2018(Geenen , 2020. Other work has shown how non-verbal actions can be pivotal to accomplishing collaborative tasks via video-conferencing technology (Geenen et al. 2021) but that the distribution of attention and the interactional demands can also result in miscommunication and misunderstanding (Norris and Geenen 2022). ...
Article
Full-text available
Red Chinese movies depict heroic individuals who make significant contributions to the nation, and these films play a vital role in constructing the national image of new China. This paper explores the images of heroic individuals in red Chinese movies using multimodal action theory. The findings indicate that these individuals are characterized by their selfless sacrifices for their country, fearless perseverance, embodiment of international communist ideals, and unwavering devotion and warmth. The study also reveals that facial expressions, body movements, and language serve as critical mediational means in shaping these characters, while other forms of mediation are underutilized. Recommendations include employing more mediational means to create more nuanced and complex characters in future red Chinese movies.
Article
Full-text available
This paper presents a concise introduction to Multimodal (inter)action analysis (MIA), which began to be developed in the early 2000s in tandem with technological advances for visual qualitative research. By now, MIA has grown into a fully-fledged research framework, including multimodal philosophy, theory, method and methodology for the study of human action, interaction and identity. With systematic phases from data collection to transcription (including transcription conventions) and data analysis, this framework allows researchers to work in a data-driven and replicable manner moving past common interpretive paradigms (Norris 2019, 2020).
Article
Multimodal interaction analysis is a holistic methodological framework that allows the analyst to integrate the verbal with the nonverbal, and to integrate these with material objects and the environment as they are being used by individuals acting and interacting in the world. In short, multimodal interaction analysis allows a researcher to study real people interacting with others, with technology, and with the environment. The point of view taken in multimodal interaction analysis is that all actions in fact are interactions and that all of these (inter)actions are linked to people (referred to as “social actors” in multimodal interaction analysis)—no matter whether you are investigating the real or the virtual world, someone buying ice cream, or using a software program. Multimodal interaction analysis is carried out through a series of analytical phases and steps. Researchers utilize multimodal transcription conventions and analytical tools, which are theoretically founded, allowing for replicability of analyses and reliability of findings. Multimodal interaction analysis allows the integration of all communicative modes, where communicative modes are defined as systems of mediated actions. In this view, all modes of communication together build one system of communication, which comes about through actions and interactions that people produce. This coherent framework and process of analysis can be taught and learned and is proving useful for research examining what people do, how they communicate, and how they interact.
Conference Paper
Book of Abstracts of the 6th Critical Approaches to Discourse Analysis across Disciplines Conference (CADAAD 2016)
Article
Full-text available
This article presents theoretical concepts and methodological tools from multimodal (inter)action analysis that allow the reader to gain new insight into the study of discourse and interaction. The data for this article comes from a video ethnographic study (with emphasis on the video data) of 17 New Zealand families (inter)acting with family members via skype or facetime across the globe. In all, 84 social actors participated in the study, ranging in age from infant to 84 years old. The analysis part of the project, with data collected between December 2014 and December 2015, is ongoing. The data presented here was collected in December 2014 and has gone through various stages of analysis, ranging from general, intermediate to micro analysis. Using the various methodological tools and emphasising the notion of mediation, the article demonstrates how a New Zealand participant first pays focused attention to his engagement in the research project. He then performs a semantic/pragmatic means, indicating a shift in his focused attention. Here, it is demonstrated that a new focus builds up incrementally: As the participant begins to focus on the skype (inter)action with his sister and nieces, modal density increases and he establishes an emotive closeness. At this point, the technology that mediates the interaction is only a mundane aspect, taken for granted by the participants.
Article
Full-text available
ABSTRACT Recent research has highlighted several areas where pragmatics plays a central role in the process of acquiring a first language. In talking with their children, adults display their uses of language in each context, and offer extensive feedback on form, meaning, and usage, within their conversational exchanges. These interactions depend critically on joint attention, physical co-presence, and conversational co-presence - essential factors that help children assign meanings, establish reference, and add to common ground. For young children, getting their meaning across also depends on realizing language is conventional, that words contrast in meaning, and that they need to observe Grice's cooperative principle in conversation. Adults make use of the same pragmatic principles as they solicit repairs to what children say, and thereby offer feedback on both what the language is and how to use it.
Article
Full-text available
In this article, we report a study of the uses of mobile video telephony based on the collection and analysis of naturally occurring mobile and Skype video exchanges, with a focus on camera motions. We provide evidence for a set of phenomena characteristic of the organization of “video-in-interaction”: (a) mobile and Skype video calls are patterned, often alternating between a “talking heads” arrangement, in which both participants are on-screen and facing the camera, and moments in which they are producing various shots of their environment in line with their current interactional purposes; (b) openings occur almost always in the talking heads arrangement; (c) the video images on either side are produced and expected to be scrutinized with respect to their relevance to the ongoing interaction; (d) the talking heads arrangement is oriented to a default mode of interaction, with the implication that when there is nothing else relevant to show, the participants should show themselves on-screen; (e) in multiparty interactions, the party who is handling the video communication apparatus has an obligation to put other speakers on-screen when they talk, and the video callers orient to their appearance on-screen as making a distinctive participation status relevant. We show how these phenomena all derive from an orientation toward a single maxim, “put the face of the current speaker on-screen,” which plays a foundational role in the organization of video-in-interaction and its articulation with that of talk-in-interaction.
Article
Full-text available
Gesture, or visible bodily action that is seen as intimately involved in the activity of speaking, has long fascinated scholars and laymen alike. Written by a leading authority on the subject, this 2004 study provides a comprehensive treatment of gesture and its use in interaction, drawing on the analysis of everyday conversations to demonstrate its varied role in the construction of utterances. Adam Kendon accompanies his analyses with an extended discussion of the history of the study of gesture - a topic not dealt with in any previous publication - as well as exploring the relationship between gesture and sign language, and how the use of gesture varies according to cultural and language differences. Set to become the definitive account of the topic, Gesture will be invaluable to all those interested in human communication. Its publication marks a major development, both in semiotics and in the emerging field of gesture studies.
Article
Full-text available
Observed communication in 3 preverbal infants, ages 2, 6, and 12 mo. Detailed qualitative descriptions report nonverbal acts that correspond to the speech acts of perlocution, or affecting others without intention, and illocution, or the use of a convention to affect others. Locutions, requiring verbal reference to objects, began to replace nonverbal acts as the Ss grew older. The development and relations of verbal and nonverbal speech acts are discussed. (37 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
A System is Presented for coding and analyzing dialogue that involves preschool-age children. In the System, each participant is seen, throughout the course of a dialogue, to assume the roles of both initiator and responder. Each role is evaluated according to different scales. As initiator, a speaker's utterances are judged for their level of cognitive complexity and for their “summoning power” (i.e., for their explicitness of a demand for a response). As responder, a speaker's utterances and behaviors are judged for their appropriateness as responses to the speaker-initiator's preceding verbalizations. In assessing dialogue, the scales are interlinked so as to permit one to assess the extent and complexity of the verbal communication that is taking place between the two members of a dyad. The application of the scales is illustrated through the analysis of dialogues recorded between six mothers and their 3-year-old daughters.
Article
Full-text available
Using data from more than ten years of research, David McNeill shows that gestures do not simply form a part of what is said and meant but have an impact on thought itself. Hand and Mind persuasively argues that because gestures directly transfer mental images to visible forms, conveying ideas that language cannot always express, we must examine language and gesture together to unveil the operations of the mind.
Book
Discourse markers - the particles oh, well, now, then, you know and I mean, and the connectives so, because, and, but and or - perform important functions in conversation. Dr Schiffrin's approach is firmly interdisciplinary, within linguistics and sociology, and her rigourous analysis clearly demonstrates that neither the markers, nor the discourse within which they function, can be understood from one point of view alone, but only as an integration of structural, semantic, pragmatic, and social factors. The core of the book is a comparative analysis of markers within conversational discourse collected by Dr Schiffrin during sociolinguistic fieldwork. The study concludes that markers provide contextual coordinates which aid in the production and interpretation of coherent conversation at both local and global levels of organization. It raises a wide range of theoretical and methodological issues important to discourse analysis - including the relationship between meaning and use, the role of qualitative and quantitative analyses - and the insights it offers will be of particular value to readers confronting the very substantial problems presented by the search for a model of discourse which is based on what people actually say, mean, and do with words in everyday social interaction.
Book
A leading expert on evolution and communication presents an empirically based theory of the evolutionary origins of human communication that challenges the dominant Chomskian view. Human communication is grounded in fundamentally cooperative, even shared, intentions. In this original and provocative account of the evolutionary origins of human communication, Michael Tomasello connects the fundamentally cooperative structure of human communication (initially discovered by Paul Grice) to the especially cooperative structure of human (as opposed to other primate) social interaction. Tomasello argues that human cooperative communication rests on a psychological infrastructure of shared intentionality (joint attention, common ground), evolved originally for collaboration and culture more generally. The basic motives of the infrastructure are helping and sharing: humans communicate to request help, inform others of things helpfully, and share attitudes as a way of bonding within the cultural group. These cooperative motives each created different functional pressures for conventionalizing grammatical constructions. Requesting help in the immediate you-and-me and here-and-now, for example, required very little grammar, but informing and sharing required increasingly complex grammatical devices. Drawing on empirical research into gestural and vocal communication by great apes and human infants (much of it conducted by his own research team), Tomasello argues further that humans' cooperative communication emerged first in the natural gestures of pointing and pantomiming. Conventional communication, first gestural and then vocal, evolved only after humans already possessed these natural gestures and their shared intentionality infrastructure along with skills of cultural learning for creating and passing along jointly understood communicative conventions. Challenging the Chomskian view that linguistic knowledge is innate, Tomasello proposes instead that the most fundamental aspects of uniquely human communication are biological adaptations for cooperative social interaction in general and that the purely linguistic dimensions of human communication are cultural conventions and constructions created by and passed along within particular cultural groups. Bradford Books imprint
Chapter
Social actors often are involved in more than one higher-level action. A father watches his toddler and converses with a client on the phone. A business woman has her child playing in the office and engages in computer mediated interactions with colleagues. A student is sending an SMS at a bus stop while conversing in person with a fellow student. Examples of simultaneous engagements in higher-level actions are abundant, a way to study them, however, is still in its infancy. In this chapter, I would like to clarify the difference between what I call “modal density” and what I call “modal configuration.” While both notions are always at play in interaction, the theoretical concepts underlying these notions differ slightly. At the current state in multimodal research, modal density is often confused with modal configuration, whereby all too often the actual underlying concept of modal density is being ignored.
Chapter
Adult language comprises three interrelated systems, phonological, lexicogrammatical (vocabulary, morphology, syntax), and semantic. Language development studies in the 1960s focused mainly on the lexicogrammatical level; they were also predominantly psycholinguistic in their orientation. More recently, interest has extended into semantics; the present paper is concerned with the learning of meaning, and proposes a complementary approach in sociolinguistic terms. The paper suggests a socio-semantic interpretation of language development, based on the intensive study of one child, Nigel, from 9 months to 2½ years. Nigel first developed (Phase I) a two-level system, having sounds and meanings but no words or structures, in which the meanings derived from the elementary social functions of interaction with others, satisfaction of needs and the like. This continued to expand for 6–9 months, at which time the child entered the stage of transition to the adult language (Phase II, corresponding to what is generally taken as the starting point). This was characterized by the interpolation of a lexicogrammatical level between meaning and sound, and by the mastery of the principle of dialogue, the adoption and assignment of speech roles. It was also marked by a generalization of the initial set of social functions to form a basic opposition between “language as learning” and “language as doing.” The transition was considered complete when the child had effectively replaced his original two-level system by a three-level one and moved from monologue into dialogue; he then entered the adult system (Phase III). He could now build up the meaning potential of the adult language, and would continue to do so all his life. From a sociolinguistic point of view the major step consisted in once again reinterpreting the concept of “function” so that it became the organizing principle of the adult semantic system, being built into the heart of language in the form of the ideational (representational, referential, cognitive) and the interpersonal (expressive-conative, stylistic, social) components of meaning. All utterances in adult speech contain both these components, which are mapped on to each other by the structure-forming agency of the grammar. The original social functions survive in their concrete sense as types of situation and setting, the social contexts in which language serves in the transmission of culture to the child
Article
Chapter
This chapter introduces the primary notions of multimodal (inter)action analysis (MIA) and demonstrates the approach with an example. MIA was developed by Norris (Norris 2004, 2011) as a way to help her understand identity production during research with participants in Germany. Since then MIA has been useful in studies that involve social action ranging from marketing, to kite surfing, to vegetarianism. Researchers continually develop the methodology through use, and it has proven especially practical for studies seeking to describe human action that go beyond the verbal mode to include all the modes involved in social action. MIA draws upon the theoretical notions explicated by Scollon in mediated discourse analysis (MDA)(Scollon 1998, 2001). As with MDA, the notion of mediated action forms the central unit of analysis, and MIA adds further methodological tools to apply the theoretical principles of MDA to the analysis of social action. These developments include delineating social action into higher and lower levels, and introducing the concept of the frozen action. Norris (2004) also developed the concept of modal density to describe more clearly where higher-level actions are positioned on a foreground-background continuum of attention. These methodological tools are introduced below using an example from research into business coaching.
Article
Contemporary research investigating the phenomena of lifestyle sport has highlighted the centrality of space, spatiality and spatial appropriation. Lifestyle sports tend to manifest in liminal and/or unbounded spaces with practitioners drawing upon the affordances of the natural environment in new and unique ways. Simultaneously, practitioners employ continuously evolving technical tools (equipment) in the undertaking of these activities. This article articulates the ways in which the specialised equipment employed as mediational means affect the perception, interpretation and valuation of physical components of the natural and man-made environment. In this article, I introduce the notion of actionary pertinence and concept of locational element through the analysis of the ways in which general geographic areas (spaces) become actual kitesurfing locations (places) for specific social actors through mediated action and in direct connection to the mediational means through which action occurs. Drawing upon data generated through a year-long video ethnography, I argue that components of the physical environment become locational elements for specific social actors in and through mediated action. This explicitly extends theorisation regarding how mediational means affect and structure the nature of the action they mediate (Vygotsky 1978, 1987; Wertsch 1991, 1998), and extends this argument by exemplifying how mediational means also affect and structure the perception and interpretation of physical components of the environment which have a bearing on mediated action. Furthermore, I articulate the ways in which these locational elements are considered, read, re-read and interpreted in terms of their actionary pertinence. Thus, the materiality of locational elements manifests as salient and/or relevant through the bearing an element has on specific mediated actions and practices.
Article
Skype, a voice over Internet protocol (VoIP) system, allows people to communicate through text, audio, and visual exchanges across short or great distances. At less than a decade old, it hosts the interaction of as many as 30 million people at one time (Skype, 2011). Its services, some free and others for fee-based, are especially valuable to fragmented families. This is despite the limits of technology to truly allow for "being" and "keeping" in touch while apart. Skype may create the illusion of being together because it brings distant sounds and images closer. However, it also may simultaneously emphasize the distance between people. The screen that draws us together also may keep us apart. Through phenomenology, the researcher presents some facets of the Skype experience. Particular attention is given to how families encounter time and space using a technology designed to overcome distant time and distant space.
Article
This article discusses a small-scale study that explores how members of one family based in Australia and the United Kingdom use remote technology to develop and maintain family relationships across generations and distance. Of particular interest was the manner in which Skype computer software was mediated to develop intersubjectivity between a 4-year-old girl and her grandparents. Encounters were filmed, transcribed and coded; participants were interviewed and asked to keep reflective diaries. Analysis showed that all the adults scaffolded the child’s interactions to sustain communication and help her negotiate meaning with her grandparents as virtual partners, but that she also took on a leadership role by appropriating the affordances of the medium to incorporate them in her play in creative ways.
Article
Hand gestures and speech form a single integrated system of meaning during language comprehension, but is gesture processed with speech in a unique fashion? We had subjects watch multimodal videos that presented auditory (words) and visual (gestures and actions on objects) information. Half of the subjects related the audio information to a written prime presented before the video, and the other half related the visual information to the written prime. For half of the multimodal video stimuli, the audio and visual information contents were congruent, and for the other half, they were incongruent. For all subjects, stimuli in which the gestures and actions were incongruent with the speech produced more errors and longer response times than did stimuli that were congruent, but this effect was less prominent for speech-action stimuli than for speech-gesture stimuli. However, subjects focusing on visual targets were more accurate when processing actions than gestures. These results suggest that although actions may be easier to process than gestures, gestures may be more tightly tied to the processing of accompanying speech.
Chapter
This article is an introduction to the theoretical and methodological backgrounds of multimodal (inter)action theory. The aim of this theory is to explain the complexities of (inter)action, connecting micro- and macro levels of analysis, focusing on the social actor. The most important theoretical antecedent, mediated discourse analysis (see Scollon 1998, 2001b), is presented with its key concepts mediated action and modes. It is shown how action is used as the unit of analysis and how modes are understood in multimodal (inter)action analysis – as complex cultural tools, as systems of mediated action with rules and regularities and different levels of abstractness. Subsequently, methodological basics are introduced, such as lower-level, higher-level and frozen action; modal density, which specifies the attention/awareness of the social actor; and horizontal and vertical simultaneity of actions. Horizontal simultaneity can be plotted on the heuristic model of foreground-background continuum of attention/ awareness. Vertical simultaneity of actions comprises the central layer of discourse (immediate actions), the intermediate layer (long-term actions) and the outer layer (institutional or societal contexts). In short, it is sketched how multimodal (inter)action analysis aims to answer questions about the interconnection of the different modes on a theoretical as well as on a practical level.
Article
Moving towards multimodal mediated theory, I propose to define a mode as a system of mediated action that comes about through concrete lower-level actions that social actors take in the world. In order to explain exactly how a mode is a system of mediated action, I turn to a perfume blog and use one blog entry as my starting point. The mode that I primarily focus on in this article is the mode of smell, explicating that the mode of smell is not synonymous with olfactory perception, even though modal development of smell is certainly partially dependent upon olfactory perception. As I am ostensibly focusing on the one mode, I once again problematize this notion of countability and delineate the purely theoretical and heuristic unit of mode (Norris, 2004). I clarify that modes a) do not exist in the world as they are purely theoretical in nature; b) that modes can be delineated in various ways; and c) that modes are never singular. Even though the concept of mode is problematical – and in my view needs to always be problematized – I argue that the term and the notion of mode is theoretically useful as it allows us to talk about and better understand communication and (inter)action in three respects: 1. The notion of mode allows us to investigate regularities as residing on a continuum somewhere between the social actor(s) and the mediational means; 2. The theoretical notion of mode embraces socio-cultural and historical as well as individual characteristics, never prioritising any of these and always embracing the tension that exists between social actor(s) and mediational means; and 3. The theoretical notion of mode demonstrates that modal development through concrete lower-level actions taken in the world, is transferable to other lower-level actions taken.
Article
The abstract for this document is available on CSA Illumina.To view the Abstract, click the Abstract button above the document title.
Article
Interactional management is one domain which is receiving attention in the communicative competence literature (Wiemann 1977). A basic management skill is the collaborative production of coherent discourse. This paper describes the acquisition of coherence production skills by preschool children. The focus is largely upon formal coherence pro duction, the repetition of lexical or structural constituents from previous utterances.
Chapter
Multimodality is an innovative approach to representation, communication and interaction which looks beyond language to investigate the multitude of ways we communicate: through images, sound and music to gestures, body posture and the use of space. The Routledge Handbook of Multimodal Analysis is the first comprehensive ‘research tool kit’ for multimodal analysis, with 22 chapters written by leading figures in the field on a wide range of theoretical and methodological issues. It clarifies terms and concepts, synthesizes the key literature with in-depth exploration and illustrative analysis, and tackles challenging methodological issues. The Handbook includes chapters on key factors for Multimodality such as technology, culture, notions of identity and macro issues such as literacy policy. The handbook takes a broad look at multimodality and engages with how a variety of other theoretical approaches have looked at multimodal communication and representation, including visual studies, anthropology, conversation analysis, socio-cultural theory, socio-linguistics and new literacy studies. Detailed multimodal analysis case studies are also included, along with an extensive glossary of key terms, to support those new to multimodality and allow those already engaged in multimodal research to explore the fundamentals further. The Routledge Handbook of Multimodal Analysis is essential reading for undergraduate and postgraduate students as well as researchers involved in the study of multimodal communication.
Book
In this monograph, the author offers a new way of examining the much discussed notion of identity through the theoretical and methodological approach called multimodal interaction analysis. Moving beyond a traditional discourse analysis focus on spoken language, this book expands our understanding of identity construction by looking both at language and its intersection with such paralinguistic features as gesture, as well as how we use space in interaction. The author illustrates this new approach through an extended ethnographic study of two women living in Germany. Examples of their everyday interactions elucidate how multimodal interaction analysis can be used to extend our understanding of how identity is produced and negotiated in context from a more holistic point of view.
Book
Our perception of our everyday interactions is shaped by more than what is said. From coffee with friends to interviews, meetings with colleagues and conversations with strangers, we draw on both verbal and non-verbal behaviour to judge and consider our experiences. Analyzing Multimodal Interaction is a practical guide to understanding and investigating the multiple modes of communication, and provides an essential guide for those undertaking field work in a range of disciplines, including linguistics, sociology, education, anthropology and psychology. The book offers a clear methodology to help the reader carry out their own integrative analysis, equipping them with the tools they need to analyze a situation from different points of view. Drawing on research into conversational analysis and non-verbal behaviour such as body movement and gaze, it also considers the role of the material world in our interactions, exploring how we use space and objects - such as our furniture and clothes - to express ourselves. Considering a range of real examples, such as traffic police officers at work, doctor-patient meetings, teachers and students, and friends reading magazines together, the book offers lively demonstrations of multimodal discourse at work. Illustrated throughout and featuring a mini-glossary in each chapter, further reading, and advice on practical issues such as making transcriptions and video and audio recordings, this practical guide is an essential resource for anyone interested in the multiple modes of human interaction.
Article
Examined the performance of 15 normal children (mean age 14.9 mo) and 15 language-disabled (LD) children (mean age 24.9 mo) on pragmatic tasks designed for this study and on a standardized Piagetian measure of sensorimotor intelligence. Ss were tested in their homes on 3–5 occasions within a 10-day period. A language sample was obtained for each S to determine mean length of utterance, and the Bayley Scales of Infant Development, the experiment measures, and the Ordinal Scales of Psychological Development were administered. Results show significant differences between the performance of normal and LD Ss in many areas. The data clearly point to a significant difference between normal and LD children at the holophrastic stage to use their lexicon to communicate to a listener in a context. Moreover, LD Ss differed markedly from normal Ss in the organization of their declarative performatives. The data seem to point to a specific representational deficit in the LD Ss that affected the dynamic aspects of symbolization. The findings stand in contrast to the popular notion of a general representational deficit in LD children, and they underscore the importance of language as a tool in social interaction. (27 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
The arguments both for and against viewing the child's initial one-word utterances as HOLOPHRASES are reviewed. Some theoretical problems – concerning the innateness of language, the acquisition of syntax, the status of prosody and the child's comprehension of language during the one-word stage – with the holophrase controversy are pointed out. We suggest that an unresolvable theoretical stalemate exists because proponents on both sides of the controversy mistakenly assume the centrality of the notion sentence in discussing holophrases. An alternative view of early language development, which takes the SPEECH ACT as the basic unit of linguistic communication, is offered as a solution to the problems with the holophrase controversy as it now stands. We propose a more integrated description of the relations between functional and grammatical aspects of early communicative competence than is currently provided by sentence-oriented theories. In particular, we suggest that the child's PRAGMATIC INTENTIONS gradually become GRAMMATICALIZED as semantic and syntactic structures. Finally, three entities – communicative functions, referring expressions and predicating expressions – are proposed as LANGUAGE UNIVERSALS, as distinct from grammatical universals.
Article
An investigation of children's ability to convey and respond to requests for action was based on the spontaneous speech of 36 dyads of nursery school children (3; 6–5; 7). Direct request forms (e.g. Give me the hammer) were frequent and the majority were acknowledged verbally. Examination of the contexts of direct requests indicated that speaker and addressee shared an understanding of the interpersonal meaning factors relevant to requesting. These meaning factors were invoked in justifying, refusing and in repeating or paraphrasing a request, and they also provided a basis for the communication of indirect requests. Examples of inferred requests are discussed, and a relationship between the structure of the speech act and conversational sequences is proposed.
Article
A speech act approach to the transition from pre-linguistic to linguistic communication is adopted in order to consider language in relation to behaviour generally and to allow for an emphasis on the USE of language rather than on its form. The structure of language is seen as non-arbitrary in that it reflects both attention structures (via predication) and action structures (via the fundamental case grammatical form of language). Linguistic concepts are first realized in action. A pilot study focusing on the regulation of JOINT attention and JOINT activity within the context of mutuality between mother and infant is discussed, with emphasis on ritualization in mutual play as a vehicle for understanding the development of the formal structures of language.
Article
Observations of early morning conversations between twin boys (2;9 at onset of research) indicate that young children are able to sustain a coherent dialogue over a number of turns. Contrary to the views of Piaget (1926), the interlocutors generally attend to one another's utterances. Coherence is achieved to a large extent by attending to the form of one another's utterances. Extended exchanges are maintained by speakers focusing on a sequence of sounds (sound play) or a constituent within an antecedent turn and reproducing it (with or without modification) in the next turn.
Article
This article presents a study of a set of pointing gestures produced together with speech in a corpus of video-recorded “locality description” interviews in rural Laos. In a restricted set of the observed gestures (we did not consider gestures with special hand shapes, gestures with arc/tracing motion, or gestures directed at referents within physical reach), two basic formal types of pointing gesture are observed: B-points (large movement, full arm, eye gaze often aligned) and S-points (small movement, hand only, casual articulation). Taking the approach that speech and gesture are structurally integrated in composite utterances, we observe that these types of pointing gesture have distinct pragmatic functions at the utterance level. One type of gesture (usually “big” in form) carries primary, informationally foregrounded information (for saying “where” or “which one”). Infants perform this type of gesture long before they can talk. The second type of gesture (usually “small” in form) carries secondary, informationally backgrounded information which responds to a possible but uncertain lack of referential common ground. We propose that the packaging of the extra locational information into a casual gesture is a way of adding extra information to an utterance without it being on-record that the added information was necessary. This is motivated by the conflict between two general imperatives of communication in social interaction: a social-affiliational imperative not to provide more information than necessary (“Don’t over-tell”), and an informational imperative not to provide less information than necessary (“Don’t under-tell”).
Conference Paper
We interviewed and observed families in their homes to understand how they communicate across generations and across distances. The phone is still the most common way for keeping children in touch with distant relatives. However, many children can't talk on the phone by themselves until 7 or 8 years old. This paper examines the challenges children have with phone conversations, and looks at how families are currently working around these issues. These findings can help inform the design of future family communications technologies.
Article
⇒ NOT written, but spoken language. (Intuitions come from written.) ⇒ NOT meaning as thing, but use of linguistic forms for communicative functions o Direct att. in shared conceptual space - like gestures (but w/conventions) ⇒ NOT grammatical rules, but patterns of use => schemas o Constructions themselves as complex symbols "She sneezed him the ball" o NOT 'a grammar' but a structured inventory of constructions: continuum of regularity => idiomaticity Œ grammaticality = normativity • Many complexities = "unification" of constructions w/ incompatibilities o NOT innate UG, but "teeming modularity" (1) symbols, pred-arg structure, social intentions/speech acts, speech/phonology, categorization, etc. (2) diff. functions • not many language universals, but some due to universals of: human cognition, social cognition/attention, vocal-auditory processing.
Article
Wallace Chafe demonstrates how the study of language and consciousness together can provide an unexpectedly broad understanding of the way the mind works. Relying on close analyses of conversational speech as well as written fiction and nonfiction, he investigates both the flow of ideas through consciousness and the displacement of consciousness by way of memory and imagination. Chafe draws on several decades of research to demonstrate that understanding the nature of consciousness is essential to understanding many linguistic phenomena, such as pronouns, tense, clause structure, and intonation, as well as stylistic usages, such as the historical present and the free indirect style. While the book focuses on English, there are also discussions of the North American Indian language Seneca and the music of Mozart and of the Seneca people. This work offers a comprehensive picture of the dynamic natures of language and consciousness that will interest linguists, psychologists, literary scholars, computer scientists, anthropologists, and philosophers.