Irene Mittelberg

Irene Mittelberg
RWTH Aachen University · Institute of English Studies & Center for Sign Language and Gesture

PhD & MA in Linguistics & Cognitive Science Cornell University & MA, French Linguistics & Art History, Hamburg University

About

59
Publications
18,723
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
725
Citations
Introduction
Skills and Expertise
Additional affiliations
April 2009 - May 2016
RWTH Aachen University
Position
  • Professor of Linguistics and Cognitive Semiotics
April 2009 - May 2016
RWTH Aachen University
Position
  • Professor (Associate)

Publications

Publications (59)
Preprint
Full-text available
CHAPTER TO APPEAR (2022, in press) in Jamin Pelkey & Paul Cobley (eds), Semiotic Movements (Bloomsbury Semiotics 4). London: Bloomsbury Academic. As semiotic beings, we build our understanding of the discourses we participate in, and more generally of the world around us, by integrating signs in different modalities into dynamic meaningful ensemb...
Article
Cognitive bias in depression may increase sensitivity to judgmental appraisal of communicative cues. Nonverbal communication encompassing co-speech gestures is crucial for social functioning and is perceived differentially by men and women, however, little is known about the effect of depression on the perception of appraisal. We investigate if a c...
Conference Paper
Generating natural embodied conversational agents within virtual spaces crucially depends on speech sounds and their directionality. In this work, we simulated directional filters to not only add directionality, but also directionally adapt each phoneme. We therefore mimic reality where changing mouth shapes have an influence on the directional pro...
Poster
During naturalistic production, sign language users regulate their hand velocity, slowing down for lexical productions compared to transitions. We asked signers and nonsigners to produce grooming gestures and pseudosigns (i.e., nonexistent but permissible DGS signs) in tandem with videos and measured hand velocity for item production versus transit...
Article
Full-text available
This paper presents an account of how Peirce’s Universal Categories (UCs) of perception and experience may, as heuristic principles, inform gesture theory and multimodal analysis. Peirce’s UCs – Firstness (possibility), Secondness (actuality), and Thirdness (law, habit) – constitute the core of his phenomenology and thus also the foundation of his...
Article
Full-text available
This paper aims to evidence the inherently metonymic nature of co-speech gestures. Arguing that motivation in gesture involves iconicity (similarity), indexicality (contiguity), and habit (conventionality) to varying degrees, it demonstrates how a set of metonymic principles may lend a certain systematicity to experientially grounded processes of g...
Article
Full-text available
Social interactions arise from patterns of communicative signs, whose perception and interpretation require a multitude of cognitive functions. The semiotic framework of Peirce’s Universal Categories (UCs) laid ground for a novel cognitive-semiotic typology of social interactions. During functional magnetic resonance imaging (fMRI), 16 volunteers w...
Presentation
Full-text available
Presentation on how interlocutors modify their head movement, such as nods and shakes in response to their dialogue partner’s utterances. Employing analysis of motion-capture data we look for for timing, intensity and frequency of head movements to differentiate phases of discourse.
Article
Full-text available
Embodied image schemas are central to experientialist accounts of meaning-making. Research from several disciplines has evidenced their pervasiveness in motivating form and meaning in both literal and figurative expressions across diverse semiotic systems and art forms (e.g., Gibbs and Colston; Hampe; Johnson; Lakoff; and Mandler). This paper aims...
Chapter
Redebegleitende Gesten sind spontane Bewegungen, insbesondere von Armen und Händen, die das Sprechen begleiten und auf verschiedenen sprachlichen Ebenen eng mit ihm verbunden sein können. Der Anthropologe Adam Kendon (1980: 207) nennt diese Art von Bewegungen gesticulation. Nach Kendon handelt es ich dabei um zwei Manifestationen desselben zugrunde...
Article
Full-text available
This paper lays out the foundations of a frame-based account of gesture pragmatics through detailing how frames and metonymy interact not only in motivating gestural sign formation, but also in guiding crossmodal processes of pragmatic inferencing. It is argued that gestures recruiting frame structures tend to profile deeply embodied, routinized as...
Article
In "Two heads are better than one," "head" stands for people and focuses the message on the intelligence of people. This is an example of figurative language through metonymy, where substituting a whole entity by one of its parts focuses attention on a specific aspect of the entity. Whereas metaphors, another figurative language device, are substit...
Article
Full-text available
Face-to-face communication is multimodal; it encompasses spoken words, facial expressions, gaze, and co-speech gestures. In contrast to linguistic symbols (e.g., spoken words or signs in sign language) relying on mostly explicit conventions, gestures vary in their degree of conventionality. Bodily signs may have a general accepted or conventionaliz...
Chapter
In light of the growing importance of digital design tools, this paper puts back into focus the human body’s internalized knowledge of the material and social world. It exploits the semiotic potential of operative manufacts, understood as encompassing both sketches on paper and gestures evolving in real time and space. Operative manufacts are seen...
Article
Full-text available
Taking an Emergent Grammar (Hopper 1998) approach to multimodal usage events in face-to-face interaction, this paper suggests that basic scenes of experience tend to motivate entrenched patterns in both language and gesture (Fillmore 1977; Goldberg 1998; Langacker 1987). Manual actions and interactions with the material and social world, such as gi...
Article
In language switching, it is assumed that in order to produce a response in one language, the other language must be inhibited. In unimodal (spoken-spoken) language switching, the fact that the languages share the same primary output channel (the mouth) means that only one language can be produced at a time. In bimodal (spoken-signed) language swit...
Article
Full-text available
Viewpoint has been shown to be a powerful construal mechanism in multimodal spoken and signed discourse, as well as in various other modalities and genres. This paper investigates embodied viewpoint strategies that have been observed when speakers combine speech, gestures, postures, gaze, and simulated action to describe their interaction with spat...
Article
Full-text available
One of the most fundamental challenges when accessing gestural patterns in 3D motion capture databases is the definition of spatiotemporal similarity. While distance-based similarity models such as the Gesture Matching Distance on gesture signatures are able to leverage the spatial and temporal characteristics of gestural patterns, their applicabil...
Conference Paper
Full-text available
The question of how to model spatiotemporal similarity between gestures arising in 3D motion capture data streams is of major significance in currently ongoing research in the domain of human communication. While qualitative perceptual analyses of co-speech gestures, which are manual gestures emerging spontaneously and unconsciously during face-to-...
Chapter
Full-text available
The construct of iconic gestures, those gestures understood as sharing certain form features with the object, action or scene they represent, has traditionally proven to be a useful tool for scholars to classify this subset of gestures, distinguishing them from other types such as indexical or emblematic gestures. More recent approaches prefer to a...
Conference Paper
Full-text available
The approach presented in this paper aims to contribute to an account of the three-dimensionality of gesture space. Here, gesture space is assumed to be dynamically constructed and adaptive to the communicative situation. Making use of an optical motion-capture system, volumetric representations of gesture spaces were generated, based on gesture da...
Conference Paper
Full-text available
This paper explores if people perceive and perform touchless gestures differently when communicating with technology vs. with humans. Qualitative reports from a lab study of 10 participants revealed that people perceive differences in the speed of performing gestures, sense of enjoyment, feedback from the communication target. Preliminary analysis...
Chapter
Hand, Hirn und Werkzeug - eine komplexe Interaktion. Dieser aus der Architekturpraxis initiierte Band untersucht aus der Perspektive verschiedener Wissensdisziplinen, inwiefern die manuelle Intelligenz des Menschen kreative Prozesse befördert. Besteht in Zeiten programmierter Umgebungen ein Widerspruch zwischen instrumentellem Handeln und intuitive...
Article
Full-text available
This paper explores how na?ve observers recognize and interpret transitive actions (actions involving manipulation of objects) without accompanying speech, in order to derive guidelines for the design of gesture interpretation systems. Semi-structured interviews with 11 observers, interpreting 106 video clips of transitive actions elicited unstaged...
Conference Paper
Full-text available
This paper builds on a strand of gesture research that regards form as the basis of any meaningful interpretation of gesture. Gesture form is considered to be derived from body part (ar-ticulator) configurations and movement. We propose that gesture form is multi-layered and can be described economically by form primitives (zero to three dimensions...
Article
Full-text available
The importance of bodily movements in the production and perception of communicative actions has been shown for the spoken language modality and accounted for by a theory of communicative actions (Cogn. Process. 2010;11:187–205). In this study, the theory of communicative actions was adapted to the sign language modality; we tested the hypothesis t...
Conference Paper
Full-text available
This paper explores how interaction with systems using touchless gestures can be made intuitive and natural. Analysis of 912 video clips of gesture production from a user study of 16 subjects communicating transitive actions (manipulation of objects with or without external tools) indicated that 1) dynamic pantomimic gestures where imagined tool/ob...
Chapter
Based on spoken academic discourse and its accompanying gestures, this chapter presents a cognitive-semiotic approach to multimodal communication that assigns equal importance to metaphor and metonymy. Combining traditional semiotics with contemporary cognitivist theories, we demonstrate how these two figures of thought jointly structure multimodal...
Article
Full-text available
Based on spoken American academic discourse and its accompanying gestures videotaped during linguistics courses, this paper presents a cognitive-semiotic approach to multimodal communication that assigns equal importance to metaphor and metonymy. Combining traditional semiotics (Peirce [1960] and Jakobson [1956, 1961, 1963]) with contemporary cogni...
Book
Full-text available
Methods in Cognitive Linguistics is an introduction to empirical methodology for language researchers. Intended as a handbook to exploring the empirical dimension of the theoretical questions raised by Cognitive Linguistics, the volume presents guidelines for employing methods from a variety of intersecting disciplines, laying out different ways of...
Chapter
Full-text available
Article
Full-text available
This study represents an interdisciplinary attempt to trace the origins of metaphorical expressions as they are found in English grammatical terminology. The perspective chosen combines cognitive and art-historical approaches to the meaning of abstract concepts: cognitive semantics and iconography. It takes into account the historical dimension of...
Article
Full-text available
System und Rede Die Gestenforschung ist ein vergleichsweise junges Feld, das sich vor allem in und zwischen den Disziplinen Linguistik, Kognitive Psychologie, Anthropologie und Semiotik bewegt. Geht man von einem primär linguistischen Ansatz aus, dann ist das sprachlich Konkrete eine Äußerung, die raumzeitlich situiert und intersubjektiv wahrnehmba...
Article
Full-text available
Thesis (Ph.D.)--Cornell University, Jan., 2006. Includes bibliographical references.

Network

Cited By