PosterPDF Available

Can you draw me a question?

August 2019

DOI:10.13140/RG.2.2.15700.14729

Conference: ICPhS 2019
At: Melbourne, Australia
Affiliation: University of Cologne

Authors:

University of Cologne

University of Cologne

Poster for the 2nd Prosody Visualisation Challenge (PVC2)

Content uploaded by Martine Grice

Content may be subject to copyright.

Content uploaded by Aviad Albert

Content may be subject to copyright.

F0 (Hz)

Periodic energy

F0 (Hz)

Periodic energy

Can you draw me a question?

F0 (Hz)

Periodic energy

Periograms:Modulating F0 trajectories with

corresponding periodic energy to reflect gradual

strength of pitch perception [1,2]

Sources: (main+image) Morning Joe, MSNBC 2014; (top) Glenn Beck,

Fox News 2009; (bottom) Morning Joe, MSNBC 2013.

Projecting controversy

Bird pattern

: global F0 fall + global rise

Sources: (main+image) Washington Journal, CSPAN 2015; (top) Right This

Minute, Fox KTVU 2015; (bottom) San Francisco Government TV, 2012.

Polite preliminary (request)

Global rising F0

Aviad Albert, Francesco Cangemi and Martine Grice (IƒL-Phonetik, University of Cologne)

Floor taking (attention grab)

Initial local F0 rise + global fall

Sources: (main+image) Tucker Carlson Tonight, Fox News 2019; (top)

Morning Joe, MSNBC 2012; (bottom) Morning Joe, MSNBC 2009.

Motivation:Conventional fixed expressions

and catchphrases rely on prosody to communicate

systematic functionality, as well as uniqueness

Conclusions:By emphasising the relation between F0 and

periodic energy, periograms can intuitively display different

phonetic realisations of the same phrase in different interactional

settings, while accounting for pervasive syllabic reduction in

naturally occurring speech [6,7].

Periograms of spontaneous speech indicate that tonal events

might not have to be associated to canonical syllables as docking

sites, paving the way for an intonational phonology of

spontaneous speech [8,9].

References: [1] Deshmukh, O., C. Y. Espy-Wilson, A. Salomon & J. Singh.2005.Use of

temporal information:Detection of periodicity, aperiodicity, and pitch in speech.IEEE Transa cti on s on

Speech and Audio Processing 13 (5): 776-786. [2] Albert, A., F. Cangemi, M. Grice.2018.Using

periodic energy to enrich acoustic representations of pitch in speech: A demonstration.Proceedings

of the 9th Speech Prosody Conference,June 2018,Poznan. [3] Schegloff,E.A.1980.Preliminaries to

preliminaries:‘Can Iask you aquestion?’.Sociological Inquiry 50:104–152. [4] TV News Archive (2012).

https://archive.org/details/tv,accessed July 2019. [5] Winter, B., M. Perlman & T. Matlock.2013.Using

space to talk and gesture about numbers:Evidence from the TV News Archive.Gesture 13(3): 377-408.

[6] Ernestus, M., N. Warner (eds.) 2011.Speech Reduction.Journal of Phonetics 39(3): 253-452. [7]

Cangemi, F., M. Clayards,O.Niebuhr, B. Schuppler,M.Zellers (Eds.) 2018.Rethinking reduction:

Interdisciplinary perspectives on conditions, mechanisms, and domains for phonetic variation.Berlin:

De Gruyter Mouton. [8] Cangemi, F., A. Albert, M. Grice.2019.Modelling intonation:Beyond

segments and tonal targets.Proceedings of the 19th International Congress of Phonetic Sciences,

August 2019,Melbourne. [9] Ward,N.G.2019.Prosodic Patterns in English Conversation.Cambridge:

Cambridge University Press.

Data: Naturally occurring ‘can I ask you a question?’ [3] from the TV News Archive [4,5] and as a unique catchphrase

Source: Alaska's Can I Asssk You A Question -

RuPaul's Drag Race Season 8 Grand Finale, World of

Wonder 2016.

Catchphrase

Trademark rendition, using

prosody to achieve uniqueness

F0 (Hz)

Periodic energy

Prosody Visualisation Challenge 2 (ICPhS 2019, Melbourne Australia)

Segmental Influences on the Perception of High Pitch Accent Scaling in American English

Article

Aug 2024
LANG SPEECH

Researchers investigating a broad array of questions in spoken language prosody routinely base their arguments on measurements taken from the F0 contours of representative speech samples. These analyses, however, frequently involve abstracting F0 contours away from the segmental strings that bear them, potentially overlooking in the process the role played by segmental qualities such as sonority or periodicity in the realization of F0 patterns by speakers and their interpretation by listeners. This paper reports the results of two experiments investigating how perception of F0 contours is affected by the segmental string over which those contours are realized. The first focuses on gaps in F0 contours created by voiceless obstruents such as stops and fricatives, while the second investigates F0 intervals spanning lower-sonority voiced segments, such as nasals and voiced fricatives. While these two scenarios might at first seem unrelated, we argue that listeners treat both with a single mechanism in perception, namely, by reducing (potentially to zero) the amount of weight accorded to those portions of the contour for determination of the speaker’s intended F0 scaling level. We present an account of both effects within a unified model of F0 scaling perception called TCoG-F, with discussion of its implications for phonetic and phonological intonation research going forward.

Using space to talk and gesture about numbers: Evidence from the TV News Archive

Article

Full-text available

Jul 2014

This paper examines naturally occurring gestures produced in descriptions of numbers and quantities in television newscasts. The results of our analysis show that gestures reveal the metaphorical and spatial nature of numerical thinking. That is, speakers’ hands mimic known spatial mappings between space and quantity, including horizontal mappings (smaller quantities left, larger quantities right), vertical mappings (smaller quantities down, larger quantities up) and size-based mappings (smaller quantities “small”, larger quantities “large”). Speakers frequently switch between these different spatial mappings, and they sometimes combine them within the same gesture. This points to the flexibility of how metaphors can become expressed in gesture, and how domains such as number and quantity can be conceptualized through multiple compatible source domains.

Rethinking reduction: Interdisciplinary perspectives on conditions, mechanisms, and domains for phonetic variation

Jan 2018

F Cangemi
M Clayards
O Niebuhr
B Cangemi
A Albert
M Grice

Cangemi, F., M. Clayards, O. Niebuhr, B. Schuppler, M. Zellers (Eds.) 2018. Rethinking reduction: Interdisciplinary perspectives on conditions, mechanisms, and domains for phonetic variation. Berlin: De Gruyter Mouton. [8] Cangemi, F., A. Albert, M. Grice. 2019. Modelling intonation: Beyond segments and tonal targets. Proceedings of the 19th International Congress of Phonetic Sciences, August 2019, Melbourne. [9] Ward, N. G. 2019. Prosodic Patterns in English Conversation. Cambridge: Cambridge University Press.

Conference Paper

A Layout Technique for Storyline-based Visualization of Consecutive Numerical Time-varying Data

August 2015

We commonly represent time-varying values as polyline charts or heatmaps; however, both type of techniques are difficult to simultaneously observe short-term features of time-varying values and cluster transitions. This poster proposes storyline-based visualization technique for consecutive numerical time-varying data. Storyline is a visualization technique to show associative feature among ... [Show full abstract] elements over time. Our technique measures similarity of each elements and draw similar elements as proximity storyline. The technique also reflects differential values on storyline as a visual variable to emphasize the amount of line changes.

Conference Paper

DSAsketch: Data structures and algorithms drawing game

November 2012

Lasse Hakulinen

Visualizations are often used to teach abstract concepts like algorithms and data structures. In this poster, we present a drawing game, DSAsketch, that can be used to practice a wide range of concepts related to data structures and algorithms. During the game, the players are viewing, constructing and presenting their own visualizations. Thus, players are required to engage with the ... [Show full abstract] visualizations in many different ways.

Conference Paper

Full-text available

Using periodic energy to enrich acoustic representations of pitch in speech: A demonstration

June 2018

This paper aims to strengthen the link between acoustic and perceptual representations of intonation, a link that has been weakened by the over-reliance on the F0 trajectory, which can only be interpreted in relation to landmarks in the segmental string, placed manually or semi-automatically at a separate stage in the analysis. Only then can F0 events be identified as linguistically relevant ... [Show full abstract] (e.g. early, medial or late peaks, accentual tones or edge tones etc.). We provide an analysis and visualization of two acoustic dimensions contributing towards the perceived pitch contour, F0 over time and, crucially, periodic energy. Periodic energy reflects the degree to which pitch is intelligible, a higher value representing a stronger F0 signal that is consequently more easily perceived. A representation of F0 that includes periodic energy is thus able to flag portions of the speech signal that are relevant for the analysis of intonation, without the need for a separate segmentation of the signal into phones and syllables. Index Terms: intonation, pitch perception, periodic energy, tonal alignment, segmentation, data visualization, sonority

Conference Paper

Full-text available

Modelling intonation: Beyond segments and tonal targets

August 2019

A widespread approach to research on intonation categories involves extracting relevant landmarks in the tune (e.g. F0 turning points) and relating them to relevant landmarks in the text (e.g. phone boundaries). This approach is problematic, both in theoretical terms (e.g. no consensus as to what is a relevant landmark in either domain) and in practical terms (e.g. locating segments and F0 ... [Show full abstract] turning points with either error prone automatic annotations or time-consuming manual annotations). We propose and test an alternative approach that overcomes both types of problem. The tune is modelled taking into account the shape of F0 contours, and the text is modelled by means of periodic energy curves. We exploit the interaction between these two dynamic trajectories to observe and quantify intonation. Compared to the standard approach, our method requires less manual work and makes fewer theoretical assumptions, but nonetheless has similar descriptive power.

Preprint

Full-text available

Rhythm is a timescale

March 2024

Many studies in the linguistic literature have tried to explain the rhythmic component of speech by resorting to the notion of isochrony. The problems with such approaches have been demonstrated in various recent works, owing to the fact that natural speech is highly irregular and quasi-periodic at best. Rhythm also plays a role in the link between brain oscillations and linguistic constituents, ... [Show full abstract] where entrainment is often assumed to be the underlying mechanism. Here too, the non-isochronous nature of the speech signal led recent works to call for a more nuanced understanding of entrainment in the context of language. We suggest that rhythm is the timescale within which temporal relationships between isolated events are perceived (about 0.5–12 Hz). Moreover, we argue that isochrony is not what defines rhythm – it is one of the goals that rhythm can achieve. In music, isochrony often characterizes the signal shared between listeners, to promote phase-locking to external sources. In speech, however, isochrony needs to be sourced from internal time-keeping mechanisms, since it is likely the baseline measurement for mental representations, allowing us to perceive the dynamic timing patterns in speech rate, which, in turn, enrich and facilitate comprehension via prosody.

Conference Paper

Full-text available

PraaPer: Simple, Rich, and Intuitive Representations for Tone and Intonation

November 2023