Elliot Saltzman

• Boston University

Distributional semantic models (DSMs) are a primary method for distilling semantic information from corpora. However, a key question remains: What types of semantic relations among words do DSMs detect? Prior work typically has addressed this question using limited human data that are restricted to semantic similarity and/or general semantic relate...

Social animals have the remarkable ability to organize into collectives to achieve goals unobtainable to individual members. Equally striking is the observation that despite differences in perceptual-motor capabilities, different animals often exhibit qualitatively similar collective states of organization and coordination. Such qualitative similar...

Many everyday tasks require individuals to work together as a team to achieve a task goal. For many complex or high-stakes multi-agent activities, team members are required to participate in simulated training exercises to develop the task- and team-work (coordination) skills needed to maximize task performance. Such training, however, can be both...

No PDF available ABSTRACT Speech units under prominence present longer, larger, and faster constriction gestures than their non-prominent counterparts. However, whether there are discrete degrees of prominence, and if so how many, has yet to be discovered, partly because the contribution of the information structure in marking prominence is unclear...

The shepherding problem is interesting for multiagent systems research as it requires multiple actors (e.g., dogs, humans) to exert indirect control over autonomous agents (e.g., sheep, cattle) for containment or transportation. Accordingly, plenty of research has focused on designing algorithms for robotic agents to solve such tasks. Almost no res...

Multiagent activity is commonplace in everyday life and can improve the behavioral efficiency of task performance and learning. Thus, augmenting social contexts with the use of interactive virtual and robotic agents is of great interest across health, sport, and industry domains. However, the effectiveness of human–machine interaction (HMI) to effe...

Human speech perception involves transforming a countinous acoustic signal into discrete linguistically meaningful units, such as phonemes, while simultaneously causing a listener to activate words that are similar to the spoken utterance and to each other. The Neighborhood Activation Model (NAM~\cite{Luce:1986,Luce:1998}) posits that phonological...

Human speech perception involves transforming a countinous acoustic signal into discrete linguistically meaningful units, such as phonemes, while simultaneously causing a listener to activate words that are similar to the spoken utterance and to each other. The Neighborhood Activation Model (NAM~\cite{Luce:1986,Luce:1998}) posits that phonological...

Biological systems are capable of acting in a shared environment to produce emergent, self-organized behavior that is the result of the constraints imposed by local interactions– such as bird flocking or ant swarming behavior. These examples present minimal demands for a shared-intention between co-actors, whereas other instances necessitate the fo...

Studies have shown that articulatory information helps model speech variability and, consequently, improves speech recognition performance. But learning speaker-invariant articulatory models is challenging, as speaker-specific signatures in both the articulatory and acoustic space increase complexity of speech-to-articulatory mapping, which is alre...

Speech production is a highly skilled sensorimotor activity defined by articulatory or acoustic coordinates. To compare the variabilities of those two conceptualizations, issues of dimension reduction, normalization, incompleteness of information, etc., need to be taken into account. Uncontrolled manifold (UCM) method analyzes high-dimensional move...

Effectively coordinating one's behaviors with those of others is essential for successful multiagent activity. In recent years, increased attention has been given to understanding the dynamical principles that underlie such coordination because of a growing interest in behavioral synchrony and complex-systems phenomena. Here, we examined the behavi...

Introduction: Task-specific focal dystonia (TSFD) is a disorder marked by degraded coordination in complex and exacting psychomotor tasks, such as musical performance. Its development is associated with prolonged and intensive practice of these tasks, but the etiology of TSFD is still unknown. The prevailing hypothesis was informed by findings in...

Meaningful parallels exist between the coordinative structures (CSs) involved in planning and controlling complex actions of the body (limbs, torso, head, face, eyes) in service of both mechanical (e.g., manipulation and/or transport of environmental objects) and social (e.g., inter-agent coordination) functions. While early treatments focused on C...

BACKGROUND: Walking parameters are important functional health indicators. However most gait research has been conducted in laboratories/clinical settings involving straight, 10-20 meter, protocol-dictated trials. Methods are needed to examine walking parameters in the context of daily life activities. OBJECTIVE: To investigate a new activity moni...

Veering while walking is often reported in individuals with Parkinson's disease (PD), with potential mechanisms being vision-based (asymmetrical perception of the visual environment) or motoric (asymmetry in stride length between relatively affected and non-affected body side). We examined these competing hypotheses by assessing veering in 13 norma...

The current project examined how changes to task constraints impacted the behavioral dynamics of an interpersonal collision avoidance task previously examined and modeled by Richardson and colleagues (2015). Overall, the results demonstrate that decreasing the cost associated with colliding influences the stability and symmetry of the movement dyna...

An investigation into the consistency of locomotive activity across days in the same week in the real-lives of older adults .

Development of biologically inspired exoskeletons to assist soldiers in carrying load is a rapidly expanding field. Understanding how the body modulates stiffness in response to changing loads may inform the development of these exoskeletons and is the purpose of the present study. Seventeen subjects walked on a treadmill at a constant preferred wa...

In articulatory phonetics, a phoneme’s identity is specified by its articulator-free (manner) and articulator-bound (place) features. Previous studies have shown that acoustic-phonetic features (APs) can be used to segment speech into broad classes determined by the manner of articulation of speech sounds; compared to MFCCs, however, APs perform po...

Articulatory features (AFs) are known to provide an invariant representation of speech, which is expected to be robust against channel and noise degradations. This work presents a deep neural network (DNN)—hidden Markov model (HMM) based acoustic model where articulatory features are used in addition to mel-frequency cepstral coefficients (MFCC) fo...

Understanding stable patterns of interpersonal movement coordination is essential to understanding successful social interaction and activity (i.e., joint action). Previous research investigating such coordination has primarily focused on the synchronization of simple rhythmic movements (e.g., finger/forearm oscillations or pendulum swinging). Very...

Speech inversion is a technique to estimate vocal tract configurations from speech acoustics. We constructed two such systems using feedforward neural networks. One was trained using natural speech data from the XRMB database and the second using synthetic data generated by the Haskins Laboratories TADA model that approximated the XRMB data. XRMB p...

The focus of this paper is on characterizing the physical movement forms (e.g., walk, crawl, roll, etc.) that can be used to actualize abstract, functionally-specified behavioral goals (e.g., locomotion). Emphasis is placed on how such forms are distinguished from one another, in part, by the set of topological patterns of physical contact between...

This paper presents a deep neural network (DNN) to extract articulatory information from the speech signal and explores different ways to use such information in a continuous speech recognition task. The DNN was trained to estimate articulatory trajectories from input speech, where the training data is a corpus of synthetic English words generated...

Troughs, defined as a discontinuity in anticipatory coarticulation (Perkell 1968), have been shown to occur in the tongue body (TB) in /ipi/ and lips in /usu/. In V1CV1 contexts articulators retract from their vowel target during C when not active during C. We investigate the trough phenomenon by testing three hypotheses: (1) high intraoral air pre...

Visuoperceptual disorders have been identified in individuals with Parkinson's disease (PD) and may affect the perception of optic flow for heading direction during navigation. Studies in healthy subjects have confirmed that heading direction can be determined by equalizing the optic flow speed (OS) between visual fields. The present study investig...

Coarticulation and invariance are two topics at the center of theorizing about speech production and speech perception. In this paper, a quantitative scale is proposed that places coarticulation and invariance at the two ends of the scale. This scale is based on physical information flow in the articulatory signal, and uses Information Theory, espe...

Interactivity is a central theme of ecological psychology. According to Gibsonian views, behavior is the emergent property of interactions between organism and environment. Hence, an important challenge for ecological psychology has been to identify physical principles that provide an empirical window into interactivity. We suspect that multifracta...

Speech can be represented as a constellation of constricting vocal tract actions called gestures, whose temporal patterning with respect to one another is expressed in a gestural score. Current speech datasets do not come with gestural annotation and no formal gestural annotation procedure exists at present. This paper describes an iterative analys...

Wearable assistive robotic devices are characterized by an interface, a meeting place of living tissue and mechanical forces, at which potential and kinetic energy are converted to one or the other form. Ecological scientists may make important contributions to the design of device interfaces because of a functional perspective on energy and inform...

The biomechanical mechanisms that link foot structure to injuries of the musculoskeletal system during gait are not well understood. This study had two parts. The purpose of part one was to determine the relation between clinical rearfoot and forefoot angles and foot angles as they make contact with the ground. The purpose of part two was to determ...

Previous research has considered infant spontaneous kicking as a form of exploration. According to this view, spontaneous kicking provides information about motor degrees of freedom and may shape multijoint coordinations for more complex movement patterns such as gait. Recent work has demonstrated that multifractal, multiplicative fluctuations in e...

Syllable complexity has been found to affect the time the speaker needs for planning and initiating utterance production. Shorter latencies for complex onsets (CCV) as compared to simple onsets (CV) have been explained by effects of segment-specific biomechanical constraints at the level of motor execution, and by neighborhood density at the planni...

This study compares the time to initiate words with varying syllable structures (V, VC, CV, CVC, CCV, CCVC). In order to test the hypothesis that different syllable structures require different amounts of time to prepare their temporal controls, or plans, two delayed naming experiments were carried out. In the first of these the initiation time was...

This study examined the effect of listening to a newly learned musical piece on subsequent motor retention of the piece. Thirty-six non-musicians were trained to play an unfamiliar melody on a piano keyboard. Next, they were randomly assigned to participate in three follow-up listening sessions over 1 week. Subjects who, during their listening sess...

Studies have shown that supplementary articulatory information can help to improve the recognition rate of automatic speech recognition systems. Unfortunately, articulatory information is not directly observable, necessitating its estimation from the speech signal. This study describes a system that recognizes articulatory gestures from speech, and...

Prior research has shown that articulatory information, if extracted properly from the speech signal, can improve the performance of automatic speech recognition systems. However, such information is not readily available in the signal. The challenge posed by the estimation of articulatory information from speech acoustics has led to a new line of...

Previous studies have proposed ways to estimate articulatory information from the acoustic speech signal and have shown that when used with standard cepstral features, they help to improve word recognition performance in noise for a connected digit recognition task. In this paper, I present results from a word recognition and a phone recognition ex...

The human speech production system can be fundamentally characterized by the kinematic relationships between low-level articulator variables and relatively high-level tasks. Such kinematics can be illuminating about many system aspects from degrees of freedom and redundancy to dynamics and even control. Since these relationships are generally compl...

Previously we have proposed different models for estimating articulatory gestures and vocal tract variable (TV) trajectories from synthetic speech. We have shown that when deployed on natural speech, such models can help to improve the noise robustness of a hidden Markov model (HMM) based speech recognition system. In this paper we propose a model...

Speech inversion is a way of estimating articulatory trajectories or vocal tract configurations from the acoustic speech signal. Traditionally, articulator flesh-point or pellet trajectories have been used in speech-inversion research; however such information introduces additional variability into the inverse problem given they are head-centered,...

Background: Classification systems (Nagi, International Classification for Function [ICF]) have become popular for categorizing the level of ability (ICF) or disability (Nagi) associated with movement disorders. Nevertheless, these classifications do not explore the ways in which one level may influence other levels. For example, how might the wea...

A shifted field of view, an altered perception of optic flow speed, and gait asymmetries may influence heading direction in Parkinson's disease (PD). PD participants (left body-side onset, LPD, n=14; right body-side onset, RPD, n=9) and Healthy Control participants (n=17) walked a virtual hallway in which the optic flow speeds of the walls varied....

The present study examined the effect of visual feedback on the ability to recognise and consolidate pitch information. We trained two groups of nonmusicians to play a piano piece by ear, having one group receiving uninterrupted audiovisual feedback, while allowing the other only to hear, but not see their hand on the keyboard. Results indicate tha...

Speech can be represented as a constellation of constricting events, gestures, which are defined at distinct vocal tract sites, in the form of a gestural score. Gestures and their output trajectories, tract variables, which are available only in synthetic speech, have recently been shown to improve automatic speech recognition (ASR) performance. In...

Articulatory Phonology views speech as an ensemble of constricting events (e.g. narrowing lips, raising tongue tip), gestures, at distinct organs (lips, tongue tip, tongue body, velum, and glottis) along the vocal tract. This study shows that articulatory information in the form of gestures and their output trajectories (tract variable time functio...

Many different studies have claimed that articulatory information can be used to improve the performance of automatic speech recognition systems. Unfortunately, such articulatory information is not readily available in typical speaker-listener situations. Consequently, such information has to be estimated from the acoustic signal in a process which...

Speech can be represented as a set of discrete vocal tract constriction gestures (gestural score) defined at functionally distinct speech organs [tract variables (TVs)]. Using such gestures as sub-word units in an ASR system, variation in speech arising from coarticulation and reduction can be addressed. Since there is a lack of test corpora annota...

Articulatory information can improve the performance of automatic speech recognition systems. Unfortunately, since such information is not directly observable, it must be estimated from the acoustic signal using speech-inversion techniques. Here, we first compare five different machine learning techniques for inverting the speech acoustics generate...

This study examines prosody in American Sign Language using the theoretical framework of articulatory phonology, which proposes that the basic units of speech are articulatory gestures. We hypothesize that articulatory gestures are also the structural primitives of sign, and we are investigating what the gestures are and how they are timed. Kinemat...

We investigated dynamic programming (DP) and state-model (SM) approaches for estimating gestural scores from speech acoustics. We performed a word-identification task using the gestural pattern vector sequences estimated by each approach. For a set of 75 randomly chosen words, we obtained the best word-identification accuracy (66.67%) using the DP...

This paper analyzes the noise robustness of vocal tract constriction variable estimation and investigates their role for noise robust speech recognition. We implemented a simple direct inverse model using a feed-forward artificial neural network to estimate vocal tract variables (TVs) from the speech signal. Initially, we trained the model on clean...

We propose a framework that leverages articulatory phonology for speech recognition. "Gestural pattern vectors" (GPV) encode the instantaneous gestural activations that exist across all tract variables at each time. Given a speech observation, recognizing the sequence of GPV recovers the ensemble of gestural activations, i.e., the gestural score. F...

The purpose of the study is to investigate whether there are age-related differences in locomotion due to changes in presence of vision, optic flow speed, and lateral flow asymmetry using virtual reality technology. Gait kinematics and heading direction were measured using a three-dimensional motion analysis system. Although older and younger adult...

The global prosodic structure of languages has been described using the typological dichotomy of stress-timed versus syllable-timed. Various indices have been successfully employed in literature for quantifying these classifications, one of which is the duration ratio between the total voiceless and total voiced stretches in the signal. It has been...

We propose an instantaneous "gestural pattern vector" to encode the instantaneous pattern of gesture activations across tract variables in the gestural score. The design of these gestural pattern vectors is the first step towards an automatic speech rec- ognizer motivated by articulatory phonology, which is expected to be more invariant to speech c...

This study examined the transverse plane kinematics of the pelvis, thorax and head while participants walked at a range of speeds on a treadmill under three load conditions: no load, with a loaded backpack with no hip belt and with a loaded backpack with a hip belt. Research has suggested that one mechanism for adapting to heavy loads carried with...

The original task-dynamic model of speech production incorporated the theoretical tenets of Articulatory Phonology and provided a dynamics of inter-articulator coordination for single and co-produced constriction gestures, given a gestural score that specifies a time-dependent vector of gestural activations for a given utterance. More recently, the...

A fundamental problem in understanding speech production is how the temporal coherence of the speech units associated with a given lexical unit is maintained despite changes due to speaking rate, prosodic embedding, and transient perturbations. To address this, a dynamical model of temporal planning of speech has been developed (21, 13, 26, 27). In...

Although the description of mature walking is fairly well established, less is known about what is being learned in the process. Such knowledge is critical to the physical therapist who wants to teach children with developmental delays. The purpose of this experiment was to test the notion that learning to walk efficiently involves fine-tuning the...

In the past, the nature of the compositional units proposed for spoken language has largely diverged from the types of control units pursued in the domains of other skilled motor tasks. A classic source of evidence as to the units structuring speech has been patterns observed in speech errors--"slips of the tongue". The present study reports, for t...

The universality of music among human cultures as well as our common experience of naturally responding to music with motion, seem to be widely recognized (Tramo 2001). Recent brain-imaging studies (Lahav et al. 2005; Lahav, Saltzman, and Schlaug 2007) show that humans, given appropriate auditory inputs, seem to be “tuned ” to produce corresponding...

The discovery of audiovisual mirror neurons in monkeys gave rise to the hypothesis that premotor areas are inherently involved not only when observing actions but also when listening to action-related sound. However, the whole-brain functional formation underlying such "action-listening" is not fully understood. In addition, previous studies in hum...

The discovery of audiovisual mirror neurons in monkeys gave rise to the hypothesis that premotor areas are inherently involved not only when observing actions but also when listening to action-related sound. However, the whole-brain functional formation underlying such "action-listening" is not fully understood. In addition, previous studies in hum...

Language can be viewed as a structuring of cognitive units that can be transmitted among individuals for the purpose of communicating information. Cognitive units stand in specific and systematic relationships with one another, and linguists are interested in the characterization of these units and the nature of these relationships. Both can be exa...

Children with cerebral palsy (CP) often are faced with difficulty in walking. The purpose of this experiment was to determine the effects of functional electrical stimulation (FES) applied to the gastrocnemius-soleus muscle complex on the ability to produce appropriately timed force and reduce stiffness (elastic property of the body) and on stride...

The authors investigated the self-selected, overground walking patterns of 7 children (aged 11 months to 1 year, 5 months) at the initiation of walking (brand-new walkers [BNWs]) and for the next 6 months at 1-month intervals. Walking speed, stride length, and stride frequency increased significantly between the first 2 visits without significant c...

We trained musically naive subjects to play a short piano melody by ear in a fully monitored computerized environment and tested their potential to acquire a functional linkage between actions and sounds. Individual notes that were simply acoustic pretraining signals became "physically meaningful" posttraining. In addition, we found preliminary evi...

Research suggests that abnormal coordination patterns between the thorax and pelvis in the transverse plane observed in patients with Parkinson's disease and the elderly might be due to alteration in axial trunk stiffness. The purpose of this study was to develop a tool to estimate axial trunk stiffness during walking and to investigate its functio...

The dynamical systems underlying the performance and learning of skilled behaviors can be analyzed in terms of state-, parameter-, and graph-dynamics. We review these concepts and then focus on the manner in which variation in dynamical graph structure can be used to explicate the temporal patterning of speech. Simulations are presented of speech g...

A computational model has been developed that plans the relative timing of speech gestures within a syllable. Planning oscillators are associated with each of the gestures in a given utterance. These oscillators are phased to one another by being coupled in a pairwise, bidirectional manner according to the structure of a coupling graph that is part...

As humans increase walking speed, there are concurrent transitions in the frequency ratio between arm and leg movements from 2:1 to 1:1 and in the phase relationship between the movements of the two arms from in-phase to out-of-phase. Superharmonic resonance of a pendulum with monofrequency excitation had been proposed as a potential model for this...

this paper are that (a) multiple (more than two) levels of prosodic boundaries can be distinguished by their temporal articulatory characteristics, specifically by the boundary-adjacent lengthening of gestures; and (b), as has been suggested in previous work, this lengthening can be modeled, at least in part, by lowered gestural sti#ness. The resul...

The atypical walking pattern in children with spastic cerebral palsy is assumed to involve kinematic and morphological adaptations that allow them to move. The purpose of this study was to explore how the requirements of the task and the energy-generating and energy-conserving capabilities of children with cerebral palsy relate to kinematic and mec...

Thy work examines th relation between phween structure andth control and coordination of articulationwithW a dynamical systems model ofspeech production. InthI context, we reviewhW speakers modulateth spatiotemporal organization of articulatorygestures as a function ofthWW phWWx. position. We present computational simulations ths capture several im...

ABSTRACT Syllable-initial consonant ,sequences ,have been shown ,to differ from syllable-final ones in two ways: they show the c-center effect [3] and they exhibit greater stability in timing [5]. Here we show ,that both of these ,differences emerge automatically from a model in which phonetic units (gestures) are modeled as oscillators, and coordi...