No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
State-estimation and cooperative control with uncertain time
Abstract
State estimation and control typically depend on the implicit assumption that actuation and measurement occur at known points in time. This assumption is predicated on sufficiently precise timekeeping afforded by engineered clocks. Biological control systems do not have access to quartz-crystal technology and yet animals perform behaviors that engineered systems cannot. Here, we examine the problem of state estimation using imprecisely timed measurements, with known temporal statistics. We consider the case that there are two controllers, each with its own imperfect clock, performing a cooperative task-formulated in a leader-follower paradigm. In our problem, the follower estimates the state of the leader's noisy clock along with other state variables. An example of a complex cooperative leader-follower task is ballroom dancing.
... Biologically inspired controllers that account for the presence of internal noise have been developed in the optimal control framework for both sensorimotor control (106) and estimation of time (107,108). Noise can confer advantages to stochastic computing, such as raising signals above threshold through stochastic resonance (109,110) and decorrelating and whitening sensory signals (111,112). ...
Here, we review the role of control theory in modeling neural control systems through a top-down analysis approach. Specifically, we examine the role of the brain and central nervous system as the controller in the organism, connected to but isolated from the rest of the animal through insulated interfaces. Though biological and engineering control systems operate on similar principles, they differ in several critical features, which makes drawing inspiration from biology for engineering controllers challenging but worthwhile. We also outline a procedure that the control theorist can use to draw inspiration from the biological controller: starting from the intact, behaving animal; designing experiments to deconstruct and model hierarchies of feedback; modifying feedback topologies; perturbing inputs and plant dynamics; using the resultant outputs to perform system identification; and tuning and validating the resultant control-theoretic model using specially engineered robophysical models.
... Carver et al. [30] showed that even for a simplified model of motor control, time estimation requires complex computations. Thus, "open-loop" control does not, after all, eliminate the need for feedback or complex computations, since time itself must be estimated. ...
How humans and other animals control rhythmic behaviors, and locomotion in particular, is one of the grand challenges of neuroscience and biomechanics. And yet remarkably few studies address the fundamental control-systems modeling of locomotor control. This thesis attempts to address several pieces of this grand challenge through the development of experimental, theoretical, and computational tools. Specifically, we focus our attention on three key features of human rhythmic motor control, namely
variability, symmetry, and dynamics.
Variability: Little is known about how haptic sensing of discrete events, such as heel-strike in walking, in rhythmic dynamic tasks enhances behavior and performance. In order to discover the role of discrete haptic cues on rhythmic motor control performance, we study a virtual paddle juggling behavior. We show that haptic sensing of a force impulse to the hand at the moment of ball-paddle collision categorically improves performance over visual feedback alone, not by regulating the rate of convergence to steady state, but rather by reducing cycle-to-cycle variability.
Symmetry: Neglecting evident characteristics of a system can certainly be a modeling convenience, but it may also produce a better statistical model. For example, the dynamics of human locomotion is frequently treated as symmetric about the sagittal plane for modeling convenience. In this work, we test this assumption by examining the statistical consequences of neglecting (or not) bilateral asymmetries in the dynamics of human walking. Indeed, we show that there are statistically significant asymmetries in the walking dynamics of healthy participants (N=8), but that by ignoring these asymmetries and fitting a symmetric model to the data, we arrive at a more consistent and predictive model of human walking.
Dynamics: Rhythmic hybrid dynamic behaviors can be observed in a wide variety of biological and robotic systems. Analytic (white-box) modeling tools of such systems are limited to the case when we have a full (and preferably simple) mathematical model that can accurately describe the system dynamics. In contrast, data-driven (block-box) system identification methods have the potential to overcome this fundamental limitation and could play a critical role in describing and analyzing the dynamics of rhythmic behaviors based on experimental data. And yet few tools exist for identifying the dynamics of rhythmic systems from input--output data. In this context, we propose a new formulation for identifying the dynamics of rhythmic hybrid dynamical systems around their limit-cycles by using discrete-time harmonic transfer functions.
... Since the animal nervous system does not have access to a precise master clock, the nervous system must estimate and process time (Buonomano and Karmarkar 2002;Barclay et al. 2012;Eagleman 2008), likely integrating feedback from sensory stimuli. Carver et al. (2013) showed that even for a simplified model of motor control, time estimation requires complex computations. Thus open-loop control does not, after all, eliminate the need for feedback or complex computations, since time itself must be estimated. ...
Stability and performance during rhythmic motor behaviors such as locomotion are critical for survival across taxa: falling down would bode well for neither cheetah nor gazelle. Little is known about how haptic feedback, particularly during discrete events such as the heel-strike event during walking, enhances rhythmic behavior. To determine the effect of haptic cues on rhythmic motor performance, we investigated a virtual paddle juggling behavior, analogous to bouncing a table tennis ball on a paddle. Here, we show that a force impulse to the hand at the moment of ball-paddle collision categorically improves performance over visual feedback alone, not by regulating the rate of convergence to steady state (e.g., via higher gain feedback or modifying the steady-state hand motion), but rather by reducing cycle-to-cycle variability. This suggests that the timing and state cues afforded by haptic feedback decrease the nervous system's uncertainty of the state of the ball to enable more accurate control but that the feedback gain itself is unaltered. This decrease in variability leads to a substantial increase in the mean first passage time, a measure of the long-term metastability of a stochastic dynamical system. Rhythmic tasks such as locomotion and juggling involve intermittent contact with the environment (i.e., hybrid transitions), and the timing of such transitions is generally easy to sense via haptic feedback. This timing information may improve metastability, equating to less frequent falls or other failures depending on the task.
Volitional rhythmic motor behaviors such as limb cycling and locomotion exhibit spatial and timing regularity. Such rhythmic movements are executed in the presence of exogenous visual and non-visual cues, and previous studies have shown the pivotal role vision plays in guiding spatial and temporal regulation. However, the influence of non-visual information conveyed through auditory or touch sensory pathways, and its effect on control, remains poorly understood. To characterize the function of non-visual feedback in rhythmic arm control, we designed a paddle juggling task, where volunteers bounced a ball off a rigid elastic surface to a target height in virtual reality by moving a physical handle with the right hand. Feedback was delivered at two key phases of movement: visual feedback at ball peaks only, and simultaneous audio and haptic feedback at ball--paddle collisions. In contrast to previous work, we limited visual feedback to the minimum required for jugglers to assess spatial accuracy, and we independently perturbed the spatial dimensions and the timing of feedback. By separately perturbing this information, we evoked dissociable effects on spatial accuracy and timing, confirming that juggling, and potentially other rhythmic tasks, involves two complementary processes with distinct dynamics: spatial error correction and feedback timing synchronization. Moreover, we show evidence that audio and haptic feedback provide sufficient information for the brain to control the timing synchronization process by acting as a metronome-like cue that triggers hand movement.
Many control methods implicitly depend on the assumption that time is accurately known. For example, the finite-horizon linear quadratic regulator is a linear policy with time-varying gains. Such policies may be infeasible for controllers without accurate clocks, such as the motor systems in humans and other animals, since gains would be applied at incorrect times. Little appears to be known, however, about control with imperfect timing. This paper gives a solution to the linear quadratic regulator problem in which the state is perfectly known, but the controller's measure of time is a stochastic process derived from a strictly increasing Lévy process. The optimal controller is linear and can be computed from a generalization of the classical Riccati differential equation.
This paper examines stochastic optimal control problems in which the state is
perfectly known, but the controller's measure of time is a stochastic process
derived from a strictly increasing L\'evy process. We provide dynamic
programming results for continuous-time finite-horizon control and specialize
these results to solve a noisy-time variant of the linear quadratic regulator
problem and a portfolio optimization problem with random trade activity rates.
For the linear quadratic case, the optimal controller is linear and can be
computed from a generalization of the classical Riccati differential equation.
Animals time events on scales that range more than 10 orders of magnitude—from microseconds to days. This review focuses on timing that occurs in the range of tens to hundreds of milliseconds. It is within this range that virtually all the temporal cues for speech discrimination, and haptic and visual processing, occur. Additionally, on the motor side, it is on this scale that timing of fine motor movements takes place. To date, psychophysical data indicate that for many tasks there is a centralized timing mechanism, but that there are separate networks for different intervals. These data are supported by experiments that show that training to discriminate between two intervals generalizes to different modalities, but not different intervals. The mechanistic underpinnings of timing are not known. However various models have been proposed, they can be divided into labeled-line models and population clocks. In labeled-line models, different intervals are coded by activity in independent and discrete populations of neurons. In population models, time is coded by the population activity of a large group of neurons, and timing requires dynamic interaction between neurons. Population models are generally better suited for parallel processing of interval, duration, order, and sequence cues and are thus more likely to underlie timing in the range of tens to hundreds of milliseconds.
Plain-tailed wrens (Pheugopedius euophrys) cooperate to produce a duet song in which males and females rapidly alternate singing syllables. We examined how sensory information from each wren is used to coordinate singing between individuals for the production of this cooperative behavior. Previous findings in nonduetting songbird species suggest that premotor circuits should encode each bird's own contribution to the duet. In contrast, we find that both male and female wrens encode the combined cooperative output of the pair of birds. Further, behavior and neurophysiology show that both sexes coordinate the timing of their singing based on feedback from the partner and suggest that females may lead the duet.
An understanding of sensory and motor processing will require elucidation of the mechanisms by which the brain tells time. Open questions relate to whether timing relies on dedicated or intrinsic mechanisms and whether distinct mechanisms underlie timing across scales and modalities. Although experimental and theoretical studies support the notion that neural circuits are intrinsically capable of sensory timing on short scales, few general models of motor timing have been proposed. For one class of models, population clocks, it is proposed that time is encoded in the time-varying patterns of activity of a population of neurons. We argue that population clocks emerge from the internal dynamics of recurrently connected networks, are biologically realistic and account for many aspects of motor timing.
Studies of time coding in the central nervous system have revealed circuits specialized for the encoding and processing of temporal information. Behavioral experiments show that animals can detect microsecond time differences, while analysis of how temporal information is processed has uncovered many common principles. Despite different neural substrates, time coding systems share similar features and implement similar algorithms for the encoding of temporal information. Timing information is generally coded by phase-locked action potentials, and processed in a dedicated pathway in parallel with other stimulus variables. The elements of time-coding circuits have morphological and physiological features suited to their function.
Animals time events on scales that range more than 10 orders of magnitude-from microseconds to days. This review focuses on timing that occurs in the range of tens to hundreds of milliseconds. It is within this range that virtually all the temporal cues for speech discrimination, and haptic and visual processing, occur. Additionally, on the motor side, it is on this scale that timing of fine motor movements takes place. To date, psychophysical data indicate that for many tasks there is a centralized timing mechanism, but that there are separate networks for different intervals. These data are supported by experiments that show that training to discriminate between two intervals generalizes to different modalities, but not different intervals. The mechanistic underpinnings of timing are not known. However various models have been proposed, they can be divided into labeled-line models and population clocks. In labeled-line models, different intervals are coded by activity in independent and discrete populations of neurons. In population models, time is coded by the population activity of a large group of neurons, and timing requires dynamic interaction between neurons. Population models are generally better suited for parallel processing of interval, duration, order, and sequence cues and are thus more likely to underlie timing in the range of tens to hundreds of milliseconds.
Recent empirical studies have implicated the use of the motor system during action observation, imitation and social interaction. In this paper, we explore the computational parallels between the processes that occur in motor control and in action observation, imitation, social interaction and theory of mind. In particular, we examine the extent to which motor commands acting on the body can be equated with communicative signals acting on other people and suggest that computational solutions for motor control may have been extended to the domain of social interaction.
In mammals, circadian rhythms of physiology and behavior are regulated by a complex network of cellular molecular oscillators distributed throughout the brain and peripheral tissues. A master clock in the hypothalamic suprachiasmatic nuclei (SCN) synchronizes internal time with the external light-dark cycle, thus entraining the overall rhythmicity of the organism. Recent findings have challenged the dominant role of the SCN in physiological regulation and it becomes increasingly evident that close interaction between different central and peripheral clocks is necessary to maintain robust circadian rhythms of physiology and metabolism. In this review, we summarize recent findings regarding circadian organization in the SCN and in other central and peripheral tissues. We outline the communication pathways between different tissue clocks and, exemplified by the regulation of glucocorticoid release from the adrenal gland and glucose homeostasis in the blood, characterize the interaction between different clocks in the regulation of physiological processes.
This paper addresses the peculiar treatment that time receives when studying control systems. For example, why is the ability to perfectly observe time assumed implicitly in virtually all control formulations? What happens if this implicit assumption is violated? It turns out that some basic control results fall apart when time cannot be perfectly measured. To make this explicit, we introduce information space concepts that permit imperfect time information to be considered in the same way as imperfect state information. We then argue that classical open-loop control should be reconsidered as perfect time-feedback control. Following this, we introduce a notion of strongly open-loop control, which does not require perfect time observations. We provide some examples of these concepts and argue that many fascinating directions for future controls research emerge.
We have constructed an optical clock with a fractional frequency inaccuracy of 8.6x10{-18}, based on quantum logic spectroscopy of an Al+ ion. A simultaneously trapped Mg+ ion serves to sympathetically laser cool the Al+ ion and detect its quantum state. The frequency of the {1}S{0}<-->{3}P{0} clock transition is compared to that of a previously constructed Al+ optical clock with a statistical measurement uncertainty of 7.0x10{-18}. The two clocks exhibit a relative stability of 2.8x10{-15}tau{-1/2}, and a fractional frequency difference of -1.8x10{-17}, consistent with the accuracy limit of the older clock.
A rich tradition of normative psychophysics has identified two ubiquitous properties of interval timing: the scalar property, a strong form of Weber's law, and ratio comparison mechanisms. Finding the neural substrate of these properties is a major challenge for neurobiology. Recently, advances have been made in our understanding of the brain structures important for timing, especially the basal ganglia and the cerebellum. Surgical intervention or diseases of the cerebellum generally result in increased variability in temporal processing, whereas both clock and memory effects are seen for neurotransmitter interventions, lesions and diseases of the basal ganglia. We propose that cerebellar dysfunction may induce deregulation of tonic thalamic tuning, which disrupts gating of the mnemonic temporal information generated in the basal ganglia through striato-thalamo-cortical loops.
Recent gene discovery approaches have led to a new era in our understanding of the molecular basis of circadian oscillators in animals. A conserved set of genes in Drosophila and mammals (Clock, Bmal1, Period, and Timeless) provide a molecular framework for the circadian mechanism. These genes define a transcription-translation-based negative autoregulatory feedback loop that comprises the core elements generating circadian rhythmicity. This circadian core provides a focal point for understanding how circadian rhythms arise, how environmental inputs entrain the oscillatory system, and how the circadian system regulates its outputs. The addition of molecular genetic approaches to the existing physiological understanding of the mammalian circadian system provides new opportunities for understanding this basic life process.
Time in the biological sense is measured by cycles that range from
milliseconds to years. Circadian rhythms, which measure time on a scale of
24 h, are generated by one of the most ubiquitous and well-studied
timing systems. At the core of this timing mechanism is an intricate molecular
mechanism that ticks away in many different tissues throughout the body.
However, these independent rhythms are tamed by a master clock in the brain,
which coordinates tissue-specific rhythms according to light input it receives
from the outside world.
Humans and other animals demonstrate the ability to perceive and respond to temporally relevant information with characteristic behavioral properties. For example, the response time distributions in peak-interval timing tasks are well described by Gaussian functions, and superimpose when scaled by the criterion duration. This superimposition has been referred to as the scalar property and results from the fact that the standard deviation of a temporal estimate is proportional to the duration being timed. Various psychological models have been proposed to account for such responding. These models vary in their success in predicting the temporal control of behavior as well as in the neurobiological feasibility of the mechanisms they postulate. A review of the major interval timing models reveals that no current model is successful on both counts. The neurobiological properties of the basal ganglia, an area known to be necessary for interval timing and motor control, suggests that this set of structures act as a coincidence detector of cortical and thalamic input. The hypothesized functioning of the basal ganglia is similar to the mechanisms proposed in the beat frequency timing model [R.C. Miall, Neural Computation 1 (1989) 359-371], leading to a reevaluation of its capabilities in terms of behavioral prediction. By implementing a probabilistic firing rule, a dynamic response threshold, and adding variance to a number of its components, simulations of the striatal beat frequency model were able to produce output that is functionally equivalent to the expected behavioral response form of peak-interval timing procedures.
This paper describes a mathematical model of the neuronal central pattern generator (CPG) that controls the rhythmic body motion of the swimming leech. The systems approach is employed to capture the neuronal dynamics essential for generating coordinated oscillations of cell membrane potentials by a simple CPG architecture with a minimal number of parameters. Based on input/output data from physiological experiments, dynamical components (neurons and synaptic interactions) are first modeled individually and then integrated into a chain of nonlinear oscillators to form a CPG. We show through numerical simulations that the values of a few parameters can be estimated within physiologically reasonable ranges to achieve good fit of the data with respect to the phase, amplitude, and period. This parameter estimation leads to predictions regarding the synaptic coupling strength and intrinsic period gradient along the nerve cord, the latter of which agrees qualitatively with experimental observations.
Animals time events on scales that span from microseconds to days. In contrast to the technologies devised by humans to keep track of time, biology has developed vastly different mechanisms for timing across these different scales.