No file available
To read the file of this research,
you can request a copy directly from the authors.
Value of Information in Feedback Control: Global Optimality
Preprints and early-stage research may not have been peer reviewed yet.
Abstract
The rate-regulation trade-off defined between two objective functions, one penalizing the packet rate and one the state deviation and control effort, can express the performance bound of a networked control system. However, the characterization of the set of globally optimal solutions in this trade-off for multi-dimensional controlled Gauss-Markov processes has been an open problem. In the present article, we characterize a policy profile that belongs to this set. We prove that such a policy profile consists of a symmetric threshold triggering policy, which can be expressed in terms of the value of information, and a certainty-equivalent control policy, which uses a conditional expectation with linear dynamics.
ResearchGate has not been able to resolve any citations for this publication.
This paper studies the optimal output-feedback control of a linear time-invariant system where a stochastic event-based scheduler triggers the communication between the sensor and the controller. The primary goal of the use of this type of scheduling strategy is to provide significant reductions in the usage of the sensor-to-controller communication and, in turn, improve energy expenditure in the network. In this paper, we aim to design an admissible control policy, which is a function of the observed output, to minimize a quadratic cost function while employing a stochastic event-triggered scheduler that preserves the Gaussian property of the plant state and the estimation error. For the infinite horizon case, we present analytical expressions that quantify the trade-off between the communication cost and control performance of such event-triggered control systems. This trade-off is confirmed quantitatively via numerical examples.
This paper considers a remote state estimation problem with multiple sensors
observing a dynamical process, where sensors transmit local state estimates
over an independent and identically distributed (i.i.d.) packet dropping
channel to a remote estimator. At every discrete time instant, the remote
estimator decides whether each sensor should transmit or not, with each sensor
transmission incurring a fixed energy cost. The channel is shared such that
collisions will occur if more than one sensor transmits at a time. Performance
is quantified via an optimization problem that minimizes a convex combination
of the expected estimation error covariance at the remote estimator and
expected energy usage across the sensors. For transmission schedules dependent
only on the estimation error covariance at the remote estimator, this work
establishes structural results on the optimal scheduling which show that 1) for
unstable systems, if the error covariance is large then a sensor will always be
scheduled to transmit, and 2) there is a threshold-type behaviour in switching
from one sensor transmitting to another. Specializing to the single sensor
case, these structural results demonstrate that a threshold policy (i.e.
transmit if the error covariance exceeds a certain threshold and don't transmit
otherwise) is optimal. We also consider the situation where sensors transmit
measurements instead of state estimates, and establish structural results
including the optimality of threshold policies for the single sensor, scalar
case. These results provide a theoretical justification for the use of such
threshold policies in variance based event triggered estimation. Numerical
studies confirm the qualitative behaviour predicted by our structural results.
An extension of the structural results to Markovian packet drops is also
outlined.
We consider sensor data scheduling for remote state estimation. Due to constrained communication energy and bandwidth, a sensor needs to decide whether it should send the measurement to a remote estimator for further processing. We propose an event-based sensor data scheduler for linear systems and derive the corresponding minimum squared error estimator. By selecting an appropriate event-triggering threshold, we illustrate how to achieve a desired balance between the sensor-to-estimator communication rate and the estimation quality. Simulation examples are provided to demonstrate the theory.
We propose an open-loop and a closed-loop stochastic event-triggered sensor
schedule for remote state estimation. Both schedules overcome the essential
difficulties of existing schedules in recent literature works where, through
introducing a deterministic event-triggering mechanism, the Gaussian property
of the innovation process is destroyed which produces a challenging nonlinear
filtering problem that cannot be solved unless approximation techniques are
adopted. The proposed stochastic event-triggered sensor schedules eliminate
such approximations. Under these two schedules, the MMSE estimator and its
estimation error covariance matrix at the remote estimator are given in a
closed-form. Simulation studies demonstrate that the proposed schedules have
better performance than periodic ones with the same sensor-to-estimator
communication rate.
For a closed-loop system, which has a contention-based multiple access
network on its sensor link, the Medium Access Controller (MAC) may discard some
packets when the traffic on the link is high. We use a local state-based
scheduler to select a few critical data packets to send to the MAC. In this
paper, we analyze the impact of such a scheduler on the closed-loop system in
the presence of traffic, and show that there is a dual effect with state-based
scheduling. In general, this makes the optimal scheduler and controller hard to
find. However, by removing past controls from the scheduling criterion, we find
that certainty equivalence holds. This condition is related to the classical
result of Bar-Shalom and Tse, and it leads to the design of a scheduler with a
certainty equivalent controller. This design, however, does not result in an
equivalent system to the original problem, in the sense of Witsenhausen.
Computing the estimate is difficult, but can be simplified by introducing a
symmetry constraint on the scheduler. Based on these findings, we propose a
dual predictor architecture for the closed-loop system, which ensures
separation between scheduler, observer and controller. We present an example of
this architecture, which illustrates a network-aware event-triggering
mechanism.
Let F= F(v1,..., vm) smooth on (R+0)m with Fvivj ≥ 0 for i≠j. Furthermore, let u1,..., um nonnegative and bounded functions on R n with compact support. We prove the inequality f RnF(u1,...,um)dx ≤ fRn F(u *1,...,u*m)dx, where* denotes symmetric decreasing rearrangement.
Although transmission of a data packet containing sensory information in a networked control system improves the quality of regulation, it has indeed a price from the communication perspective. It is, therefore, rational that such a data packet be transmitted only if it is valuable in the sense of a cost-benefit analysis. Yet, the fact is that little is known so far about this valuation of information and its connection with traditional event-triggered communication. In the present article, we study this intrinsic property of networked control systems by formulating a rate-regulation trade-off between the packet rate and the regulation cost with an event trigger and a controller as two distributed decision makers, and show that the valuation of information is conceivable and quantifiable grounded on this trade-off. In particular, we characterize an equilibrium in the rate-regulation trade-off, and quantify the value of information
${\rm{VoI}}_k$
there as the variation in a so-called value function with respect to a piece of sensory information that can be communicated to the controller at each time
$k$
. We prove that, for a multi-dimensional Gauss--Markov process,
${\rm{VoI}}_k$
is a symmetric function of the discrepancy between the state estimates at the event trigger and the controller, and that a data packet containing sensory information at time
$k$
should be transmitted to the controller only if
${\rm{VoI}}_k$
is nonnegative. Moreover, we discuss that
${\rm{VoI}}_k$
can be computed with arbitrary accuracy, and that it can be approximated by a closed-form quadratic function with a performance guarantee.
This paper studies a remote state estimation problem where a sensor, equipped with energy harvesting capabilities, observes a dynamical process and transmits local state estimates over a packet dropping channel to a remote estimator. The objective is to decide, at every discrete time instant, whether the sensor should transmit or not, in order to minimize the expected estimation error covariance at the remote estimator over a finite horizon, subject to constraints on the sensor’s battery energy governed by an energy harvesting process. We establish structural results on the optimal scheduling which show that, for a given battery energy level and a given harvested energy, the optimal policy is a threshold policy on the error covariance. Similarly, for a given error covariance and a given harvested energy, the optimal policy is a threshold policy on the current battery level. An extension to the problem of transmission scheduling and control with an energy harvesting sensor is also considered.
Consider the following communication scenario. An encoder observes a stochastic process and causally decides when and what to transmit about it, under a constraint on bits transmitted per second. A decoder uses the received codewords to causally estimate the process in real time. We aim to find the optimal encoding and decoding policies that minimize the end-to-end estimation mean-square error under the rate constraint. For a class of continuous Markov processes satisfying regularity conditions, we show that the optimal encoding policy transmits a 1-bit codeword once the process innovation passes one of two thresholds. The optimal decoder noiselessly recovers the last sample from the 1-bit codewords and codeword-generating time stamps, and uses it as the running estimate of the current process, until the next codeword arrives. In particular, we show the optimal causal code for the Ornstein-Uhlenbeck process and calculate its distortion-rate function.
This paper investigates remote state estimation over a bandwidth-limited channel. To improve the estimation quality, an event-based schedule is proposed to determine whether to transmit the current measurement to the remote estimator or not. We first show it is erroneous to apply the widely used Gaussian-approximation assumption in the event-based scheduling. As a replacement, the generalized closed skew normal (GCSN) distribution is introduced to accurately portray the system state distribution in the event-based scheduling. Furthermore, we present the probability density functions of the minimum mean-squared error estimation algorithm exactly without approximation for the first time. The closed-form expressions for the mean and covariance of the GCSN distribution are derived as well. Numerical examples validate the effectiveness of the proposed algorithm.
This paper studies a joint transmission scheduling and controller design problem, which minimizes a linear combination of the control cost and expected energy usage of the sensor. Assuming that the sensor transmission decisions are event-based and determined using the random estimation error covariance information available to the controller, we show a separation in the design of the transmission scheduler and controller. The optimal controller is given as the solution to an LQG-type problem, while the optimal transmission policy is a threshold policy on the estimation error covariance at the controller.
In this paper, we consider a sampling and remote estimation problem, where samples of a Wiener process are forwarded to a remote estimator via a channel with queueing and random delay. The estimator reconstructs an estimate of the real-time signal value from causally received samples. We obtain the jointly optimal sampling and estimation strategy that minimizes the mean-square estimation error subject to a maximum sampling rate constraint. We prove that a threshold-based sampler and a minimum mean-square error (MMSE) estimator are jointly optimal, and the optimal threshold is found exactly. Our jointly optimal solution exhibits an interesting coupling between the source and channel, which is different from the source-channel separation in many previous information theoretical studies. If the sampling times are independent of the observed Wiener process, the joint sampling and estimation optimization problem reduces to an age-of-information optimization problem that has been recently solved. Our theoretical and numerical comparisons show that the estimation error of the optimal sampling policy can be much smaller than those of age-optimal sampling, zero-wait sampling, and classic periodic sampling.
This paper investigates the optimal design of event-triggered estimation for linear systems. The synthesis approach is posed as a team decision problem where the decision makers are given by the event-trigger and the estimator. The event-trigger decides upon its available measurements whether the estimator shall obtain the current state information by transmitting it through a resource constrained channel. The objective is to find the optimal trade-off between the mean square estimation error and the expected number of transmissions over a finite horizon. After deriving basic characteristics of the optimal solution, we propose an iterative algorithm that alternates between optimizing one decision maker while fixing the other and vice versa. By analyzing the dynamical behavior of the iterative method, it is shown that the algorithm converges to a symmetric threshold policy for first-order systems if the statistics of the uncertainties are even and unimodal. In the case of bimodal distributions, we show numerically that the iterative method may find asymmetric threshold policies that outperform symmetric rules.
Multisensor data fusion
is the process of combining observations from a number of different sensors to provide a robust and complete description of an environment or process of interest. Data fusion finds wide application in many areas of robotics such as object recognition, environment mapping, and localization.
This chapter has three parts: methods, architectures, and applications. Most current data fusion methods employ probabilistic descriptions of observations and processes and use Bayes’ rule to combine this information. This chapter surveys the main probabilistic modeling and fusion techniques including grid-based models, Kalman filtering, and sequential Monte Carlo techniques. This chapter also briefly reviews a number of nonprobabilistic data fusion methods. Data fusion systems are often complex combinations of sensor devices, processing, and fusion algorithms. This chapter provides an overview of key principles in data fusion architectures from both a hardware and algorithmic viewpoint. The applications of data fusion are pervasive in robotics and underly the core problem of sensing, estimation, and perception. We highlight two example applications that bring out these features. The first describes a navigation or self-tracking application for an autonomous vehicle. The second describes an application in mapping and environment modeling.
The essential algorithmic tools of data fusion are reasonably well established. However, the development and use of these tools in realistic robotics applications is still developing.
The problem of sequential vector quantization of a stationary Markov source is cast as an equivalent stochastic control problem with partial observations. This problem is analyzed using the techniques of dynamic programming, leading to a characterization of optimal encoding schemes
The fundamental limits of remote estimation of Markov processes under
communication constraints are presented. The remote estimation system consists
of a sensor and an estimator. The sensor observes a discrete-time Markov
process, which is a symmetric countable state Markov source or a Gauss-Markov
process. At each time, the sensor either transmits the current state of the
Markov process or does not transmit at all. Communication is noiseless but
costly. The estimator estimates the Markov process based on the transmitted
observations. In such a system, there is a trade-off between communication cost
and estimation accuracy. Two fundamental limits of this trade-off are
characterized for infinite horizon discounted cost and average cost setups.
First, when each transmission is costly, we characterize the minimum achievable
cost of communication plus estimation error. Second, when there is a constraint
on the average number of transmissions, we characterize the minimum achievable
estimation error. Transmission and estimation strategies that achieve these
fundamental limits are also identified.
Digital control design is commonly constrained to time-triggered control systems with periodic sampling. The emergence of more and more complex and distributed systems urges the development of advanced triggering schemes that utilize communication, computation, and energy resources more efficiently. This technical note addresses the question whether certainty equivalence is optimal for an event-triggered control system with resource constraints. The problem setting is an extension of the stochastic linear quadratic system framework, where the joint design of the control law and the event-triggering law minimizing a common objective is considered. Three differing variants are studied that reflect the resource constraints: a penalty term to acquire the resource, a limitation on the number of resource acquisitions, and a constraint on the average number of resource acquisitions. By reformulating the underlying optimization problem, a characterization of the optimal control law is possible. This characterization shows that the certainty equivalence controller is optimal for all three optimization problems.
To reduce the amount of data transfer in networked systems, measurements are usually taken only when an event occurs rather than at each synchronous sample instant. However, this complicates estimation problems considerably, especially in the situation when no measurement is received anymore. The goal of this paper is therefore to develop a state estimator that can successfully cope with event based measurements and attains an asymptotically bounded error-covariance matrix. To that extent, a general mathematical description of event sampling is proposed. This description is used to set up a state estimator with a hybrid update, i.e., when an event occurs the estimated state is updated using the measurement, while at synchronous instants the update is based on knowledge that the sensor value lies within a bounded subset of the measurement space. Furthermore, to minimize computational complexity of the estimator, the algorithm is implemented using a sum of Gaussians approach. The benefits of this implementation are demonstrated by an illustrative example of state estimation with event sampling.
The outputs of a discrete time source with memory are to be encoded (“quantized” or “compressed”) into a sequence of discrete variables. From this latter sequence, a receiver must attempt to approximate some features of the source sequence. Operation is in real time, and the distortion measure does not tolerate delays. Such a situation has been investigated over infinite time spans by B. McMillan. In the present work, only finite time spans are considered. The main result is the following. If the source is kth-order Markov, one may, without loss, assume that the encoder forms each output using only the last k source symbols and the present state of the receiver's memory. An example is constructed, which shows that the Markov property is essential. The case of delay is also considered.
Consider a first order linear time-invariant discrete time system driven by process noise, a pre-processor that accepts causal measurements of the state of the system, and a state estimator. The pre-processor and the state estimator are not co-located, and, at every time-step, the pre-processor transmits either a real number or a free symbol to the estimator. We seek the pre-processor and the estimator that jointly minimize a cost that combines two terms; the expected squared state estimation error and a communication cost. In our formulation, the transmission of a real number from the pre-processor to the estimator incurs a positive cost while free symbols induce zero cost. This paper proves analytically that a symmetric threshold policy at the pre-processor and a Kalman-like filter at the estimator, which updates its estimate linearly in the presence of free symbols, are jointly optimal for our problem.
When a sensor has continuous measurements but sends limited messages over a
data network to a supervisor which estimates the state, the available packet
rate fixes the achievable quality of state estimation. When such rate limits
turn stringent, the sensor's messaging policy should be designed anew. What are
the good causal messaging policies ? What should message packets contain ? What
is the lowest possible distortion in a causal estimate at the supervisor ? Is
Delta sampling better than periodic sampling ? We answer these questions under
an idealized model of the network and the assumption of perfect measurements at
the sensor. For a scalar, linear diffusion process, we study the problem of
choosing the causal sampling times that will give the lowest aggregate squared
error distortion. We stick to finite-horizons and impose a hard upper bound on
the number of allowed samples. We cast the design as a problem of choosing an
optimal sequence of stopping times. We reduce this to a nested sequence of
problems each asking for a single optimal stopping time. Under an unproven but
natural assumption about the least-square estimate at the supervisor, each of
these single stopping problems are of standard form. The optimal stopping times
are random times when the estimation error exceeds designed envelopes. For the
case where the state is a Brownian motion, we give analytically: the shape of
the optimal sampling envelopes, the shape of the envelopes under optimal Delta
sampling, and their performances. Surprisingly, we find that Delta sampling
performs badly. Hence, when the rate constraint is a hard limit on the number
of samples over a finite horizon, we should should not use Delta sampling.
The symbols produced by a finite Markov source are causally encoded so as to be transmitted through a noisy memoryless channel. The encoder is assumed to have channel feedback information and the decoder to be causal. The feedback information is shown to be useful in general. Separation results are derived and used to prove that encoding is useless for a class of symmetric channels.
A linear system with a cost function that is quadratic in the control and terminal position error is given. The initial state is unknown and noise-corrupted observations are taken. Due to the cost of taking the observations, they are limited to a given number. Using the tools of optimum stochastic control theory, the optimum timing of the available observations is determined (minimizing the expected value of the loss function). For the cases considered, the locations can be determined a priori, and do not depend on the values of previous observations. Graphs of the optimum observation locations for several scalar cases are given. Graphs of the true cost, as a function of the locations of the observations, are also given for several scalar cases. The change in cost for small variations from the optimum location is usually not significant.
Optimal estimation with limited measurements
- O C Imer
- T Başar
O. C. Imer and T. Başar, "Optimal estimation with limited measurements," Intl. Journal of Systems, Control and Communications, vol. 2,
no. 1-3, pp. 5-29, 2010.
Jointly optimal LQG quantization and control policies for multi-dimensional systems
S. Yüksel, "Jointly optimal LQG quantization and control policies for
multi-dimensional systems," IEEE Trans. on Automatic Control, vol. 59,
no. 6, pp. 1612-1617, 2013.