-
[show abstract]
[hide abstract]
ABSTRACT: We introduce a stochastic approximation method for the solution of an ergodic
Kullback-Leibler control problem. A Kullback-Leibler control problem is a
Markov decision process on a finite state space in which the control cost is
proportional to a Kullback-Leibler divergence of the controlled transition
probabilities with respect to the uncontrolled transition probabilities. The
algorithm discussed in this work allows for a sound theoretical analysis using
the ODE method. In a numerical experiment the algorithm is shown to be
comparable to the power method and the related Z-learning algorithm in terms of
convergence speed. It may be used as the basis of a reinforcement learning
style algorithm for Markov decision problems.
12/2011;
-
[show abstract]
[hide abstract]
ABSTRACT: In this article we consider the problem of stochastic optimal control in continuous-time and state-action space of systems with state constraints. These systems typically appear in the area of robotics, where hard obstacles constrain the state space of the robot. A common approach is to solve the problem locally using a linear-quadratic Gaussian (LQG) method. We take a different approach and apply path integral control as introduced by Kappen (Kappen, H.J. (2005a), ‘Path Integrals and Symmetry Breaking for Optimal Control Theory’, Journal of Statistical Mechanics: Theory and Experiment, 2005, P11011; Kappen, H.J. (2005b), ‘Linear Theory for Control of Nonlinear Stochastic Systems’, Physical Review Letters, 95, 200201). We use hybrid Monte Carlo sampling to infer the control. We introduce an adaptive time discretisation scheme for the simulation of the controlled dynamics. We demonstrate our approach on two examples, a simple particle in a halfspace and a more complex two-joint manipulator, and we show that in a high noise regime our approach outperforms the iterative LQG method.
International Journal of Control 04/2011; 84(3). · 0.98 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Bayesian networks are widely accepted as models for reasoning with uncertainty. In this chapter, we focus on models that are
created using domain expertise only. After a short review of Bayesian network models and common Bayesian network modeling
approaches, we will discuss in more detail three applications of Bayesian networks.With these applications, we aim to illustrate
the modeling power and flexibility of the Bayesian networks, which go beyond the standard textbook applications. The first
network is applied in a system for medical diagnostic decision support. A distinguishing feature of this network is the large
amount of variables in the model. The second one involves an application for petrophysical decision support to determine the
mineral content of a well, based on borehole measurements. This model differs from standard Bayesian networks in terms of
its continuous variables and nonlinear relations. Finally, we will discuss an application for victim identification by kinship
analysis based on DNA profiles. The distinguishing feature in this application is that Bayesian networks are generated and
computed on-the-fly based on case information.
03/2010: pages 547-578;
-
Expert Syst. Appl. 01/2010; 37:7526-7532.
-
J. Artif. Intell. Res. (JAIR). 01/2008; 32:95-122.
-
[show abstract]
[hide abstract]
ABSTRACT: In this paper we derive the equations for Loop Corrected Belief Propagation on a continuous variable Gaussian model. Using the exactness of the averages for belief propagation for Gaussian models, a different way of obtaining the covariances is found, based on Belief Propagation on cavity graphs. We discuss the relation of this loop correction algorithm to Expectation Propagation algorithms for the case in which the model is no longer Gaussian, but slightly perturbed by nonlinear terms.
07/2007;
-
Adaptive Agents and Multi-Agent Systems III. Adaptation and Multi-Agent Learning, 5th, 6th, and 7th European Symposium, ALAMAS 2005-2007 on Adaptive and Learning Agents and Multi-Agent Systems, Revised Selected Papers; 01/2007
-
Journal of Machine Learning Research - Proceedings Track. 01/2007; 2:331-338.
-
6th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2007), Honolulu, Hawaii, USA, May 14-18, 2007; 01/2007
-
[show abstract]
[hide abstract]
ABSTRACT: We propose a method for improving approximate inference methods that corrects
for the influence of loops in the graphical model. The method is applicable to
arbitrary factor graphs, provided that the size of the Markov blankets is not
too large. It is an alternative implementation of an idea introduced recently
by Montanari and Rizzo (2005). In its simplest form, which amounts to the
assumption that no loops are present, the method reduces to the minimal Cluster
Variation Method approximation (which uses maximal factors as outer clusters).
On the other hand, using estimates of the effect of loops (obtained by some
approximate inference algorithm) and applying the Loop Correcting (LC) method
usually gives significantly better results than applying the approximate
inference algorithm directly without loop corrections. Indeed, we often observe
that the loop corrected error is approximately the square of the error of the
approximate inference method used to estimate the effect of loops. We compare
different variants of the Loop Correcting method with other approximate
inference methods on a variety of graphical models, including "real world"
networks, and conclude that the LC approach generally obtains the most accurate
results.
12/2006;
-
[show abstract]
[hide abstract]
ABSTRACT: In this article we show that traditional Cox survival analysis can be improved upon when supplemented with sensible priors and analysed within a neural Bayesian framework. We demonstrate that the Bayesian method gives more reliable predictions, in particular for relatively small data sets. The obtained posterior (the probability distribution of network parameters given the data) which in itself is intractable, can be made accessible by several approximations. We review approximations by Hybrid Markov Chain Monte Carlo sampling, a variational method and the Laplace approximation. We argue that although each Bayesian approach circumvents the shortcomings of the original Cox analysis, and therefore yields better predictive results, in practice the use of variational methods or Laplace is preferable. Since Cox survival analysis is infamous for its poor results with (too) many inputs, we use the Bayesian posterior to estimate p-values on the inputs and to formulate an algorithm for backward elimination. We show that after removal of irrelevant inputs Bayesian methods still achieve significantly better results than classical Cox.
Statistics in Medicine 11/2004; 23(19):2989-3012. · 1.88 Impact Factor
-
UAI '03, Proceedings of the 19th Conference in Uncertainty in Artificial Intelligence, Acapulco, Mexico, August 7-10 2003; 01/2003
-
[show abstract]
[hide abstract]
ABSTRACT: Exact inference in large, complex Bayesian networks is computationally intractable. Approximate schemes are therefore of great importance for real world computation. In this paper we consider an approximation scheme in which the original Bayesian network is approximated by another Bayesian network. The approximating network is optimised by an iterative procedure, which minimises the Kullback-Leibler divergence between the two networks. The procedure is guaranteed to converge to a local minimum of the Kullback-Leibler divergence. An important question in this scheme is how to choose the structure of the approximating network. In this paper we show how redundant structures of the approximating model can be pruned in advance. Simulation results of model selection and model optimisation are provided to illustrate the methods.
03/2000;
-
[show abstract]
[hide abstract]
ABSTRACT: We consider a stochastic nonlinear dynamical process with annihilation of parti-cles. This process can be viewed as the continuous time version of the extended Kalman filter/smoother. It also plays an important role in stochastic optimal con-trol theory. We derive a Gaussian approximation for this process. With the use of the path integral formalism we derive Euler-Lagrange equations for the mode. Furthermore, we derive a linear noise approximation to estimate the size of the fluctuations around the mode, and estimates of the partition function, based on the mode and Gaussian corrections. Numerical experiments confirm the validity of the approximation method. In addition, they show that the Gaussian correction provides a significant improvement of the estimate of the partition function.
-
[show abstract]
[hide abstract]
ABSTRACT: We describe two specific examples of neural-Bayesian approaches for complex modeling tasks: survival analysis and multitask learning. In both cases, we can come up with reasonable priors on the parameters of the neural network. As a result, the Bayesian approaches improve their (maximum likelihood) frequentist counterparts dramatically. By illustrating their application on the models under study, we review and compare algorithms that can be used for Bayesian inference: Laplace approximation, variational algorithms, Monte Carlo sampling, and empirical Bayes.
Theoretical Computer Science.
