Conference Paper

A hierarchical learning architecture with multiple-goal representations based on adaptive dynamic programming

Dept. of Electr., Comput., & Biomed. Eng., Univ. of Rhode Island, Kingston, RI, USA
DOI: 10.1109/ICNSC.2010.5461483 Conference: Networking, Sensing and Control (ICNSC), 2010 International Conference on
Source: IEEE Xplore

ABSTRACT In this paper we propose a hierarchical learning architecture with multiple-goal representations based on adaptive dynamic programming (ADP). The key idea of this architecture is to integrate a reference network to provide the internal reinforcement representation (secondary reinforcement signal) to interact with the operation of the learning system. Such a reference network serves an important role to build the internal goal representations. Furthermore, motivated by recent research in neurobiological and psychology research, the proposed ADP architecture can be designed in a hierarchical way, in which different levels of internal reinforcement signals can be developed to represent multi-level goals for the intelligent system. Detailed system level architecture, learning and adaptation principle, and simulation results are presented in this work to demonstrate the effectiveness of this work.

0 Bookmarks
 · 
31 Views
  • [Show abstract] [Hide abstract]
    ABSTRACT: We are interested in developing a multi-goal generator to provide detailed goal representations that help to improve the performance of the adaptive critic design (ACD). In this paper we propose a hierarchical structure of goal generator networks to cascade external reinforcement into more informative internal goal representations in the ACD. This is in contrast with previous designs in which the external reward signal is assigned to the critic network directly. The ACD control system performance is evaluated on the ball-and-beam balancing benchmark under noise-free and various noisy conditions. Simulation results in the form of a comparative study demonstrate effectiveness of our approach.
    Neural Networks (IJCNN), The 2012 International Joint Conference on; 01/2012
  • [Show abstract] [Hide abstract]
    ABSTRACT: This paper aims to integrate the fuzzy control with adaptive dynamic programming (ADP) scheme, to provide an optimized fuzzy control performance, together with faster convergence of ADP for the help of the fuzzy prior knowledge. ADP usually consists of two neural networks, one is the Actor as the controller, the other is the Critic as the performance evaluator. A fuzzy controller applied in many fields can be used instead as the Actor to speed up the learning convergence, because of its simplicity and prior information on fuzzy membership and rules. The parameters of the fuzzy rules are learned by ADP scheme to approach optimal control performance. The feature of fuzzy controller makes the system steady and robust to system states and uncertainties. Simulations on under-actuated systems, a cart-pole plant and a pendubot plant, are implemented. It is verified that the proposed scheme is capable of balancing under-actuated systems and has a wider control zone.
    Neural Networks (IJCNN), The 2012 International Joint Conference on; 01/2012
  • [Show abstract] [Hide abstract]
    ABSTRACT: Adaptive dynamic programming (ADP) is an effective method for learning while fuzzy controller has been put into use in many applications because of its simplicity and no need of accurate mathematic modeling. The combination of ADP and fuzzy control has been studied a lot. Before this paper, we have studied using ADP to learn the fuzzy rules of a Monotonic controller, which shows good performance. In this paper, a hyperbolic fuzzy model is adopted to make an improvement. In this way, both membership function and fuzzy rules are learned. With ADP algorithm, fuzzy controller has the capacity of learning and adapting. Simulations on a single cart-pole plant and a rotational inverted pendulum are implemented to observe the performance, even with uncertainties and disturbances.
    Intelligent Control and Automation (WCICA), 2012 10th World Congress on; 01/2012