No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Creating efficient nonlinear neural network process models that allow model interpretation
Abstract
The KBANN (knowledge-based artificial neural networks) approach uses neural networks to refine knowledge that can be written in the form of simple propositional rules. This idea is extended by presenting the MANNIDENT (multivariable artificial neural network identification) algorithm by which the mathematical equations of linear process models determine the topology and initial weights of a network, which is further trained using backpropagation. This method is applied to the task of modelling a non-isothermal CSTR in which a first-order exothermic reaction is occurring. This method produces statistically significant gains in accuracy over both a standard neural network approach and a linear model. Furthermore, using the approximate linear model to initialize the weights of the network produces statistically less variation in accuracy. By structuring the neural network according to the approximate linear model, the model can be readily interpreted.
... The proposed algorithm is simple to implement and it can be obtained without the need of a detailed knowledge of the plant; therefore it is suited for industrial applications where standard solutions are generally preferred. The performance of the proposed technique for nonlinear process control was tested for two case studies, referring to a SISO [14] and a MIMO [15] control problem for a nonisothermal continuously stirred tank reactor (CSTR). These two cases, which are well known benchmarks for testing control methodology, were selected because the strong nonlinearities, due to the Arrhenius dependence of the kinetic rate on temperature, lead to a very rapid response of the process variables in regions of high conversion and a very mild response in regions of low conversion. ...
... As mentioned in Introduction, two continuously stirred tank reactor systems are considered to show the performance of the proposed control algorithm. The two proposed case studies are well known benchmarks for advanced control methodology testing [14,15,18]. ...
... As a second case study, a CSTR in which an exothermic first-order reaction takes place is considered. This benchmark was proposed by Scott and Ray [15] in order to demonstrate the inadequacy of a standard PI controller in such nonlinear systems and is described in a dimensionless form by the following differential equations: ...
This paper presents a gain-scheduling design technique that relies upon neural models to approximate plant behaviour. The controller design is based on generic model control (GMC) formalisms and linearization of the neural model of the process. As a result, a PI controller action is obtained, where the gain depends on the state of the system and is adapted instantaneously on-line. The algorithm is tested on a nonisothermal continuous stirred tank reactor (CSTR), considering both single-input single-output (SISO) and multi-input multi-output (MIMO) control problems. Simulation results show that the proposed controller provides satisfactory performance during set-point changes and disturbance rejection.
... Therefore, the non linear mapping feature of the artificial neural network (ANN) provides an attractive alternative for the modeling of chemical processes. Several neural network architectures are proposed for the modeling of non-linear dynamic systems (Karendra and Parthasarathy: 1990;Chen et al. , 1990: Qin et al. , 1992vVillis et al.. 1992 ;Pollard et al., 1992;Scott andRay 1993: Dayal et al.,1994 ;Karjala and Himmelblau, 1994). One architecture is the feed forwhere the measured (process) output is fed back as the input to the neural network for the future output prediction (N arendra and Parthasarathy,1990 ;Chen et al. , 1990 ;Bhat and McAvoy, 1990 ;Willis et al. , 1992 ;Pollard et al. , 1992;Dayal et al., 1994). ...
... The other architecture is the recurrent network (RecN) (or the parallel model) where the predicted output is fed back to the network for the future output prediction (Werbos , 1988 ;Qin et al. , 1992;Su et al.. 1992). The recurrent network studied here is the output (or external) recurrent system and the state (or internal) recurrent network such as Elman type of neural network will not be discussed here (Elman , 1990: Scott andRay, 1993 ;Karjala and Himmelblau , 1994).Qin et al. (1992)point out a fundamental difference between the two networks. That is : for noise-free system , FFK architecture is a better choice and as the noise level increases Rec;'\ gives better modeling capability. ...
A nonlinear dynamic model is useful for the purposes of monitoring, diagnosis and control of chemical processes. Since most process measurements are corrupted with different degree of measurement noises, any realistic on-line application of nonlinear dynamic model should take the effect of noises into account. In this work, a new neural network architecture is proposed. It is a hybrid between the feedforward network (FFN) and recurrent network (RecN) and the degree of recurrence depends on the noise level of the process. That is: part of the measured output and part of the predicted output are fed back to the network for the future output prediction. Furthermore, a procedure is proposed to adjust the feedback gain. Similar to the adjustment of the Kalman filter gain, the filter gain is changed according to the noise covariance and the prediction error covariance. Several simulated and experimental nonlinear systems are used to illustrate the effectiveness of the proposed neural network. The results show that proposed hybrid neural network gives better description of dynamical behavior then the fixed structure neural networks (i.e., FFN or RecN) over a wide range of noise levels
... Good examples are globally recurrent neural networks (e.g. [43,49]), Elman networks [13,40], dynamic feed forward networks with filters [34], and locally recurrent networks [14,47,53]. In globally recurrent networks, the network outputs are fed back to the network inputs through time delay units. ...
... Such a topology is similar to a nonlinear state space representation in dynamic systems. Scott and Ray [40] demonstrated the performance of an Elman network for nonlinear process modelling. Filter networks incorporate filters of the form N(s)/D(s) into the network interconnections [34]. ...
This chapter presents neurofuzzy networks for nonlinear process modeling and control. The neurofuzzy network uses local linear models to model a nonlinear process and the local linear models are combined using center of gravity (COG) defuzzification. In order to be able to provide accurate long-range predictions, a recurrent neurofuzzy network structure is developed. An advantage of neurofuzzy network models is that they are easy to interpret. Insight about the process characteristics at different operating regions can be easily obtained from the neurofuzzy network parameters. Based on the neurofuzzy network model, nonlinear model predictive controllers can be developed as a nonlinear combination of several local linear model predictive controllers that have analytical solutions. Applications to the modeling and control of a neutralization process and a fed-batch process demonstrate that the proposed recurrent neurofuzzy network is very effective in the modeling and control of nonlinear processes.
... Similar to a human brain the ANN gets trained from the existing set of data and then the trained ANN predicts the unknown output from a given set of input data. Among different types of neural network, multilayer perception (MLP) feed forward neural network is the most commonly used [8][9][10][11]. The MLP neural network normally consists of an input layer, an output layer and one or more hidden layer(s) based on the complexity of the problem in hand. The signal passes from the input layer to the output layer through hidden layer(s). ...
... During training weights associated with neurons are adjusted so that the error between predicted output and known output is minimized. The details about the learning method could be read from literature [8][9][10][11]. In the present study sigmoid function has been used as the activation function which is given below. ...
In conventional chromite beneficiation plant, huge quantity of chromite is used to loss in the form of tailing. For recovery these valuable mineral, a gravity concentrator viz. wet shaking table was used. Optimisation along with performance prediction of the unit operation is necessary for efficient recovery. So, in this present study, an artificial neural network (ANN) modeling approach was attempted for predicting the performance of wet shaking table in terms of grade (%) and recovery (%). A three layer feed forward neural network (3:3–11–2:2) was developed by varying the major operating parameters such as wash water flow rate (L/min), deck tilt angle (degree) and slurry feed rate (L/h). The predicted value obtained by the neural network model shows excellent agreement with the experimental values.
... Further enhancements using the MANNIDENT method are possible and are discussed more fully in Scott (1993) and Scott and Ray (1993a). Exploring different operating regions of the nonlinear CSTR, the ANN-CT network was able to model multiple steady state and limit cycle oscillations when the training data set contained these phenomena. ...
... Since these models can be of arbitrary order, more complex network structures result. An example is given in Scott (1993) and Scott and Ray (1993a). Finally, the ANN-CT models can be incorporated into nonlinear model-based controllers, which show excellent performance in the face of setpoint changes and process disturbances and have excellent robustness properties (Scott and Ray 1993b ...
The KBANN (Knowledge-Based Artificial Neural Networks) approach uses neural networks to refine knowledge that can be written in the form of simple propositional rules. This idea is extended by presenting the MANNIDENT (Multivariable Artificial Neural Network Identification) algorithm by which the mathematical equations of linear dynamic process models determine the topology and initial weights of a network, which is further trained using backpropagation. This method is applied to the task of modeling a nonisothermal chemical reactor in which a first-order exothermic reaction is occurring. This method produces statistically significant gains in accuracy over both a standard neural network approach and a linear model. Furthermore, using the approximate linear model to initialize the weights of the network produces statistically less variation in model fidelity. By structuring the neural network according to the approximate linear model, the model can be readily interpreted.
... In [26] the authors describe a method to use a traditional modeling paradigm to determine the topology and the initial weights of a neural network, and apply it to the problem of learning the model of a non linear multiple-inputs, multiple-outputs dynamic system. In particular, they present an illustrative example consisting of modeling the temperature and concentration in a non-isothermal, Continuously Stirred Tank Reactor (CSTR) where an exothermic, first order reaction is taking place. ...
... Even if the considered CRST is one of the simplest reactor model, it exhibits several nonlinear characteristics as multiple steady states, oscillations and high sensitivity. There are a number of successful applications of neural networks to this problem [26,27], and data are available to compare the performances of different approaches. The system has two inputs: the coolant temperature, u 1 , and the input feed rate, u 2 , and two outputs: the outflow concentration, y 1 , and the outflow temperature, y 2 . ...
: a general purpose implementation of the Tabu Search metaheuristic, called Universal Tabu Search, is used to optimally design a Locally Recurrent Neural Network architecture. In fact, generally, the design of a neural network is a tedious and time consuming trial and error operation that leads to structures whose optimality is not guaranteed. In this paper, the problem of choosing the number of hidden neurons and the number of taps and delays in the FIR and IIR network synapses is formalised as an optimisation problem whose cost function to be minimised is the network error calculated on a validation data set. The performances of the proposed approach have been tested on the problem of modelling non isothermal, continuously stirred tank reactor, in which a first order exothermic reaction is occurring. Comparison with alternative neural approaches are reported, showing the usefulness of the proposed method. I. INTRODUCTION Many engineering problems such as, for instance, ti...
... Meanwhile, the feed forward and back propagation is the most famous training algorithm to train the ANN [36]. The multilayer perceptron (MLP) feed forward neural network is the commonest prediction architecture [22,37]. The MLP architecture includes one input layer, one output layer and one or more hidden layers depending on the complexity of the problem being examined. ...
The artificial neural network (ANN) was used to predict the modulus of rupture (MOR) of the laminated wood products adhered by melamine/urea formaldehyde (MUF) resin with different formaldehyde to melamine/urea molar ratios combined with different weight ratios of the protein adhesive resulting from the alkaline treatment (NaOH) of the soybean oil meal to MUF resin pressed at different temperatures according to the central composite design (CCD). After making the boards and performing the mechanical test to measure the MOR, based on experimental data, different statistics such as determination coefficient (R 2), root mean square error (RMSE), mean absolute error (MAE) and sum of squares error (SSE) were determined, and then the suitable algorithm was selected to determine the estimated values. After comparing estimated values with the experimental values, the direct and interactive effects of the independent variables on MOR were determined. The results indicated that using suitable algorithms to train the ANN well, a very good estimate of the bending strength of the laminated wood products can be offered with the least error. In addition, based on the estimated and measured strengths and FTIR and TGA diagnostic analyses, it was found that the replacement of the MUF resin by the protein bio-based adhesive when using low F to M/U molar ratios, the MOR is maximized if a high range of temperature is used during the press.
... Among them, the multilayer perceptron (MLP) is the most widely used network architecture to make the predictions [20]. An MLP is designed with a combination of an input layer, an output layer and one or more hidden layers [26]. The mathematical representation of the prediction output is shown in Eq. 1. [5] where Y is the output of the prediction model; X i is the input variable; β j is the bias value for jth hidden neurons; w ij is the representation of weight between ith input and jth hidden neurons; while bias for output neuron represented by θ; g and f refer to the activation functions. ...
Constructing a prediction model of machining performance is useful to improve its process efficiency. Artificial neural network (ANN) has been widely used in prediction works, capable of solving complex problems with numerous parameters. The present study aims to describe the application of the ANN technique in predicting the machining performance of a natural material. Bovine horns were the selected natural materials. Bovine horns are sustainable, recyclable, and abundant source for industrial applications. The outputs of the predictive model were surface roughness and energy consumption, whereas the input data were spindle speed, depth of cut and feed rate of a face milling. It was found that the ANN-based prediction model of bovine horns produced a high accuracy prediction (95.4%). The outcome of this study may be referred by similar studies on other natural materials, supporting the global efforts in improving the industrialization of natural materials.
[You can access full-text of this paper by using the following link:
https://rdcu.be/b4FKw ]
... There are different kinds of neural networks (all of them are characterized by a specific architecture or some other features) and many different training algorithms used for identification problems for example Scott and Harmon [2,3,4], Codeca and Casella [5], Li [6], Gil et al [7], Ko et al [8], Kosmatopoulos et al [9], Alanis et al [10], ...
A back-propagation feed forward neural network with three layers was applied for different systems identifications. The intermediate layer (hidden layer) of the proposed neural network is built with hybrid activation functions provided with two different activation functions namely linear activation function and hyperbolic tangent activation function. The results show the ability of the network to identify linear as well as nonlinear system with high speed of convergence comparing with traditional neural network structures. Without the need to build networks that have activation factions correspond the applied system.
... There are different kinds of neural networks (all of them are characterized by a specific architecture or some other features) and many different training algorithms used for identification problems for example Scott and Harmon [2,3,4], Codeca and Casella [5], Li [6], Gil et al [7], Ko et al [8], Kosmatopoulos et al [9], Alanis et al [10], ...
... It is well known that non-linear multi-step ahead prediction models can be built using dynamic neural networks . Good examples are globally recurrent neural networks (Su et ai, 1992), Elman networks (Elman, 1990;Scott and Ray, 1993) , dynamic feed forward networks with filters (Montague et ai, 1992), and locally recurrent networks (Frasconi et ai, 1992;Tsoi and Back , 1994;Zhang and Morris, 1995a). In this paper, a recurrent fuzzy neural network is proposed which allows the construction of a "global" non-linear multi-step-ahead model from the fuzzy conjunction of a number of "local" dvn _mic models. ...
A multi-step ahead recurrent fuzzy neural network topology is proposed which enables the building of prediction models of nonlinear dynamic processes for heterogeneous model based predictive control. Inputs to the fuzzy neural network are partitioned into several overlapping fuzzy operating regions. Within each region, a simplified linear process model is used. The overall ‘global’ model output is calculated through centre of gravity defuzzification. Process knowledge is used to initially set up the fuzzy network with process input-output data being used to train the network. The new approach is demonstrated by the application to the highly nonlinear pH dynamics in a neutralisation process.
... The multilayer perceptron (MLP) feed forward neural network is the most commonly used architecture for prediction. The MLP architecture consists of an input layer, an output layer and one or more hidden layer(s) depending on the degree of the complexity of the problem in hand [24,25]. Each layer in the MLP is made up of a collection of interconnected elements called neurons that allow the network to produce a specific output from input variables. ...
... There are various kinds of neural network approaches available, among which multilayer perception (MLP)-a feedforward backpropagation neural network-is the most commonly used ANN approach. Backpropagation is an algorithm which is commonly applied for training (Scott and Ray 1993;Jorjani, Chelgani, and Mesroghli 2008;Kalyani et al. 2008;Liao, Liu, and Wang 2011;Wu and Jiang 2011;Panda et al. 2012). Each neuron layer in ANN is connected to the next adjacent layer of the neuron system, and all neurons are particularly associated with different weights. ...
Present paper describes an attempt made to study the possibility of beneficiating low-grade iron ore fines of Barbil Area of Orissa state, India, using Multi Gravity Separator (MGS) after grinding the −10 mm fines to < 75 micron size and prepare a pellet feed of 65% Fe content. For the performance analysis, an artificial neural network (ANN) mathematical modelling approach was attempted. A three layer feed forward neural network with a back propagation method has been adopted, considering the three significant parameters of MGS, mainly drum inclination, drum speed, and shake amplitude, were varied and the results were evaluated for grade, recovery and separation efficiency. The results of beneficiation studies showed that, good recovery of hematite is possible with simultaneous increase in Fe(T) grade from 50.74 to 65.26% with 71.25% recovery. The predicted value obtained by ANN shows good agreement with the experimental values.
... The use of dynamic neural network for long range prediction models has been studied in globally recurrent neural networks (RNN, e.g. Su, McAvoy, & Werbos, 1992;Werbos, 1990), Elman networks (Elman, 1990;Scott & Ray, 1993), dynamic feed-forward network with filters (Montague, Tham, Willis, & Morris, 1992;Morris et al., 1994) and locally recurrent networks (Frasconi, Gori, & Soda, 1992;Tsoi & Back, 1994;Zhang & Morris, 1995a). Zhang and Morris (1995a) presented a sequential orthogonal training strategy which allows for hidden neurons to be added gradually, avoiding an unnecessarily large network structure. ...
Nonlinearities in system dynamics and the multivariable nature of processes offer a stiff challenge in designing predictive controllers that improve process performance in industries. This investigation presents a recurrent neuro fuzzy network (RNFN) model for a nonlinear multivariable system in process industries and a methodology to design model-predictive controllers (MPCs) using the proposed model. The RNFN model combines the learning features of artificial neural networks with human cognition capabilities of fuzzy systems. Therefore, RNFN leads to a modelling framework that has the ability not only to learn the model parameters, but also makes decision on operating region of the nonlinear model depending on the input–output data. Furthermore, the recurrent structure and the introduction of a memory unit between the fuzzy inference and fuzzification layer enhance the prediction capability due to the use of past input–output data, making the model more suitable for designing predictive controllers. Next, the MPC design methodology that exploits the advantages of the RNFN model to optimize the control moves is presented. The proposed MPC uses the gradient descent algorithm to minimize the control moves as against the traditional state-space approaches that require complex computations and solvers. Therefore, implementing the proposed MPC in embedded hardware becomes easier. The proposed modelling framework and the MPC design methodology are illustrated using experiments on a laboratory-scale quadruple tank. Our experiments show that the proposed RNFN-based MPC performs better than the neuro fuzzy network-based MPC for both servo and regulatory responses.
... Moreover, from an identi cation point of view, the nonlinear parameterization of many neural network based model representations is a serious drawback, in particular as long as there exists powerful linearly parameterized alternative model structures. Recently, there has been some interest in the application a prior knowledge for structuring neural nets (Mavrovouniotis and Chang 1992), initialization of neural network parameters and interpretation of the resulting model through linearizations (Scott and Ray 1993). A further step was taken in (Kramer et al. 1992, Thompson and Kramer 1994, Su et al. 1992, Psichogios and Ungar 1992, Aoyama and Venkatasubramanian 1993, Brown, Ruchti and Feng 1993 where certain combinations of neural network structures and mechanistic model structures was suggested. ...
This paper presents a non-linear modeling framework that supports model development in between empirical and mechanistic modeling. A model is composed of a number of local models valid in different operating regimes. The local models are combined by smooth interpolation into a complete global model. It is illustrated how different kinds of empirical and mechanistic knowledge and models can be combined with process data within this framework. Furthermore, we describe a flexible computer aided modeling tool that supports modeling within this framework. Simple but illustrative examples from chemical engineering are used to highlight the flexibility and power of the framework.
... A great variety of ANN exist classi®ed by their topology, learning laws, and training algorithms (Kasabov 1996). However, most of the currently used ANN for dynamic modeling and control are layered feedforward neural networks (FNN), also called multi-layer perceptron (Scott and Harmon Ray 1993) and recurrent neural networks (RNN) (You and Nikolaou 1993). A FNN requires as inputs a number of past values for each physical input and output of the dynamical structure (Bhat and McAvoy 1990). ...
In this article a task-oriented neural network (NN) solution is proposed for the problem of article recovering real process outputs from available distorted measurements. It is shown that a neural network can be used as approximator of inverted first-order measurement dynamics with and without time delay. The trained NN is connected in series with the sensor, resulting in an identity mapping between the inputs and the outputs of the composed system. In this way the network acts as a software mechanism to compensate for the existing dynamics of the whole measurement system and recover the actual process output. For those cases where changes in the measurement system occur, a multiple concurrent-NN recovering scheme is proposed. This requires a periodical path-finding calibration to be performed. A procedure for such a calibration purpose has also been developed, implemented, and tested. It is shown that it brings adequate robustness to the overall compensation scheme. Results showing the performance of both the NN compensator and the calibration procedure are presented for closed loop system operation.
... It results in highly complex processes with a strong nonlinear nature. The ordinary differential equations (ODE) based dynamical modelling of this kind of systems faced various difficulties that stimulated the developments of neural network as non linear black box models to represent bioprocesses (Thibault et al., 1990;Karim and Rivera, 1992;Scott and Ray, 1993;Montague and Morris, 1994). ...
We propose a general methodology to develop a hybrid neural model for a wide range of biotechnological processes. The hybrid neural modelling approach combines the flexibility of a neural network representation of unknown process kinetics with a global mass-balance based process description. The hybrid model is built in such a way that its trajectories keep their physical and biological meaning (mass balance, positivity of the concentrations, boundness, saturation or inhibition of kinetics) even far from the identification data conditions. We examine the constraints (a priori knowledge) that must be satisfied by the model and that provide additional conditions to be imposed on the neural network. We illustrate our approach with various biotechnological processes showing how to select the appropriate neural network architecture. The method is detailed for modelling an anaerobic wastewater treatment bioreactor using experimental data.
... Thibault et al. introduced a neural network as nonlinear black box for the on-line estimation of fermentation variables (Thibault et al., 1990). Similar approach is reported in (Karim and Rivera, 1992;Scott and Ray, 1993;Montague and Morris, 1994). Neural networks are also increasingly used in hybrid modelling (Psichogios and Ungar, 1992;Schubert et al., 1994a;Simutis et al., 1995;Feyo de Azevedo et al., 1997) which aims at including all available knowledge of the process. ...
This paper presents a hybrid approach for the modelling of an anaerobic digestion process. The hybrid model combines a feed-forward network, describing the bacterial kinetics, and the a priori knowledge based on the mass balances of the process components. We have considered an architecture which incorporates the neural network as a static model of unmeasured process parameters (kinetic growth rate) and an integrator for the dynamic representation of the process using a set of dynamic differential equations. The paper contains a description of the neural network component training procedure. The performance of this approach is illustrated with experimental data.
... The Elman network (Elman 1990) is an important recurrent network, where the outputs of neurons in the hidden layer at the previous time step are fed back to the context neurons. Scott and Ray (1993) demonstrated the performance of the Elman network for nonlinear process modelling. Pham and Karaboga (1999) developed a modi¯ed version of the Elman network to facilitate its application in dynamic system identi¯cation. ...
A new type of recurrent neural network is discussed in this paper, which provides the potential for the modelling of unknown nonlinear systems with multi-inputs and multi-outputs. The proposed network is a generalization of the network described by Elman. It is shown that the proposed network with appropriate neurons in the context layer can model unknown nonlinear systems. Based on the PID-like training objective function, the learning algorithm of the proposed network is considerably faster through the introduction of dynamic backpropagation, which is used to estimate the weights of both the feedforward and feedback connections. The techniques have been successfully applied to the modelling nonlinear plants and simulation results are included. 1 Introduction The feedforward neural networks have been widely applied to dynamic system identiflcation in recent years (Billings et al: 1992, Narendra and Parthasarathy 1990, Reynold and Tenorio 1990). However, the feedforward network can n...
This paper describes a novel technique for multiple parameter extraction of the S12X TEM cell model using a fuzzy logic system (FLS). The FLS is utilized to capture the circuit information and to extract the circuit parameters based on experiential knowledge. The proposed extraction technique uses both linguistic information (i.e., human-like knowledge and experience) and numerical data of measurement to construct the fuzzy macromodel. The simulation results confirm the validity and estimation performance of the equivalent circuit by the advocated design methodology.
This paper gives a highly abbreviated overview of some of the key issues in empirical nonlinear modeling for chemical process applications. This task is complicated by the inherent nature of nonlinearity: the term describes a class of systems by the one feature they lack. In fact, this division — linear vs. nonlinear — suggests a “unity” or “homogeneity” of the class of nonlinear systems that does not exist. Consequently, this review will focus on specific sub-classes of nonlinear models that have analytically useful structural characteristics and comparisons will be made both between theses classes and with the more familiar linear models. Length limitations restrict these discussions somewhat, but it is hoped that the range of examples will be great enough to demonstrate how nonlinear model identification is both similar to and different from linear model identification. The general conclusion of this paper is that nonlinear input/output modeling is a vitally important practical art with many unresolved issues; the principal objective of this paper is to elucidate some of these issues.
In locally recurrent neural networks, the output of a dynamic neuron is only fed back to itself. This particular structure makes it possible to train the network sequentially. A sequential orthogonal training method is developed in this chapter to train locally recurrent neural networks. The networks considered here contain a single-hidden-layer and dynamic neurons are located in the hidden layer. During network training, the first hidden neuron is used to model the relationship between inputs and outputs whereas other hidden neurons are added sequentially to model the relationship between inputs and model residuals. When adding a hidden neuron, its contribution is due to that part of its output vector which is orthogonal to the space spanned by the output vectors of the previous hidden neurons. The Gram-Schmidt orthogonalisation technique is used at each training step to form a set of orthogonal bases for the space spanned by the hidden neuron outputs. The optimum hidden layer weights can be obtained through gradient based optimisation method while the output layer weights can be found using least squares regression. Hidden neurons are added sequentially and the training procedure terminates when the model error is lower than a predefined level. Using this training method, the necessary number of hidden neurons can be found and, hence, avoiding the problem of over fitting. Neurons with mixed types of activation functions and dynamic orders can be incorporated into a single network. Mixed node networks can offer improved performance in terms of representation capabilities and network size parsimony. The excellent performance of the proposed technique is demonstrated by application examples.
In this work, a comparison between alternative neural approaches to model chaotic systems is reported. In particular, two different approaches have been presented. The first, is a Locally Recurrent Neural Network that, keeping the feedforward architecture of the MLP, replaces the classical synapses with Finite Impulse Response and Infinite Impulse Response filters. The second, is a novel dynamic neural network obtained by making recurrent the neurons in the output layer. The performances of the proposed approaches have been tested on the problem of modeling the dynamics of a non-isothermal, continuously stirred tank reactor when two consecutive first order reactions lead to a chaotic behavior. Moreover, the obtained dynamic neural networks have been used to develop a Generic Model Control controller.
There are two major applications of neural networks for process forecasting, modeling, and control of time-dependent processes: (1) process optimization; and (2) adaptive process control. Modeling chemical and biological processes using conventional techniques is often difficult because the processes are very nonlinear and complicated. For example, many chemical processes have a large number of reacting components and follow very complex reaction mechanisms. The neural network approach provides a method to develop a dynamic model that accounts for the instabilities and unsteady-state operating conditions that often occur in chemical systems. The neural networks used for data compression and data filtering are divided into the two main categories: signal-processing networks and image-processing networks. The most commonly used neural network for signal processing is the auto-associative back-propagation network. Other architectures used in data compression are learning-vector-quantization networks, self-organizing map networks, and recirculation networks. Developing traditional models based on first principles is typically very time-consuming, and it is difficult to achieve accurate results with them.
This is a review of the most important and recent (1990-2002) approaches to the use of neural networks in the polymer reaction engineering. It has two parts: the first contains general notions about neural networks - topology, types of networks, methods of establishing their architecture and training; the second part presents different possibilities of using neural networks for the modeling, optimization and control of polymerization processes. The paper outlines the correlation of the type and structure of networks with the characteristics of the process and the purpose of modeling. A short analysis of the classical phenomenological modeling shows the difficulties of the mathematical modeling of polymerization processes and of using these models for optimization and automatic control. Thus the necessity of the modeling based on neural networks is justified and the advantages of this technique are stressed. The systems and the situations are presented in which dynamic or static neural networks can be used and also individual or stacked neural networks. Beside the general recommendations stated, the paper tries, through some case studies, to present clearly the steps of the neural networks modeling of polymerization processes, the use of models in optimization and control and the results of these applications as compared to those of the mechanistic models.
A bed may be fluidized in the sense that all the particles are fully supported by the gas, but may still be segregated in the sense that the local bed composition does not correspond with the overall average. Segregation is likely to occur when there is substantial difference in the drag/unit weight between different particles. Particles having a higher drag/unit weight migrate to the distributor. Strongly segregating fully fluidized beds exhibit a characteristic segregation pattern in which the bed stratifies into two layers; a bottom layer of pure jetsam and an upper layer of uniform composition containing both flotsam and jetsam. Binary mixtures of same size but different density materials are fluidized in a bed where co-axial mixing promoters are introduced. The segregation characteristic of jetsam particles has been determined for different mixtures in terms of the segregation distance by empirically correlating the results with the various system parameters for both the un-promoted and the promoted beds. Correlations have also been developed with the above system parameters from an Artificial Neural Network approach. Segregation distances for the promoted and un-promoted beds have been compared. In this study, correlation for segregation distance in the following form has been developed for the heterogeneous systems from dimensional analysis approach using the various system parameters viz. initial static bed height, particle density, composition of the bed mixture, height of any layer of particles above the bottom grid and superficial gas mass velocity for the un-promoted and promoted beds in the following form, Ys = K × [(ρf / ρj × ρm/ρ j)a (Hs/Dc)b (H b/Dc)c(Uo/Umf) d]n The developed correlations are validated by means of correlations developed by Artificial Neural Network Approach (ANN-Model) in the Supervised Learning framework, where separate correlations have been developed for different beds with the same system parameters. But different values for the exponents and the correlation constant were obtained in each case. The considered sigmoidal activation function is expressed as follows, f(x) = 2.0x(1/1+e-λ.x - 0.5) Where, X is the slope parameter which determines the slope of the activation function for ANN- Model. The optimum network structure was selected for the minimum mean square error at the output. It is observed that the segregation tendency is enhanced by the presence of a promoter.
Neural Networks have made their mark in various fields of endeavor through their superior learning skills. In this paper, a Neural Network (NN) based reference model of a Brushless Direct Current Motor (BLDCM) has been developed in MATLAB/Simulink. During careful scrutiny, it was observed that the electromagnetic torque varies with the magnitude of the phase currents for motor operation between commutations and during commutation. Thus, constant torque production involves phase currents as the vital correction parameter. A novel torque control scheme through phase current control has been proposed in this work. The NNBLDCM has been employed to nullify the deviations in phase current waveforms at loaded condition of a real BLDCM through Neural Network based Current Injection Technique (NNCIT). The simulation results prove that the new technique improves the constant torque characteristic which, in turn, leads to an increase in the reliability of the BLDCM.
Reliability is one of the most critical factors that is to be considered during the designing phase of any product. There are many factors that contribute to make a system more reliable in terms of area, power, operating frequency and accuracy. This paper proposes the design of a 4bit fault tolerant ALU system using backend designing. Parallel processing along with triple modular redundancy (TMR) has been used to increase the operating speed of the system and also to make the system fault tolerant and accurate. The backend designing has been done using Micro-wind 3.1.7
An earlier paper introduced the MANNIDENT (multivariable artificial neural network identification) approach to process modelling in which approximate linear models determine the topology and initial weights of a neural network and which provides a structure for interpretation of the final model. This paper introduces the MANNCON (multivariable artificial neural network control) system, in which these models are incorporated into model-based controllers. Designs using internal model control (IMC) and direct synthesis control (DSC) as well as a supervisory ANN-based controller are presented. These methods are applied to the task of controlling a non-isothermal CSTR in which a first-order exothermic reaction is occurring. The nonlinear ANN-based controllers perform much better than their linear counterparts in controlling the process over a wide range of conditions. Furthermore, the superivosyr ANN controller showed better robustness properties with respect to disturbance rejection and to plant-model mismatch than the other controllers studied.
This paper deals with the experiments conducted to obtain pressure drop in a squared gas-solid fluidized bed under varying conditions. The flow rate, the initial static bed height, particle size and density have been altered and the bed pressure drops under fluidized conditions have been measured. The data have been correlated in terms of Euler Number. Correlations, relating Euler Number with the system parameters have been developed for the unpromoted bed and beds with rod and disc promoters. Developed correlations have been authenticated by an ANN model. The values of Euler Number, thus predicted, as well as the bed pressure drop values calculated there of have been compared with the experimental values and have been found to agree fairly well.
The paper focuses on the modeling of the growing process of tomato planted in greenhouse. The modified Elman network and fuzzy genetic algorithm are used for modeling in this paper. The growing process of tomato planted in greenhouse has some features as control variables being complex . the growing process being randomicity, nonlinear and changing all of a sudden, etc.. One of the main apparatus, caudexes is chosen as the object. The solar radiation, the temperature, humidity and carbon dioxide strength are chosen from all influential factors as control input according to the experts' advices. According to the characters, modified Elman network is used for modeling because of its good performance in dynamic system identification. And fuzzy genetic algorithm is used for learning the parameters of neural network. Results of the simulations show that the model based on modified Elman network is of better performance than those based on other method.
To overcome the limitation of linear feedfoforward controllers in nonlinear chemical processes, Lee and Park proposed the feedforward control scheme using a neural network and a dynamic matrix control (DMC). The neural feedforward controller (NFFC) using a neural network and a general linear controller is proposed to extend Lee and Park's control scheme. The generalized disturbance error learning method using the pseudo desired output is also proposed to train the neural network in the NFFC. Results are given for the implementation of the NFFC on a pilot-scale distillation column. The control performance of the NFFC is compared with conventional linear feedforward-feedback and feedback-only controllers such as the PI and the DMC controllers through various experiments. The trained NFFC shows excellent control performance compared with the linear controllers. The results indicate that the NFFC copes well with high nonlinearities and interactions, and may by useful in practice.
Binary mixtures of particles of the same size but of different densities are fluidized in a 15 cm diameter column with a perforated plate distributor and two coaxial promoters. In the present work an attempt has been made to study the fluidization and the segregation characteristic of density-variant solids of the same size in terms of segregation distance. The dimensionless segregation distance has been correlated with other dimensionless groups relating to various system parameters: ratio of the density of jetsam particles to that of flotsam, initial static bed height, height of layer of particles above the bottom grid, superficial gas velocity, and average density of the mixture on the basis of the dimensional analysis approach for both un-promoted and promoted beds. Correlations have also been developed with the above system parameters by using an artificial neural network approach for different types of fluidized beds, and the findings with respect to both approaches have been compared with each other. The values of segregation distance for promoted beds have also been compared with those for the un-promoted bed in this work.
Many industrial chemical processes are difficult to control effectively using existing techniques because they are complex, interconnected, nonlinear systems which lack reliable on-line measurements of key process variables. Due to increasingly stringent demands on product quality, energy utilization, and environmental responsibility, more effective control strategies are needed for these processes. By contrast, extremely complex biological systems routinely operate under more stringent requirements on "product quality" and "failure tolerance" as a result of the robust, high performance computation and control functions provided by the brain. Thus, studying and understanding these biological control systems, and ultimately "reverse engineering" their functions, should provide ample alternative techniques for developing effective control systems for chemical processes. The objective of this paper is to present one such biological control system - the baroreceptor reflex, which provides short-term regulation of arterial blood pressure - and identify potential applications in chemical process control. Novel process monitoring, modeling, and control strategies which are currently being developed by "reverse engineering" the architectural and computational properties of this reflex are discussed. Preliminary results on techniques for sensor fusion based control, nonlinear modeling, and control of multiple-input, single-output systems which have been abstracted from the reflex are also presented.
The stringent industrial requirements coupled with complex optimization issues make industrial processes challenging to control. These inherent challenges include mathematical modeling of physical processes, modeling deficiencies, multivariable interactions, time varying parameters, time delays, uncertainties, nonlinearities etc… It is far beyond the scope of conventional control algorithms to deal with these challenges. Over the past few decades, several advanced process control strategies have been proposed and deployed on commercial basis to overcome these challenges. The paper presents a succinct overview of advanced process control strategies including adaptive control, model predictive control, artificial intelligent control systems etc… The paper also details the status quo of advanced process control schemes in several industries like oil industry, petrochemical industry, paper and pulp etc… Keywords: advanced process control, adaptive control, model predictive control, non linear control, soft computing, artificial intelligence.
An internal model control strategy employing a fuzzy neural network is proposed for SISO nonlinear process. The control-affine model is identified from both steady state and transient data using back-propagation. The inverse of the process is obtained through algebraic inversion of the process model. The resulting model is easier to interpret than models obtained from the standard neural network approaches. The proposed approach is applied to the tasks of modelling and control of a continuous stirred tank reactor and a pH neutralization process which are not inherently control-affine. The results show a significant performance improvement over a conventional PID controller. In addition, an additional neural network which models the discrepancy between a control-affine model and real process dynamics is added, and is shown to lead to further improvement in the closed-loop performance.
This paper addresses attitudes and forms of process modelling in biochemical engineering. Baker's yeast production in a fed-batch fermenter, at laboratory scale, is employed as case-study. Three modelling approaches are described and compared, viz. — the conventional mechanistic approach, formulations based on different artificial neural network (ANN) topologies and a hybrid mechanistic-ANN structure. A standard 2-step procedure of model development, estimation (training) and validation with two independent sets of experiments, has been carried out. The mechanistic model, using reaction kinetic schemes from the literature, fine tuned by classical non-linear regression, gave smooth predictions for the validation data runs, but showed limited ability in predicting the test data. The ANN were able to describe experiments at the training stage, but failed the validation (i.e. extrapolation) procedure, giving oscillatory predictions of the process state. Additionally, this approach suffers from a strong influence of the net parameters, which must be chosen by trial and error. The hybrid model predictions are good with the training and very satisfactory with the experimental test data. The indication is that the latter is a powerful tool for process modelling in biochemical engineering, particularly when limited theoretical knowledge of the process is available.
Mixed order locally recurrent neural networks are proposed to build long term prediction models for nonlinear processes. In a mixed order locally recurrent network, the output of a hidden neuron is fed back to its input through several units of time delay, different hidden neurons can have different numbers of feedbacks. A sequential orthogonal training method is proposed to train locally recurrent neural networks. The first hidden neuron essentially models any underlying linear relationship in the process data with additional hidden neurons being introduced sequentially to model the relationship between the model inputs and the model residuals to improve the ability of the model to represent the underlying process. Network training is terminated when the model error on the testing data cannot be further reduced. Through this training strategy, an appropriate network topology is automatically determined. The classical Gram–Schmidt orthogonalisation technique in conjunction with regularisation is used as part of the hidden neuron selection procedure to improve the generalisation capability of the network. Applications of the technique to a continuous polymerisation process and a neutralisation process demonstrate that mixed order locally recurrent networks can provide good long-term predictions for nonlinear processes.
This paper presents a novel approach for process control that uses neural networks to model the steady-state inverse of a process which is then coupled with a simple reference system synthesis to generate a multivariable controller. The control strategy is applied to dynamic simulations of two methanol-water distillation columns that express distinctly different behaviour from each other (one simulates a lab column, while the second simulates an industrial-scale high-purity column). A steady-state process simulation package was used to generate all the neural network training data. An efficient training algorithm based on a nonlinear least-squares technique was used to train the networks. The neural network model-based controllers show robust performance for both setpoints and disturbances, and performed better than conventional feedback proportional-integral (PI) controllers with feedforward features.
In this work, a general framework for modelling and simulation of particulate solid drying processes is presented, based on fundamental conservation laws associated with a neural network, for model uncertain parameter estimation. The modeling approach, which leads to a hybrid-neural model, is applied in order to describe the dynamic behavior of two important drying systems: a direct flow rotary dryer and a batch fluidized bed dryer. Both models are built using simple mass and energy balances, where heat and mass transfer parameters are estimated with neural networks. Model behavior was evaluated by comparing experimental and simulation data. It is concluded that the hybrid-neural modeling approach is better for adaptation and prediction than its black box type counterpart.
This paper gives a highly abbreviated overview of some of the key issues in empirical nonlinear modelling for chemical process applications. This task is complicated by the inherent nature of nonlinearity: the term describes a class of systems by the one feature they lack. In fact, this division — linear vs. nonlinear — suggests a ‘unity’ or ‘homogeneity’ of the class of nonlinear systems that does not exist. Consequently, this review will focus on specific sub-classes of nonlinear models that have analytically useful structural characteristics, and comparisons will be made both between these classes and with the more familiar linear models. Length limitations restrict these discussions somewhat, but it is hoped that the range of examples will be great enough to demonstrate how nonlinear model identification is both similar to and different from linear model indentification. The general conclusion of this paper is that nonlinear input/output modelling is a vitally important practical art with many unresolved issues; the principal objective of this paper is to elucidate some of these issues.
In a previous paper we introduced a supervisory control framework for setting the operating point of an industrial process with unknown characteristics. Here we extend this methodology for the case of MIMO processes. The proposed supervisory control framework is agent-based and it involves modeling agents and intelligent agents. The modeling agents derive local steady-state models of the process during normal process operation by using various identification techniques, each of which corresponds to a different instantiation of our system. The intelligent agents are proposed to be implemented using finite automata units. The performance of the proposed scheme is illustrated through simulation results on well-known benchmark problems of chemical industry.
This paper describes a novel technique for multiple parameter extraction of the S12X TEM cell model using a fuzzy logic system FLS. The FLS is utilized to capture the circuit information and to extract the circuit parameters based on experiential knowledge. The proposed extraction technique uses both linguistic information i.e., human-like knowledge and experience and numerical data of measurement to construct the fuzzy macromodel. The simulation results confirm the validity and estimation performance of the equivalent circuit by the advocated design methodology.
A general purpose implementation of the tabu search metaheuristic, called Universal Tabu Search, is used to optimally design
a locally recurrent neural network architecture. The design of a neural network is a tedious and time consuming trial and
error operation that leads to structures whose optimality is not guaranteed. In this paper, the problem of choosing the number
of hidden neurons and the number of taps and delays in the FIR and IIR network synapses is formalised as an optimisation problem,
whose cost function to be minimised is the network error calculated on a validation data set. The performance of the proposed
approach has been tested on the problem of modelling the dynamics of a non-isothermal, continuously stirred tank reactor,
in two different operating conditions: when a first order exothermic reaction is occurring; and when two consecutive first
order reactions lead to a chaotic behaviour. Comparisons with alternative neural approaches are reported, showing the usefulness
of the proposed method.
In the present case, an extensive experimentation has been conducted to study the combined effect of promoter and distributor parameters on bed dynamics relating to minimum fluidization velocity, bed pressure drop, bed expansion and fluctuation. A total of three different types of promoters viz. rod type (four number of varying rod configuration and blockage), disk type (eleven number of varying disk thickness and diameter) and one blade type have been used in the investigation. Other system variables include gas distributor (five number of different opening area), bed materials of different size and density and initial static bed height.
This work presents the results of a hybrid neural model (HNM) technique as applied to modeling supercritical fluid extraction (SCFE) curves obtained from two Brazilian vegetable matrices. The serial HNM employed uses a neural network to estimate parameters of a phenomenological model. A small set of SCFE data for each vegetable was used to generate a semi-empirical extended data set, large enough for efficient network training, using three different approaches. Afterwards, other sets of experimental data, not used during the training procedure, were used to validate each approach. The HNM correlates well withthe experimental data, and it is shown that the predictions accomplished with this technique may be promising for SCFE purposes.
A type of recurrent neuro-fuzzy network is proposed in this paper to build long-term prediction models for nonlinear processes. The process operation is partitioned into several fuzzy operating regions. Within each region, a local linear model is used to model the process. The global model output is obtained through the centre of gravity defuzzification which is essentially the interpolation of local model outputs. This modeling strategy utilizes both process knowledge and process input output data. Process knowledge is used to initially divide the process operation into several fuzzy operating regions and to set up the initial fuzzification layer weights. Process input output data are used to train the network. Network weights are such trained so that the long-term prediction errors are minimized. Through training, membership functions of fuzzy operating regions are refined and local models are learn. Based on the recurrent neuro-fuzzy network model, a novel type of nonlinear model-based long range predictive controller can be developed and it consists of several local linear model-based predictive controllers. Local controllers are constructed based on the corresponding local linear models and their outputs are combined to form a global control action by using their membership functions. This control strategy has the advantage that control actions can be calculated analytically avoiding the time consuming nonlinear programming procedures required in conventional nonlinear model-based predictive control. The techniques have been successfully applied to the modeling and control of a neutralization process.
The problem of adjusting the set point of industrial processes with unknown characteristics is studied. An agent based supervisory control system framework is proposed. Modeling agents as well as intelligent agents are included. The modeling agents derive local steady - state models around operating points of the process. These models are considered to be constructed during normal process operation using various identification techniques, each of which corresponds to a different instantiation of our system. The intelligent agents are proposed to be implemented using finite automata units. The performance of the proposed scheme is illustrated through simulation results to some well-known benchmark problems of chemical industry
Error backpropagation in feedforward neural network models is a popular learning algorithm that has its roots in nonlinear estimation and optimization. It is being used routinely to calculate error gradients in nonlinear systems with hundreds of thousands of parameters. However, the classical architecture for backpropagation has severe restrictions. The extension of backpropagation to networks with recurrent connections will be reviewed. It is now possible to efficiently compute the error gradients for networks that have temporal dynamics, which opens applications to a host of problems in systems identification and control.
Several strategies are described that overcome limitations of basic network models as steps towards the design of large connectionist speech recognition systems. The two major areas of concern are the problem of time and the problem of scaling. Speech signals continuously vary over time and encode and transmit enormous amounts of human knowledge. To decode these signals, neural networks must be able to use appropriate representations of time and it must be possible to extend these nets to almost arbitrary sizes and complexity within finite resources. The problem of time is addressed by the development of a Time-Delay Neural Network; the problem of scaling by Modularity and Incremental Design of large nets based on smaller subcomponent nets. It is shown that small networks trained to perform limited tasks develop time invariant, hidden abstractions that can subsequently be exploited to train larger, more complex nets efficiently. Using these techniques, phoneme recognition networks of increasing complexity can be constructed that all achieve superior recognition performance.
Thesis (Ph. D.)--Harvard University, 1975. Includes bibliographical references.
The dynamic behavior of the continuous stirred tank reactor is analysed and classified for a variable reactor residence time. Although earlier work, treating the bifurcation to limit cycles and steady states with changing Damköhler number, yields a complete description of the problem, the evolution of multiple steady states and limit cycles is much more bizarre as the reactor residence time varies. In addition, it is the reactor residence time which is most easily varied experimentally so that the present results are more readily compared with experiment. Plots are given to show the influence of system parameters on the reactor behavior.
An abstract is not available.
The general specifications and structure of a computer aided control system analysis and design package suitable for chemical process control are outlined and a specific realization, the package CONSYD (CONtrol SYstem Design) is discussed.
Time underlies many interesting human behaviors. Thus, the question of how to represent time in connectionist models is very important. One approach is to represent time implicitly by its effects on processing rather than explicitly (as in a spatial representation). The current report develops a proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory. In this approach, hidden unit patterns are fed back to themselves; the internal representations which develop thus reflect task demands in the context of prior internal states. A set of simulations is reported which range from relatively simple problems (temporal version of XOR) to discovering syntactic/semantic features for words. The networks are able to learn interesting internal representations which incorporate task demands with memory demands; indeed, in this approach the notion of memory is inextricably bound up with task processing. These representations reveal a rich structure, which allows them to be highly context-dependent, while also expressing generalizations across classes of items. These representations suggest a method for representing lexical categories and the type/token distinction.
Neural computing is one of the fastest growing areas of artificial intelligence. Neural nets are inherently parallel and they hold great promise because of their ability to “learn” nonlinear relationships. This paper discusses the use of backpropagation neural nets for dynamic modeling and control of chemical process systems. The backpropagation algorithm and its rationale are reviewed. The algorithm is applied to model the dynamic response of pH in a CSTR. Compared to traditional ARMA modeling, the backpropagation technique is shown to be able to pick up more of the nonlinear characteristics of the CSTR. The use of backpropagation models for control, including learning process inverses, is briefly discussed.
It is demonstrated that neural networks can be used effectively
for the identification and control of nonlinear dynamical systems. The
emphasis is on models for both identification and control. Static and
dynamic backpropagation methods for the adjustment of parameters are
discussed. In the models that are introduced, multilayer and recurrent
networks are interconnected in novel configurations, and hence there is
a real need to study them in a unified fashion. Simulation results
reveal that the identification and adaptive control schemes suggested
are practically feasible. Basic concepts and definitions are introduced
throughout, and theoretical questions that have to be addressed are also
described
The use of neural nets for modeling nonlinear chemical systems is discussed. Three cases are considered: a steady-state reactor, a dynamic pH stirred tank system, and interpretation of biosensor data. In all cases, a back-propagation net is used successfully to model the system. One advantage of neural nets is that they are inherently parallel and, as a result, can solve problems much faster than a serial digit computer. Furthermore, neural nets have the ability to learn. Rather than programming neural computers, one presents them with a series of examples, and from these examples the nets learn the governing relationships involved in the training database.< >
- Elman