Content uploaded by Md Fahmi Abd Samad

Author content

All content in this area was uploaded by Md Fahmi Abd Samad on May 26, 2022

Content may be subject to copyright.

Content uploaded by Md Fahmi Abd Samad

Author content

All content in this area was uploaded by Md Fahmi Abd Samad on Nov 07, 2019

Content may be subject to copyright.

Content uploaded by Md Fahmi Abd Samad

Author content

All content in this area was uploaded by Md Fahmi Abd Samad on Nov 07, 2019

Content may be subject to copyright.

Content uploaded by Md Fahmi Abd Samad

Author content

All content in this area was uploaded by Md Fahmi Abd Samad on Apr 23, 2018

Content may be subject to copyright.

VOL. 11 NUM. 1 YEAR 2018 ISSN

1985-6571

Combating Corrosion: Risk Identification, Mitigation and Management

Mahdi Che Isa, Abdul Rauf Abdul Manaf & Mohd Hambali Anuar 1 - 12

Mechanical Properties Extraction of Composite Material Using Digital Image Correlation via

Open Source NCorr

Ahmad Fuad Ab Ghani, Jamaluddin Mahmud, Saiful Nazran & Norsalim Muhammad

13 - 24

Effects of Mapping on the Predicted Crash Response of Circular Cup-Shape Part

Rosmia Mohd Amman, Sivakumar Dhar Malingam, Ismail Abu-Shah & Mohd Faizal Halim 25 - 35

Effect of Nickle Foil Width on the Generated Wave Mode from a Magnetostrictive Sensor

Nor Salim Muhammad, Ayuob Sultan Saif Alnadhari, Roszaidi Ramlan, Reduan Mat Dan,

Ruztamreen Jenal & Mohd Khairi Mohamed Nor

36 - 48

Rapid Defect Screening on Plate Structures Using Infrared Thermography

Nor Salim Muhammad, Abd Rahman Dullah, Ahmad Fuad Ad Ghani, Roszaidi Ramlan &

Ruztamreen Jenal

49 - 56

Optimisation of Electrodeposition Parameters on the Mechanical Properties of Nickel Cobalt

Coated Mild Steel

Nik Hassanuddin Nik Yusoff, Othman Mamat, Mahdi Che Isa & Norlaili Amir

57 - 65

Preparation and Characterization of PBXN-109EB as a New High Performance Plastic Bonded

Explosive

Mahdi Ashrafi, Hossein Fakhraian, Ahmad Mollaei & Seyed Amanollah Mousavi Nodoushan

66 - 76

Nonlinear ROV Modelling and Control System Design Using Adaptive U-Model, FLC and PID

Control Approaches

Nur Afande Ali Hussain, Syed Saad Azhar Ali, Mohamad Naufal Mohamad Saad & Mark Ovinis

77 - 89

Flight Simulator Information Support

Vladimir R. Roganov, Elvira V. Roganova, Michail J. Micheev, Tatyana V. Zhashkova, Olga A.

Kuvshinova & Svetlan M. Gushchin

90 - 98

Implementation of Parameter Magnitude-Based Information Criterion in Identification of a Real

System

Md Fahmi Abd Samad & Abdul Rahman Mohd Nasir

99 - 106

Design Method for Distributed Adaptive Systems Providing Data Security for Automated

Process Control Systems

Aleksei A. Sychugov & Dmitrii O. Rudnev

107 - 112

Low Contrast Image Enhancement Using Renyi Entropy

Vijayalakshmi Dhurairajan,Teku Sandhya Kumari & Chekka Anitha Bhavani 113 - 122

Determination of Artificial Recharge Locations Using Fuzzy Analytic Hierarchy Process (AHP)

Marzieh Mokarram & Dinesh Sathyamoorthy 123 - 131

Availability Oriented Contract Management Approach: A Simplified View to a Complex Naval

Issue

Al-Shafiq Abdul Wahid, Mohd ZamaniAhmad, Khairol Amali Ahmad & Aisha Abdullah

132 - 153

Ministry of Defence

Malaysia

SCIENCE & TECHNOLOGY RESEARCH INSTITUTE

FOR DEFENCE (STRIDE)

99

IMPLEMENTATION OF PARAMETER MAGNITUDE-BASED INFORMATION

CRITERION IN IDENTIFICATION OF A REAL SYSTEM

Md Fahmi Abd Samad* & Abdul Rahman Mohd Nasir

Faculty of Mechanical Engineering, Universiti Teknikal Malaysia Melaka (UTeM), Malaysia

*Email: mdfahmi@utem.edu.my

ABSTRACT

Model structure selection is one among crucial steps in system identification and in order to carry out

this, an information criterion is needed. It plays an important role in determining an optimum model

structure with the aim of selecting an adequate model to represent a real system. A good information

criterion should not only evaluate predictive accuracy but also the parsimony of the model. In the

past, there had not been, or scarcely have been, any information criterion that evaluates the

parsimony of model structures (bias contribution) based on the magnitude of parameter or coefficient.

However, recently, some efforts had been made that took into account such strategy and proved the

criterion to be effective for simulated datasets. This paper presents the comparison between two

information criteria that are based on parameter magnitude information in selecting a good model to

represent a real system based on gas furnace. Genetic algorithm (GA) was used to optimise the

implementation. The selected models were then tested using correlation tests for model validation. It

is shown that parameter magnitude based information criterion 2 (PMIC2) is able to select a more

parsimonious model than PMIC but with similar validation results.

Keywords: Parameter magnitude; information criterion; system identification; linear regressive

model; genetic algorithm (GA).

1. INTRODUCTION

In many fields, nonlinear dynamic modelling is applied in approximating a wide range of systems.

Basically, system identification is used for estimating a model to represent their systems. System

identification can be considered a regression problem, where the relationship between input and

output variables of a dynamical system has to be estimated. This task is typically accomplished by

minimising a certain information criterion, which measures how well the estimated relationship

approximates the one that truly links the available input-output data pairs (Ljung, 1999). Its basic idea

is to compare the time dependent responses of the actual system and identified model based on a

performance function, hereby referred to as information criterion, giving a measure of how well the

model response fits the system response (Alfi & Fateh, 2010).

An identification procedure typically consists of estimating the parameters of different models, and

next, selecting the optimal model complexity within that set. Increasing the model complexity will

decrease the systematic errors, but, at the same time, the model variability increases (Riddef et al.,

2004). Model accuracy and parsimony, known as variance and bias: , are

important considerations in selecting a model structure (Ljung, 1999). Hence, selecting a model with

the smallest variance is not a good idea because when the number of parameters increase, the variance

will continue to decrease but will present a complex model. At a certain complexity, the additional

parameters no longer reduce the systematic errors but are used to follow the actual noise realisation on

the data. Often, in order to deal with the bias-variance trade-off, the information criterion is

augmented with a penalty term intended to guide the search for the “optimal” relationship penalising

undesired regressors, where regressors refer to possible terms and variables. Regularised estimation

has been widely applied in the context of system identification (Prando et al., 2015). Model validation

100

is the final step of system identification. It is used to check the goodness of the estimated model

(Zhang et al., 2005).

In Samad & Nasir (2017a, b, c, 2018a), parameter magnitude-based information criterion 2 (PMIC2)

was tested on simulated datasets generated from autoregressive with exogenous input (ARX) and

nonlinear autoregressive with exogenous input (NARX) models. Comparisons were made between

PMIC2, Akaike information criterion (AIC) (Akaike, 1974), Bayesian information criterion (BIC)

(Schwartz, 1978) and corrected Akaike information criterion (AICc) (Anderson et al., 1994). In the

simulation results, PMIC2 proved that it performed better than AIC, BIC and AICc for all the

simulated models (ARX and NARX). In this paper, the effectiveness of PMIC2 will be studied by

testing on real system data, in this case, gas furnace data. The models were generated as NARX

models. Since there are many possible models to test, genetic algorithm (GA) was used to optimise

the search. The models selected by PMIC2 were tested by correlation tests for model validation. The

results of PMIC2 were then compared to another parameter-magnitude based information criterion

(Samad et al., 2013), hereby denoted PMIC, in order to evaluate the performance between both

information criteria.

The next sections are as follows: Section 2 introduces information criteria; Section 3 explains about

GA, which is used as part of the search for model structure; Section 4 explains the model validation

tests; Section 5 provides simulation setup; Section 6 presents result and discussions; and lastly,

Section 7 concludes the findings of this paper.

2. INFORMATION CRITERIA

According to Samad et al. (2013), parameter magnitude-based information criterion (PMIC) evaluates

the bias contribution by the sum of squared residuals while the variance contribution is calculated by a

penalty function (PF). This is written as follows:

∑

(1)

where PF =ln, is the number of terms satisfying 1 while represents the

absolute value of the parameter for term and is a fixed value termed penalty function

parameter. The penalty function penalises terms with the absolute values of the estimated parameter

less than the penalty. The penalty parameter value will be set equal to or slightly lower than the

parameter value in cases where the smallest tolerable absolute parameter value is known or can be

roughly estimated.

By referring to Samad & Nasir (2018b), the effectiveness of PMIC was compared to AIC, AICc and

BIC, where, overall, PMIC was found to perform better in model structure selection, where it can

select the true model in the form of ARX and NARX models for given datasets.

The PMIC2 was developed from the approach of using PMIC. In order to overcome the hassle of

determining the suitable penalty function parameter of PMIC, it modifies the bias term or known as

penalty function, and begins as follows:

2 (2)

where RSS is the residual sum of square or defined as the maximised value of the likelihood function

for the estimated model. It can be written as follows:

∑

∑

(3)

101

The basic theory of the penalty function for PMIC and PMIC2 revolves around the consideration that

the magnitude of parameter could have a big role in choosing whether a term is significant enough to

be included in a model. This assists one's judgment in choosing or discarding a term / variable. In

PMIC2:

Considering, ⋯

One should give: ⋯

where θ is the value of parameter in a model and is the value of penalty applied for having the

variable / term associated with the parameter. Applying a big penalty for having a variable / term that

has small parameter magnitude may cause the model to be unfavourable in comparison to other

models. In other words, the variable / term that was penalised is said to be insignificant to the model’s

accuracy in comparison to a variable / term that has big parameter magnitude. One way to realise this

is using the following equation:

(4)

Hence, the equation of PMIC2 is:

2∑

∑

(5)

where and are the k-step-ahead predicted output and actual output value at time

respectively, is the number of data, is the magnitude of parameter in the model, and j is the

number of parameters.

3. GENETIC ALGORITHM (GA)

Among all methods in evolutionary computation, GA is probably the most widely known. Its

application is recorded in various fields, including image processing, pattern recognition, operational

research, biology and computer sciences (see e.g., Haupt and Haupt, 2004, Kumar et al., 2010). There

are three main characteristics of GA (Holland, 1992, Eshelman, 2000):

i. The algorithm uses binary bit string representation.

ii. The selection method used is fitness-proportional selection.

iii. It uses crossover as its main genetic search operator.

Other genetic operators are mutation, which is used as a ‘background operator’ to prevent loss of

important gene information or allele, and inversion, which is particularly significant for permutation-

based coding. The outline of the algorithm was initially known as reproductive plan using genetic

operators (Holland, 1973).

The number of strings is known as the population size, typically denoted popsize. Each chromosome

consists of genes separated by different positions also defined as locus. The genes carry information

of the chromosome. In the simulation, the chromosomes represent different model structures and are

evaluated based on a specified information criterion. Based on the evaluation, the selection of highly

fit individuals and genetic manipulation stage, that includes the crossover and mutation then takes

place. The selection and genetic manipulation of these chromosomes are usually performed to a fixed

number of generations, denoted maxgen. Other related terminologies can be referred in Haupt &

Haupt (2004) and Bäck & Fogel (2000). The flow chart of a simple GA is given in Figure 1.

102

Figure 1: Flow chart of a simple GA.

4. MODEL VALIDATION TESTS

Once the final model is identified, the final step of system identification, which is model validation, is

carried out. A model can only be accepted as valid once it is proven that the selected terms and / or

variables do not contribute bias to its accuracy. The correlation tests were used to ensure that no other

significant terms and / or variables were omitted from the model. Nonlinear models require more tests

than linear models since nonlinear models contain polynomials of variables. The tests are as follows

(Billings & Voon, 1986):

i. Autocorrelation of residuals:

(6)

where is the Kronecker delta such that 01and 0, for 0.

ii. Cross correlation of residual with input:

0,∀ (7)

iii.

0,0 (8)

iv.

0,∀ (9)

v.

0,∀ (10)

no

yes Output

solution

Initialiseandevaluate

p

o

p

ulation

Reproduceselectedparents

Crossoverandmutate

Evaluatenewsolutions

Updatepopulation

Termination

criterion

reached?

103

where is the input, is the lag order and ∙ is the expectation operator that can be calculated

based on the formula

∑̅

(11)

and

with set to zero. The residual, , is calculated by the following:

(12)

with as the actual output and as the predicted output. The overbar denotes the time average

so that is given by:

∑

(13)

In all the tests above, the accepted bandwidth for a model’s fit to a system is approximately 1.96/

√ when allowed 95% confidence interval with as the number of data points.

5. SIMULATION SETUP

In this section, PMIC2 was tested with real data, in this case gas furnace data, through simulation

using MATLAB software. This input-output data was described as an actual process plant data

available in Jenkins & Watts (1968) and Box et al. (1994), consisting of a discrete stochastic input

series of gas feed rate in ft3/min and output series of carbon dioxide concentration in outlet gas. There

were 296 pairs of input-output data sampled at an interval of 9 s. In Jenkins & Watts (1968) and Box

et al. (1994), the process was found to be adequately represented by a second-order input and output

lags, but not tested using correlation tests. In Jamaluddin et al. (2007), PMIC was used for this data

set to choose a mathematical model to represent the gas furnace system and the model was also tested

using correlation tests.

To allow NARX modelling, the following specification, which was recommended in Ahmad et al.

(2004), was used in this study: output lag, = 2; input lag, = 2; and nonlinearity, = 2. With this

specification, the number of regressors amounts to 15 and the search space has 32,767 solution points.

The least squares method was used as parameter estimation method.

The specification of the GA is fixed where the population size, with popsize set to 500; the maximum

generation is 100; the mutation probability, pm = 0.01; and the crossover probability, pc = 0.6. The

algorithm implements roulette-wheel selection, single-point crossover and binary bit mutation (Bäck

et al., 2000).

For model validation test, the model selected by PMIC2 was tested using correlation tests. All five

tests listed in Section 4 were made.

6. RESULTS AND DISCUSSION

From the simulation test using GA, PMIC2 selected the model, represented in binary string as [111

010 000 000 000] as the best chromosome. Each bit in the string represents a specific variable or term.

By knowing the sequence of variable/term in the simulation program, the selected regressor may be

easily be traced back when bit 1 is found. Other than the calculated direct current level (which is

equivalent to a constant), the variables selected are 1, 2 and 2. From

Jamaluddin et al. (2007) and Samad (2017), PMIC selected the model [111 010 010 000 000] as the

104

best chromosome. The selected regressors and its parameter values selected by both PMIC2 and

PMIC are provided in Table 1. Comparisons could be made directly, where it shows that PMIC2

selected a more parsimonious model than PMIC. PMIC2 selected a model with 4 out of 15 regressors,

while PMIC selected more regressors, which is 5 out of 15 regressors. As can be seen from the table,

a nonlinear term, 11, is also selected by PMIC. With this additional term, considering

that a control system is to be developed for the gas furnace system, additional control plan needs to be

developed, which in turn, will require higher cost and thus undesirable. The model selected by PMIC2

promises simpler control plan as there are less variables to be controlled.

Table 1: Variables, terms and parameter values of selected model by PMIC2 and PMIC for gas furnace

data.

Variable / Terms Parameter Value

using PMIC2 Parameter Value

using PMIC

Constant or d. c. level 6.4103 7.1795

1 1.5073 1.4449

2 -0.6274 -0.5793

2 -0.3790 -0.6600

11 - 0.0047

Figure 2 shows the result of correlation tests on the model selected by PMIC2. It seems to be

acceptable where only a few points were out of the interval. Compared to the correlation test for a

model selected by PMIC in Figure 3 (Samad, 2017), it looks quite similar for all tests. According to

Jamaluddin et al. (2004), these validation deficiencies may be inherent from wrong selection of lag

orders or nonlinearity of the model. It is also plausible that based on the similarities of the situation to

the heat exchanger problem in Billings & Fadzil (1985), the deficiencies may be caused by lack of

noise terms.

Figure 2: Correlation tests of selected model by PMIC2 using GA for gas furnace data.

105

Figure 3: Correlation tests of selected model by PMIC using GA for gas furnace data

7. CONCLUSION

From the results, PMIC2 selected a more parsimonious model for gas furnace data than PMIC. This

was also achieved without having to try all possible models, but by utilising GA. The results of

correlation tests for the model seem good. It must be emphasised that PMIC2 overcomes the hassle of

selecting the suitable value of penalty function parameter before use in PMIC. Together with other

studies done using PMIC2 on simulated models, PMIC2 can be considered as a reliable information

criterion for model structure selection of discrete-time model. Future work shall be concentrated on

PMIC2 use for other real data, datasets with specifications of bigger search space and different forms

of discrete-time modelling such as linear, and nonlinear autoregressive moving average with

exogenous variable.

ACKNOWLEDGEMENT

The authors would like to acknowledge the support from Universiti Teknikal Malaysia Melaka and

Ministry of Higher Education Malaysia for research grant FRGS/1/2015/TK03/FKM/02/F000271.

REFERENCES

Ahmad, R., Jamaluddin, H. & Hussain, M.A. (2004). Selection of a Model Structure in System

Identification Using Memetic Algorithm. Proc. 2nd Int. Conf. Artificial Intelligence and

Engineering Technology. Aug 3-5. Universiti Malaysia Sabah. 714-720.

Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. Automat. Contr.,

19: 667-674.

Alfi, A. & Fateh, M.M. (2010). Parameter identification based on a modified PSO applied to

suspension system. J. Softw. Eng. Appl., 3: 221-229.

Anderson, D.R., Burnham K.P. & White, G.C. (1994) AIC model selection in overdispersed capture–

recapture data. Ecology, 75: 1780-1793.

Bäck, T, Fogel, D.B. & Michalewicz, Z. (Eds.) (2000). Evolutionary Computation 1: Basic

Algorithms and Operators. Institute of Physics Publishing.

106

Bäck, T. & Fogel, D.B. (2000). Glossary. In Bäck, T, Fogel, D. B. and Michalewicz, Z. (Eds.).

Evolutionary Computation 1: Basic Algorithms and Operators. Institute of Physics Publishing,

Bristol, pp. xxi-xxxvii.

Billings, S.A. & Fadzil, M.B. (1985). The Practical Identification of Systems with Nonlinearities.

Proc. 7th IFAC Symp. Identification System and Parameter Estimation. York, United Kingdom,

155-160.

Billings, S.A. & Voon, W.S.F. (1986). Correlation Based Model Validity Tests for Non-Linear

Models. Int. J. Contr., 44: 235-244.

Box, G.E.P., Jenkins, G.M. & Reinsel, G.C. (1994). Time Series Analysis: Forecasting and Control,

3rd Ed. Prentice-Hall Englewood Cliffs, New Jersey.

Eshelman, L.J. (2000). Genetic Algorithms. In Bäck, T, Fogel, D. B. and Michalewicz, Z. (Eds.),

Evolutionary Computation 1: Basic Algorithms and Operators, Institute of Physics Publishing,

Bristol, pp. 64-80.

Haupt, R.L. & Haupt, S.E. (2004). Practical Genetic Algorithms, 2nd Ed., John Wiley and Sons,

Hoboken, New Jersey.

Holland, J.H. (1973). Genetic Algorithms and the Optimal Allocation of Trials. SIAM J. Comput., 2:

88-105.

Holland, J.H. (1992). Adaptation in Natural and Artificial Systems. MIT Press, Massachusetts.

Jamaluddin, H., Samad, M.F.A., Ahmad, R. & Yaacob, M.S. (2007). Optimum Grouping in a

modified genetic algorithm for discrete-time, non-linear system identification. Proc. IMechE

Part I: J. Syst. Contr. Eng.: 221: 975-989.

Jenkins, G.M. & Watts, D.G. (1968). Spectral Analysis and Its Applications. Holden-Day, San

Francisco.

Kumar, M., Husian, M., Upreti N. & Gupta D. (2010). Genetic Algorithm: Review and Application.

Int. J. Inf. Tech. Knowl. Manage., 2: 451-454.

Ljung, L. (1999). System Identification: Theory for the User, 2nd Ed., Prentice Hall, Upper Saddle

River, New Jersey.

Prando, G., Pillonetto G. & Chiuso, A. (2015). The role of rank penalties in linear system

identification. IFAC-Papers Online, 48: 1293-1300.

Riddef', F.D., Pintelon, R., Schoukens, J. & Gillikinb, D.P. (2004). Modified AIC and MDL model

selection criteria for short data records, IEEE Trans. Instrum. Meas., 54: 144-150.

Samad, M.F.A (2017). Applications of Evolutionary Computation in System Identification. UTeM

Press, Universiti Teknikal Malaysia Melaka (UTeM), Malaysia.

Samad, M.F.A. & Nasir, A.R.M. (2017a). Parameter-magnitude based information criterion for

identification of linear discrete-time model. Proc. Innovative Res. Industrial Dialogue’16. 129-

130.

Samad, M.F.A. & Nasir, A.R.M. (2017b). Parameter magnitude-based information criterion in

identification of discrete-time dynamic system. J. Mech. Eng., 4: 119-128.

Samad, M.F.A. & Nasir, A.R.M. (2017c). Discrete-time system identification based on novel

information criterion using genetic algorithm. J. Fundam. Appl. Sci., 9: 584-599.

Samad, M.F.A. & Nasir, A.R.M. (2018a). Performance of parameter-magnitude based information

criterion in identification of linear discrete-time model. Commun. Math. Appl., to be published.

Samad, M.F.A. & Nasir, A.R.M. (2018b). Comparison of information criterion on identification of

discrete-time dynamic system. J. Eng. Appl. Sci. 12: 5660-5665.

Samad, M.F.A., Jamaluddin, H., Ahmad, R., Yaacob M.S. & Azad, A.K.M. (2013). Effect of penalty

function parameter in objective function of system identification. Int. J. Automot. Mech. Eng., 7:

940-954.

Schwartz, G. (1978). Estimating the dimension of a model. Annals Stat., 6: 461-464.

Zhang, L.F., Zhu Q. M. & Longden A. (2005). Nonlinear model validation using novel correlation

tests. 2005 IEEE Int. C. Syst. Man Cybern., 3: 2879-2884.