VOL. 11 NUM. 1 YEAR 2018 ISSN
Combating Corrosion: Risk Identification, Mitigation and Management
Mahdi Che Isa, Abdul Rauf Abdul Manaf & Mohd Hambali Anuar 1 - 12
Mechanical Properties Extraction of Composite Material Using Digital Image Correlation via
Open Source NCorr
Ahmad Fuad Ab Ghani, Jamaluddin Mahmud, Saiful Nazran & Norsalim Muhammad
13 - 24
Effects of Mapping on the Predicted Crash Response of Circular Cup-Shape Part
Rosmia Mohd Amman, Sivakumar Dhar Malingam, Ismail Abu-Shah & Mohd Faizal Halim 25 - 35
Effect of Nickle Foil Width on the Generated Wave Mode from a Magnetostrictive Sensor
Nor Salim Muhammad, Ayuob Sultan Saif Alnadhari, Roszaidi Ramlan, Reduan Mat Dan,
Ruztamreen Jenal & Mohd Khairi Mohamed Nor
36 - 48
Rapid Defect Screening on Plate Structures Using Infrared Thermography
Nor Salim Muhammad, Abd Rahman Dullah, Ahmad Fuad Ad Ghani, Roszaidi Ramlan &
49 - 56
Optimisation of Electrodeposition Parameters on the Mechanical Properties of Nickel Cobalt
Coated Mild Steel
Nik Hassanuddin Nik Yusoff, Othman Mamat, Mahdi Che Isa & Norlaili Amir
57 - 65
Preparation and Characterization of PBXN-109EB as a New High Performance Plastic Bonded
Mahdi Ashrafi, Hossein Fakhraian, Ahmad Mollaei & Seyed Amanollah Mousavi Nodoushan
66 - 76
Nonlinear ROV Modelling and Control System Design Using Adaptive U-Model, FLC and PID
Nur Afande Ali Hussain, Syed Saad Azhar Ali, Mohamad Naufal Mohamad Saad & Mark Ovinis
77 - 89
Flight Simulator Information Support
Vladimir R. Roganov, Elvira V. Roganova, Michail J. Micheev, Tatyana V. Zhashkova, Olga A.
Kuvshinova & Svetlan M. Gushchin
90 - 98
Implementation of Parameter Magnitude-Based Information Criterion in Identification of a Real
Md Fahmi Abd Samad & Abdul Rahman Mohd Nasir
99 - 106
Design Method for Distributed Adaptive Systems Providing Data Security for Automated
Process Control Systems
Aleksei A. Sychugov & Dmitrii O. Rudnev
107 - 112
Low Contrast Image Enhancement Using Renyi Entropy
Vijayalakshmi Dhurairajan,Teku Sandhya Kumari & Chekka Anitha Bhavani 113 - 122
Determination of Artificial Recharge Locations Using Fuzzy Analytic Hierarchy Process (AHP)
Marzieh Mokarram & Dinesh Sathyamoorthy 123 - 131
Availability Oriented Contract Management Approach: A Simplified View to a Complex Naval
Al-Shafiq Abdul Wahid, Mohd ZamaniAhmad, Khairol Amali Ahmad & Aisha Abdullah
132 - 153
Ministry of Defence
SCIENCE & TECHNOLOGY RESEARCH INSTITUTE
FOR DEFENCE (STRIDE)
IMPLEMENTATION OF PARAMETER MAGNITUDE-BASED INFORMATION
CRITERION IN IDENTIFICATION OF A REAL SYSTEM
Md Fahmi Abd Samad* & Abdul Rahman Mohd Nasir
Faculty of Mechanical Engineering, Universiti Teknikal Malaysia Melaka (UTeM), Malaysia
Model structure selection is one among crucial steps in system identification and in order to carry out
this, an information criterion is needed. It plays an important role in determining an optimum model
structure with the aim of selecting an adequate model to represent a real system. A good information
criterion should not only evaluate predictive accuracy but also the parsimony of the model. In the
past, there had not been, or scarcely have been, any information criterion that evaluates the
parsimony of model structures (bias contribution) based on the magnitude of parameter or coefficient.
However, recently, some efforts had been made that took into account such strategy and proved the
criterion to be effective for simulated datasets. This paper presents the comparison between two
information criteria that are based on parameter magnitude information in selecting a good model to
represent a real system based on gas furnace. Genetic algorithm (GA) was used to optimise the
implementation. The selected models were then tested using correlation tests for model validation. It
is shown that parameter magnitude based information criterion 2 (PMIC2) is able to select a more
parsimonious model than PMIC but with similar validation results.
Keywords: Parameter magnitude; information criterion; system identification; linear regressive
model; genetic algorithm (GA).
In many fields, nonlinear dynamic modelling is applied in approximating a wide range of systems.
Basically, system identification is used for estimating a model to represent their systems. System
identification can be considered a regression problem, where the relationship between input and
output variables of a dynamical system has to be estimated. This task is typically accomplished by
minimising a certain information criterion, which measures how well the estimated relationship
approximates the one that truly links the available input-output data pairs (Ljung, 1999). Its basic idea
is to compare the time dependent responses of the actual system and identified model based on a
performance function, hereby referred to as information criterion, giving a measure of how well the
model response fits the system response (Alfi & Fateh, 2010).
An identification procedure typically consists of estimating the parameters of different models, and
next, selecting the optimal model complexity within that set. Increasing the model complexity will
decrease the systematic errors, but, at the same time, the model variability increases (Riddef et al.,
2004). Model accuracy and parsimony, known as variance and bias: , are
important considerations in selecting a model structure (Ljung, 1999). Hence, selecting a model with
the smallest variance is not a good idea because when the number of parameters increase, the variance
will continue to decrease but will present a complex model. At a certain complexity, the additional
parameters no longer reduce the systematic errors but are used to follow the actual noise realisation on
the data. Often, in order to deal with the bias-variance trade-off, the information criterion is
augmented with a penalty term intended to guide the search for the “optimal” relationship penalising
undesired regressors, where regressors refer to possible terms and variables. Regularised estimation
has been widely applied in the context of system identification (Prando et al., 2015). Model validation
is the final step of system identification. It is used to check the goodness of the estimated model
(Zhang et al., 2005).
In Samad & Nasir (2017a, b, c, 2018a), parameter magnitude-based information criterion 2 (PMIC2)
was tested on simulated datasets generated from autoregressive with exogenous input (ARX) and
nonlinear autoregressive with exogenous input (NARX) models. Comparisons were made between
PMIC2, Akaike information criterion (AIC) (Akaike, 1974), Bayesian information criterion (BIC)
(Schwartz, 1978) and corrected Akaike information criterion (AICc) (Anderson et al., 1994). In the
simulation results, PMIC2 proved that it performed better than AIC, BIC and AICc for all the
simulated models (ARX and NARX). In this paper, the effectiveness of PMIC2 will be studied by
testing on real system data, in this case, gas furnace data. The models were generated as NARX
models. Since there are many possible models to test, genetic algorithm (GA) was used to optimise
the search. The models selected by PMIC2 were tested by correlation tests for model validation. The
results of PMIC2 were then compared to another parameter-magnitude based information criterion
(Samad et al., 2013), hereby denoted PMIC, in order to evaluate the performance between both
The next sections are as follows: Section 2 introduces information criteria; Section 3 explains about
GA, which is used as part of the search for model structure; Section 4 explains the model validation
tests; Section 5 provides simulation setup; Section 6 presents result and discussions; and lastly,
Section 7 concludes the findings of this paper.
2. INFORMATION CRITERIA
According to Samad et al. (2013), parameter magnitude-based information criterion (PMIC) evaluates
the bias contribution by the sum of squared residuals while the variance contribution is calculated by a
penalty function (PF). This is written as follows:
where PF =ln, is the number of terms satisfying 1 while represents the
absolute value of the parameter for term and is a fixed value termed penalty function
parameter. The penalty function penalises terms with the absolute values of the estimated parameter
less than the penalty. The penalty parameter value will be set equal to or slightly lower than the
parameter value in cases where the smallest tolerable absolute parameter value is known or can be
By referring to Samad & Nasir (2018b), the effectiveness of PMIC was compared to AIC, AICc and
BIC, where, overall, PMIC was found to perform better in model structure selection, where it can
select the true model in the form of ARX and NARX models for given datasets.
The PMIC2 was developed from the approach of using PMIC. In order to overcome the hassle of
determining the suitable penalty function parameter of PMIC, it modifies the bias term or known as
penalty function, and begins as follows:
where RSS is the residual sum of square or defined as the maximised value of the likelihood function
for the estimated model. It can be written as follows:
The basic theory of the penalty function for PMIC and PMIC2 revolves around the consideration that
the magnitude of parameter could have a big role in choosing whether a term is significant enough to
be included in a model. This assists one's judgment in choosing or discarding a term / variable. In
One should give: ⋯
where θ is the value of parameter in a model and is the value of penalty applied for having the
variable / term associated with the parameter. Applying a big penalty for having a variable / term that
has small parameter magnitude may cause the model to be unfavourable in comparison to other
models. In other words, the variable / term that was penalised is said to be insignificant to the model’s
accuracy in comparison to a variable / term that has big parameter magnitude. One way to realise this
is using the following equation:
Hence, the equation of PMIC2 is:
where and are the k-step-ahead predicted output and actual output value at time
respectively, is the number of data, is the magnitude of parameter in the model, and j is the
number of parameters.
3. GENETIC ALGORITHM (GA)
Among all methods in evolutionary computation, GA is probably the most widely known. Its
application is recorded in various fields, including image processing, pattern recognition, operational
research, biology and computer sciences (see e.g., Haupt and Haupt, 2004, Kumar et al., 2010). There
are three main characteristics of GA (Holland, 1992, Eshelman, 2000):
i. The algorithm uses binary bit string representation.
ii. The selection method used is fitness-proportional selection.
iii. It uses crossover as its main genetic search operator.
Other genetic operators are mutation, which is used as a ‘background operator’ to prevent loss of
important gene information or allele, and inversion, which is particularly significant for permutation-
based coding. The outline of the algorithm was initially known as reproductive plan using genetic
operators (Holland, 1973).
The number of strings is known as the population size, typically denoted popsize. Each chromosome
consists of genes separated by different positions also defined as locus. The genes carry information
of the chromosome. In the simulation, the chromosomes represent different model structures and are
evaluated based on a specified information criterion. Based on the evaluation, the selection of highly
fit individuals and genetic manipulation stage, that includes the crossover and mutation then takes
place. The selection and genetic manipulation of these chromosomes are usually performed to a fixed
number of generations, denoted maxgen. Other related terminologies can be referred in Haupt &
Haupt (2004) and Bäck & Fogel (2000). The flow chart of a simple GA is given in Figure 1.
Figure 1: Flow chart of a simple GA.
4. MODEL VALIDATION TESTS
Once the final model is identified, the final step of system identification, which is model validation, is
carried out. A model can only be accepted as valid once it is proven that the selected terms and / or
variables do not contribute bias to its accuracy. The correlation tests were used to ensure that no other
significant terms and / or variables were omitted from the model. Nonlinear models require more tests
than linear models since nonlinear models contain polynomials of variables. The tests are as follows
(Billings & Voon, 1986):
i. Autocorrelation of residuals:
where is the Kronecker delta such that 01and 0, for 0.
ii. Cross correlation of residual with input:
where is the input, is the lag order and ∙ is the expectation operator that can be calculated
based on the formula
with set to zero. The residual, , is calculated by the following:
with as the actual output and as the predicted output. The overbar denotes the time average
so that is given by:
In all the tests above, the accepted bandwidth for a model’s fit to a system is approximately 1.96/
√ when allowed 95% confidence interval with as the number of data points.
5. SIMULATION SETUP
In this section, PMIC2 was tested with real data, in this case gas furnace data, through simulation
using MATLAB software. This input-output data was described as an actual process plant data
available in Jenkins & Watts (1968) and Box et al. (1994), consisting of a discrete stochastic input
series of gas feed rate in ft3/min and output series of carbon dioxide concentration in outlet gas. There
were 296 pairs of input-output data sampled at an interval of 9 s. In Jenkins & Watts (1968) and Box
et al. (1994), the process was found to be adequately represented by a second-order input and output
lags, but not tested using correlation tests. In Jamaluddin et al. (2007), PMIC was used for this data
set to choose a mathematical model to represent the gas furnace system and the model was also tested
using correlation tests.
To allow NARX modelling, the following specification, which was recommended in Ahmad et al.
(2004), was used in this study: output lag, = 2; input lag, = 2; and nonlinearity, = 2. With this
specification, the number of regressors amounts to 15 and the search space has 32,767 solution points.
The least squares method was used as parameter estimation method.
The specification of the GA is fixed where the population size, with popsize set to 500; the maximum
generation is 100; the mutation probability, pm = 0.01; and the crossover probability, pc = 0.6. The
algorithm implements roulette-wheel selection, single-point crossover and binary bit mutation (Bäck
et al., 2000).
For model validation test, the model selected by PMIC2 was tested using correlation tests. All five
tests listed in Section 4 were made.
6. RESULTS AND DISCUSSION
From the simulation test using GA, PMIC2 selected the model, represented in binary string as [111
010 000 000 000] as the best chromosome. Each bit in the string represents a specific variable or term.
By knowing the sequence of variable/term in the simulation program, the selected regressor may be
easily be traced back when bit 1 is found. Other than the calculated direct current level (which is
equivalent to a constant), the variables selected are 1, 2 and 2. From
Jamaluddin et al. (2007) and Samad (2017), PMIC selected the model [111 010 010 000 000] as the
best chromosome. The selected regressors and its parameter values selected by both PMIC2 and
PMIC are provided in Table 1. Comparisons could be made directly, where it shows that PMIC2
selected a more parsimonious model than PMIC. PMIC2 selected a model with 4 out of 15 regressors,
while PMIC selected more regressors, which is 5 out of 15 regressors. As can be seen from the table,
a nonlinear term, 11, is also selected by PMIC. With this additional term, considering
that a control system is to be developed for the gas furnace system, additional control plan needs to be
developed, which in turn, will require higher cost and thus undesirable. The model selected by PMIC2
promises simpler control plan as there are less variables to be controlled.
Table 1: Variables, terms and parameter values of selected model by PMIC2 and PMIC for gas furnace
Variable / Terms Parameter Value
using PMIC2 Parameter Value
Constant or d. c. level 6.4103 7.1795
1 1.5073 1.4449
2 -0.6274 -0.5793
2 -0.3790 -0.6600
11 - 0.0047
Figure 2 shows the result of correlation tests on the model selected by PMIC2. It seems to be
acceptable where only a few points were out of the interval. Compared to the correlation test for a
model selected by PMIC in Figure 3 (Samad, 2017), it looks quite similar for all tests. According to
Jamaluddin et al. (2004), these validation deficiencies may be inherent from wrong selection of lag
orders or nonlinearity of the model. It is also plausible that based on the similarities of the situation to
the heat exchanger problem in Billings & Fadzil (1985), the deficiencies may be caused by lack of
Figure 2: Correlation tests of selected model by PMIC2 using GA for gas furnace data.
Figure 3: Correlation tests of selected model by PMIC using GA for gas furnace data
From the results, PMIC2 selected a more parsimonious model for gas furnace data than PMIC. This
was also achieved without having to try all possible models, but by utilising GA. The results of
correlation tests for the model seem good. It must be emphasised that PMIC2 overcomes the hassle of
selecting the suitable value of penalty function parameter before use in PMIC. Together with other
studies done using PMIC2 on simulated models, PMIC2 can be considered as a reliable information
criterion for model structure selection of discrete-time model. Future work shall be concentrated on
PMIC2 use for other real data, datasets with specifications of bigger search space and different forms
of discrete-time modelling such as linear, and nonlinear autoregressive moving average with
The authors would like to acknowledge the support from Universiti Teknikal Malaysia Melaka and
Ministry of Higher Education Malaysia for research grant FRGS/1/2015/TK03/FKM/02/F000271.
Ahmad, R., Jamaluddin, H. & Hussain, M.A. (2004). Selection of a Model Structure in System
Identification Using Memetic Algorithm. Proc. 2nd Int. Conf. Artificial Intelligence and
Engineering Technology. Aug 3-5. Universiti Malaysia Sabah. 714-720.
Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. Automat. Contr.,
Alfi, A. & Fateh, M.M. (2010). Parameter identification based on a modified PSO applied to
suspension system. J. Softw. Eng. Appl., 3: 221-229.
Anderson, D.R., Burnham K.P. & White, G.C. (1994) AIC model selection in overdispersed capture–
recapture data. Ecology, 75: 1780-1793.
Bäck, T, Fogel, D.B. & Michalewicz, Z. (Eds.) (2000). Evolutionary Computation 1: Basic
Algorithms and Operators. Institute of Physics Publishing.
Bäck, T. & Fogel, D.B. (2000). Glossary. In Bäck, T, Fogel, D. B. and Michalewicz, Z. (Eds.).
Evolutionary Computation 1: Basic Algorithms and Operators. Institute of Physics Publishing,
Bristol, pp. xxi-xxxvii.
Billings, S.A. & Fadzil, M.B. (1985). The Practical Identification of Systems with Nonlinearities.
Proc. 7th IFAC Symp. Identification System and Parameter Estimation. York, United Kingdom,
Billings, S.A. & Voon, W.S.F. (1986). Correlation Based Model Validity Tests for Non-Linear
Models. Int. J. Contr., 44: 235-244.
Box, G.E.P., Jenkins, G.M. & Reinsel, G.C. (1994). Time Series Analysis: Forecasting and Control,
3rd Ed. Prentice-Hall Englewood Cliffs, New Jersey.
Eshelman, L.J. (2000). Genetic Algorithms. In Bäck, T, Fogel, D. B. and Michalewicz, Z. (Eds.),
Evolutionary Computation 1: Basic Algorithms and Operators, Institute of Physics Publishing,
Bristol, pp. 64-80.
Haupt, R.L. & Haupt, S.E. (2004). Practical Genetic Algorithms, 2nd Ed., John Wiley and Sons,
Hoboken, New Jersey.
Holland, J.H. (1973). Genetic Algorithms and the Optimal Allocation of Trials. SIAM J. Comput., 2:
Holland, J.H. (1992). Adaptation in Natural and Artificial Systems. MIT Press, Massachusetts.
Jamaluddin, H., Samad, M.F.A., Ahmad, R. & Yaacob, M.S. (2007). Optimum Grouping in a
modified genetic algorithm for discrete-time, non-linear system identification. Proc. IMechE
Part I: J. Syst. Contr. Eng.: 221: 975-989.
Jenkins, G.M. & Watts, D.G. (1968). Spectral Analysis and Its Applications. Holden-Day, San
Kumar, M., Husian, M., Upreti N. & Gupta D. (2010). Genetic Algorithm: Review and Application.
Int. J. Inf. Tech. Knowl. Manage., 2: 451-454.
Ljung, L. (1999). System Identification: Theory for the User, 2nd Ed., Prentice Hall, Upper Saddle
River, New Jersey.
Prando, G., Pillonetto G. & Chiuso, A. (2015). The role of rank penalties in linear system
identification. IFAC-Papers Online, 48: 1293-1300.
Riddef', F.D., Pintelon, R., Schoukens, J. & Gillikinb, D.P. (2004). Modified AIC and MDL model
selection criteria for short data records, IEEE Trans. Instrum. Meas., 54: 144-150.
Samad, M.F.A (2017). Applications of Evolutionary Computation in System Identification. UTeM
Press, Universiti Teknikal Malaysia Melaka (UTeM), Malaysia.
Samad, M.F.A. & Nasir, A.R.M. (2017a). Parameter-magnitude based information criterion for
identification of linear discrete-time model. Proc. Innovative Res. Industrial Dialogue’16. 129-
Samad, M.F.A. & Nasir, A.R.M. (2017b). Parameter magnitude-based information criterion in
identification of discrete-time dynamic system. J. Mech. Eng., 4: 119-128.
Samad, M.F.A. & Nasir, A.R.M. (2017c). Discrete-time system identification based on novel
information criterion using genetic algorithm. J. Fundam. Appl. Sci., 9: 584-599.
Samad, M.F.A. & Nasir, A.R.M. (2018a). Performance of parameter-magnitude based information
criterion in identification of linear discrete-time model. Commun. Math. Appl., to be published.
Samad, M.F.A. & Nasir, A.R.M. (2018b). Comparison of information criterion on identification of
discrete-time dynamic system. J. Eng. Appl. Sci. 12: 5660-5665.
Samad, M.F.A., Jamaluddin, H., Ahmad, R., Yaacob M.S. & Azad, A.K.M. (2013). Effect of penalty
function parameter in objective function of system identification. Int. J. Automot. Mech. Eng., 7:
Schwartz, G. (1978). Estimating the dimension of a model. Annals Stat., 6: 461-464.
Zhang, L.F., Zhu Q. M. & Longden A. (2005). Nonlinear model validation using novel correlation
tests. 2005 IEEE Int. C. Syst. Man Cybern., 3: 2879-2884.