An Artificial Neural-Network-Based Approach to Software Reliability Assessment

Abstract

In this paper, we propose an artificial neural- network-based approach for software reliability estimation and modeling. We first explain the network networks from the mathematical viewpoints of software reliability modeling. That is, we will show how to apply neural network to predict software reliability by designing different elements of neural networks. Furthermore, we will use the neural network approach to build a dynamic weighted combinational model. The applicability of proposed model is demonstrated through four real software failure data sets. From experimental results, we can see that the proposed model significantly outperforms the traditional software reliability models.

Full text

Page 1

An Artificial Neural-Network-Based Approach to
Software Reliability Assessment

Yu-Shen Su, Chin-Yu Huang, Yi-Shin Chen, and Jing-Xun Chen
Department of Computer Science
National Tsing Hua University
Hsinchu, Taiwan

Abstract—In this paper, we propose an artificial neural-
network-based approach for software reliability estimation
and modeling. We first explain the network networks from
the mathematical viewpoints of software reliability modeling.
That is, we will show how to apply neural network to predict
software reliability by designing different elements of neural
networks. Furthermore, we will use the neural network
approach to build a dynamic weighted combinational model.
The applicability of proposed model is demonstrated through
four real software failure data sets. From experimental
results, we can see that the proposed model significantly
outperforms the traditional software reliability models.
Index-term—Neural Network, Software Testing, Software
Reliability, Combinational Model.
I. INTRODUCTION

different applications, such as nuclear reactors, aircraft,
banking systems, and hospital patient monitoring systems.
As the demand of the application quality becomes higher
and higher, the research of the computer software
reliability becomes more and more essential. The software
reliability is defined as the probability that the software
will operate without a failure under a given environmental
condition during a specified period of time [1]. To date,
the software reliability assessment is one of the most
important processes during the software development.
Since 1970, many software reliability growth models
(SRGMs) [2-4] have been proposed. In general, there are
two major types of software reliability models: the
deterministic and the probabilistic. The deterministic one
is employed to study the number of distinct operators and
operands in the program. The probabilistic one represents
the failure occurrences and the fault removals as
probabilistic events.
The probabilistic models can be further classified into
different classes, such as error seeding, failure rate, and
non-homogeneous Poisson process (NHPP). Among these
classes, the NHPP models are the most popular ones. The
reason is the NHPP model has ability to describe the
software failure phenomenon. The first NHPP model,
which strongly influences the development of many other
models, was proposed by Goel and Okumoto [6]. Later,
Ohba [7] presented a NHPP model with S-shaped mean
value function. Yamada and Osaki [8, 9] also made further
progress in various S-Shaped NHPP models. Although
these NHPP models are widely used, they impose certain
restrictions or a priori assumptions about the nature of
software faults and the stochastic behavior of software
failure process.
To overcome this problem, several alternative
In modern society, computers are used for many
solutions are introduced. One possible solution is to
employ the neural network, since it can build a model
adaptively from the given data set of failure processes.
Many researchers [10-20] have been successfully adapted
neural networks to software reliability issues. Motivated
by these successful cases, we employ the neural network
to solve the problems for software reliability assessment.
The major contributions of this paper are:
• Derive mathematic expressions that can be applied to
neural networks from traditional software reliability
models.
• Propose a generic model for all software projects. This
proposed model is adaptive based on the characteristic
of the given data set.
The rest of this paper is organized as follows.
Section II gives an overview of neural networks. In
Section III, we present our methodology. Section IV
depicts the results of our evaluation as compared to the
existing SRGMs. Section V concludes the paper.
II. OVERVIEW OF NEURAL NETWORKS

to assess the software reliability. However, since most
software reliability models embed certain restrictions or
assumptions, selecting an appropriate model based on the
characteristics of the software projects is challenging. In
order to locate the suitable model, two approaches are
adapted. The first one is to design a guideline, which could
suggest fitting models for software projects. The other is to
select the one with the highest confidence after various
assessments. Basically, these two approaches both require
users to manually opt for candidates. The decision-making
processes would be a huge overhead while the software
projects are huge and complicated.
In order to reduce such overhead, researchers
proposed an alternative approach that can adapt the
characteristics of failure processes from the actual data set
by using neural networks. For example, Karunaithi et al.
[11] applied some kinds of neural network architecture to
estimate the software reliability and used the execution
time as input, cumulative the number of detected faults as
desired output, and encoded the input and output into the
binary bit string. The results showed that the neural
network approach was good at identifying defect-prone
modules software failures. Khoshgoftaar et al. [14-15]
ever used the neural network as a tool for predicting the
number of faults in programs. They introduced an
approach for static reliability modeling and concluded that
the neural networks produce models with better quality of
fit and predictive quality. In addition, Cai et al. [19]
The software reliability models are widely employed

Page 2

examined the effectiveness of the neural network approach
in handling dynamic software reliability data overall and
present several new findings. They found that the neural
network approach is more appropriate for handling
datasets with `smooth' trends than for handling datasets
with large fluctuations and the training results are much
better than the prediction results in general.
Next, we briefly introduce the concept of the neural
networks. Neural networks are learning mechanisms that
can approximate any non-linear continuous functions
based on the given data. In general, neural networks
consist of three components as follow:
1. Neurons: each neuron can receive signal, process the
signals and finally produce an output signal. Fig. 1
depicts a neuron, where f is the activation function that
processes the input signals and produces an output of
the neuron, x are the outputs of the neurons in the
previous layer, and w are the weights connected to the
neurons of the previous layer.

Fig. 1 A neuron
2. Network architecture: The most common type of
neural network architecture is called feed-forward
network shown in Fig. 2. This architecture is
composed of three distinct layers: an input layer, a
hidden layer, and an output layer. Note that the circles
are represented as neurons and the connection of
neurons across layers is called the connecting weight.

Fig. 2 A feed-forward network
3. Learning algorithm: The algorithm describes a
process to adjust the weights. During the learning
processes, the weights of network are adjusted to
reduce the errors of the network outputs as compared
to the standard answers. The back-propagation
algorithm is the most widely employed one. In
back-propagation algorithm, the weights of the
network are iteratively trained with the errors
propagated back from the output layer.

Subsequently, we describe the learning algorithm of
the neural network in a mathematical form. The objective
of the neural networks is to approximate an non-linear
function that can receive the vector (x1, …, xn) in RN and
output the vector (y1,.., ym) in RM. Thus, the network can be
denoted as:
( )y F x
=
where x=(x1, …, xn) and y= (y1,.., ym). The values of yk are
given by
H
y g b w h
=
where
jk
w is the output “weight” from hidden layer node
j to output layer node k, bk is the bias of the node k, hj is the
output of the hidden layer, and g is an activation function
in output layers. The values of the hidden layer are given
by
N
h f b
=
where
ij
w is the input “weight” from input layer node i to
hidden layer node j, bj is the bias of the node in the hidden
layer, xi is the value at input node i, and f is the activation
function in hidden layer.
The approximated function of the neural networks
can be considered as some compound functions, which can
be rewritten as nested functions, such as f(g(x)). Because
of this feature, the neural network can be applied to
software reliability modeling since software reliability
modeling is likely to build a model to explain the software
failure behavior.
, (1)
0
jk
1
( ),1,,
kkj
j
kM
=+=
∑

, (2)
0
1
ij
1
( ), 1,,
jji
i
w xjH
=+=
∑

, (3)
1
III. A NEURAL-NETWORK-BASED APPROACH
FOR SOFTWARE RELIABILITY MODELING

objective function of the neural network can be considered
as compound functions. In other words, if we can derive a
form of compound functions from the conventional
software reliability models,
neural-network-based model for software reliability. In
Section III.A, the deriving processes are described one by
one. In Section III.B, we will further demonstrate a
generic neural-network-based model that can combine
existing models. The implementation details of this
approach are illustrated in Section III.C.

As we mentioned in the previous section, the
we can build a
A. The Derivations of Proposed Approach in Software
Reliability Modeling
We first consider the logistic growth curve model [3].
This model simply fits the mean value function with a
form of the logistic function. Its mean value function is
given by:
a
m ta
ke−
+
We can derive a form of compound functions
from its mean value function as the following:
Replace k with e-c:
a
m t
kee e
++
Assume that

( )g x bx
=+ , (5)
( ), 0, 0,0
1
bt
bk
=>>>
. (4)

()
( )
111
btc btbt c
+
a
−
a
−
e
−−
===
+
.
c
1x
ix
N x
Input layer Hidden layer Output layer
ih
H
h
1
ij
w
0
jk
w
1y
jy
M
y

I


nx
1
w
2
w
n
w
∑
1x
2x
()
i
f net
Bias b
Activation function
1
n
ijj
j
netbx w
=
= +∑

Page 3

1
e−
+
( )
f x
1
x
=
, (6)
and
( )
m x ax
=
. (7)


Therefore, we can get
( ( ( )))
m f g x
( ())
m f bxc
=+

()()
1
−
()
11
bx c bx c
a
−
m
ee
++
==
++
. (8)

growth curve model is composed of g(x), f(x), and m(x).
Subsequently, we derive the compound functions from the
viewpoints of neural network. Consider the basic
feed-forward network shown in Fig. 3. Note that the
network has only one neuron in each layer and
the weights and
at time t is fed to the input layer, we can derive the
following form.
The input of the hidden layer is:
_ ( ) h in tw t
=
The output of the hidden layer is:
( )( _ h t f h in t
=
where f(x) is the activation function in the hidden layer.
The input of the output layer is

_ ( )y in tw h t
=
The output of the output layer is

( ) ( _y t g y in t
=
where g(x) is the activation function in the output ayer.
( ) h t

This means that the mean value function of logistic
1
11
w ,
0
11
w are
1b,
0b are the biases. When the input, x(t),
1
111
b
+
. (9)
( ))
, (10)
0
110
( )b
+
. (11)
( ))
, (12)

Fig. 3 Feed-forward neural network with single neuron in each layer

assume the activation functions f(x) and g(x) as:

( )
f x
After the derivation above, we find that if we
1
1
x
e−
=
+
,

Furthermore, we remove the bias in the output layer. We
can consequently get

11
( )_ ( )( ) y ty in t w h t
==
( ) g xx
=
.
0


1
111
0
11
(
w t b
0
11
)
( _( ))
1
w
w f h in t
e−+
==
+
. (13)
According to Eq. (13), we have successfully derived the
neural network into a logistic growth curve model. By the
same process, we can derive the neural network into many
other existing models. For example, we can construct a
neural network with an activation function as
the hidden layer,
11
w =b,
11
w =a, and there is no bias in
both hidden layer and output layer. Thus, we have
th
−=1)(
and
1 ()(aty
=
Obviously, it is the Goel-Okumoto (GO) model [6] which
was firstly proposed by Goel and Okumoto and the model
has strongly influenced the development of other models.
We show another example to confirm that neural
network can be applied for software reliability modeling.
x
e−
−
1
in
10
bt
e
−
, (14)
)
bt
e
−
−
. (15)
If we construct a network with an activation function as
1 (1)
x e−
−+
in the hidden layer,
there is no bias in both hidden layer and output layer. Then,
we have
( ) 1 (1
h t
= −
and
( ) (1 (1
y ta
=−
It is the Yamada Delay S-Shaped model [8, 9]. The model
describes the fault detection process as a learning process
in which testing members become familiar with the test
environment or testing tools. In other words, their testing
skills gradually improved.

x
1
11
w =b,
0
11
w =a, and
)
bt
bt e−
+
, (16)
))
bt
bt e−
+
. (17)
B. Extended Modeling

is very important to software reliability assessment. But
sometimes, software projects can not fit the assumptions
of a unique model. To overcome this problem, Lyu and
Allen [21] have proposed a solution by combining the
results of different software reliability models. This
approach inspired us to use the neural-network-based
approach to combine the models. Next, we demonstrate an
application of proposed approach to reach combinational
models.
We construct the neural network with single input
single output but more than one neuron in the hidden layer.
The network is depicted in Fig. 4. Note that we determine
the number of neurons in the hidden layer by the number
of models which assumptions are partially suitable to the
software project. We use different activation functions in
the hidden layer at the same time to achieve combinational
models.
More models selected will result that the
combinational model has more accurate but it will take
much more computation time. In this example, we will try
to combine the GO model, the Delay-S-Shaped, and the
logistic growth curve model. We use1
1
1
e−
+
The output of the network is:
We have mentioned that selecting a particular model
x
e−
−
, 1 (1
−
)
x
x e−
+
,
and
x
as the activation functions in the hidden layer.
1112
13
21
e
212212
( )
y t
(1)(1 (1
−
))
1
w tw t
w
w
weww t e
−−
−
=−+++
+
, (18)
where
12
,
jj
ww are the weights of the network. Note that
w are represented as the weight of each individual
models and they are determined by the training algorithm
(refer to Appendix for the proof of this deduction).
2 j

Fig. 4 the network architecture of the combinational model

characteristic of the given data set. Hence, this model itself
can be considered as a general model for all software
projects. Note that this method differs from those proposed
This combinational model can adapt the
( ) x t
1
11
w
0
11
w
( )y t
x
y
11
w
12
w
13
w
21
w
22
w
23
w
jh

Page 4

by Lyu et al., since their model only combined the results
from various models based on assigned weights. Our
proposed approach automatically determines the weight of
each model based on the characteristic of the given data
set.

C. An Implementation of Proposed Approach

proposed approach is achievable. In this section, the
implementation of the proposed neural-network-based
models will be described. The following steps are required
to apply our approach for software reliability prediction.
1. Analyze the assumptions of the software project and if
there is a suitable model, select the model. If there is no
suitable model, select those models which partially
meet the assumptions of project.
2. Construct the neural network of selected models by
designing the activation functions and bias.
3. Given n fault-detection time interval data x1, x2, …, xn,
we accumulate the execution time and divide it into 100
time units t1, t2, …, t100 , and then calculate the number
of failures, T1, T2, …, T100.
4. Feed the pairs of { ti, Ti } to the network to train the
network by the back-propagation algorithm which is
mentioned in the previous section.
5. When the network trained, feed the future testing time
to the network, and the network output is represented as
the forecasting number of faults in the future.
By these five steps, we can use the neural-network-based
models to assess the software reliability, predict the total
faults to be detected in the future. Note that in the Steps 2,
the activation functions should be designed under some
conditions which are (1) continuous and (2) differentiable
everywhere, since we use the back-propagation algorithm
in the Steps 4. And we will prove in the appendix that the
activations which are used in the paper are satisfied these
conditions.
As mentioned eariler, we have shown that the
IV.PROFORMANCE EVALUATION

approach, we use a real software failure data set. The data
set was collected by John D. Musa of Bell Telephone
Laboratories for a Real Time Command & Control (S1)
which consists 21,700 lines of code (LOCs). During the
testing phase, 136 faults were detected [2]. We first
analyze the data set by using Laplace trend test [22]
because software reliability studies are usually based on
the application of growth models to obtain various
measures.
Let the time interval [0, t] be divided into k units of
time, and let n(i) be the number of faults observed during
time unit i. The Laplace factor, u(k), is given by
k
in i
u k
k
−
To validate the performances of our proposed
11
2
1
( 1)
(1) ( )( )
2
( )
1
( )
12
k
ii
k
i
K
n i
n i
==
=
−
−−
=
∑∑
∑
. (19)
Note that negative value of u(k) indicates decreasing
failure intensity, and thus reliability growth, while positive
value indicates increasing failure intensity, and reliability
decrease. Fig. 6 shows the Laplace trend test for the S1
data set. We can find that the reliability of this system is
growth. If the trend of the data set is shown as reliability
growth, then this application of data set is suitable to use
exponential type of model, for example, the GO model. If
the trend of the data set is shown as reliability decay
followed by growth, then the data set is suitable to use the
S-Shaped model, for example, the Delay S-Shaped model
or the Inflection S-Shaped model.
0 1020 30 405060 708090 100
-9
-8
-7
-6
-5
-4
-3
-2
-1
0
Laplace factor
time

Fig. 6 Laplace trend test
A. Criteria for Model's Comparison
The comparison criteria we engage to compare
various models’ performance are described as follows.
1.
The root Mean of Square fitting errors (RMSE) [21]

2
1
1
k
[ ( ) m t]
k
ii
i
RMSEm
=
=−
∑
(20)
Notice that m(ti) is the predicted number of faults at
ti, and mi is actual number of faults at ti. The lower
RMSE indicates less fitting error and better
performance.
The Relative Error (RE) [2]
(
i end
i
R
=
2.

) M tk
k
−
(21)
where k is detected faults at the end of testing, and
Mi is the predicted mean value functions at testing
time i (i<tend ,tend is the end of testing).
The Average Error (AE) [12, 13]
3.

1
k
100%*
k
i
i
AER
=
∑
(22)
The AE measures how well a model predicts
throughout the test phase.
The Average Bias (AB) [12, 13]
4.

1
k
100*
k
i
AB Ri
=
∑
(23)
The AB indicates the general bias of the model, and
it can be positive or negative. A negative value
shows that the model tends to underestimation,
while a positive one means the model is prone to
overestimation.
The short-term predictions, u-plot and y-plot.
The general technique for detecting systematic
objective differences between predicted and observed
failure behavior is call the u-plot [1, 23]. The purpose of
the u-plot is to determine if the predictions are on average
close to the true distributions or not. If we have perfect
5.

Page 5

predictions, then the u-plot will look like the line of unit
slope. A common way to test the significant difference
between predictions and actual values is Kolmogorov
distance, KD (the maximum vertical distance between
u-plot and the line of unit slope) [1, 23]. The y-plot is to
ensure that the u-plot can be trusted. That is because if
predictions are optimism in the early predictions and
pessimism in the later predictions, this will result a small
KD of u-plot. Therefore we need to apply the y-plot to
make it consistent.
B. Experimental Results

proposed neural-network-based models with three selected
SRGMs: the Goel-Okumoto model, the Yamada Delay
S-Shaped model, and the Inflection S-Shaped model [1-3].
We use the least square estimation to solve the parameters
of above model. In the rest of this paper, the NN-1 means
the neural-network-based model with one neuron in the
hidden layer, and the NN-3 means combinational model
with three neurons in the hidden layer. (Actually, the NN-1
is the logistic growth curve model but it is derived from
neural-network-based approach as we derive in the section
III.)
Firstly, we can find that the NN models give the
better fit than others; related root mean square errors are
shown in Table 1. Note that NN-3 model has larger
improvement in RMSE than the NN-1 model does.
Next, we compare the prediction ability of each
model with the u-plot and y-plot. Fig. 7 and Fig. 8 show
the u-plot and y-plot analyses of predictions of each model
for the S1 data, and the KD is shown in Table1. From Fig.
7, we find that the u-plot of each model is above the line
of unit slope and this means that the prediction of all
models tends to be optimistic for this data set. Note that,
the NN-3 model has better than NN-1 model. We can
confirm that the performance of prediction has improved
by using combinational models. According to Fig. 8, we
can trust the result of u-plot since the NN-1 and NN-3 also
have smaller KD of y-plot
While the u-plot and the y-plot can measure the
ability of the short-term prediction, the relative error can
be thought as the ability of the endpoint prediction
(long-term prediction). We train the network with different
sizes of the training set varied from 30% of the total
testing time to 100% testing time. After training, we feed
the t100 to forecast the number of faults, M30(t100). Then, we
calculate the forecasting error R30 between the actual
number of faults and the forecasting one. Following the
similar procedures, we can calculate the R40, …, R100. Table
2 gives the relative error of long-term predictions for each
model. This result shows that the endpoint predicted value
of the NN model can be much closed to the actual value.
That implies the neural-network-based models have better
long-term predictions than others. The AE and AB are
shown in Table 1. The smaller value of the AE means the
less underestimation. Note that each model has negative
values of the AB. This result makes it consistent that the
long-term predictions are also optimistic.
Here, we will analyze the performances of our
0 0.10.20.3 0.40.5 0.60.70.8 0.91
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Goel-Okumoto
DelaUed-S-Shaped
Infection-S-Shaped
Neural-network
Unit Slope

Fig. 7 u-plot for four selected models
0 0.10.20.3 0.4 0.50.6 0.7 0.8 0.91
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Goel-Okumoto
Delayed-S-Shaped
Infection-S-Shaped
Neural-network
Unit Slope

Fig.8 y-plot for four selected models.

Table 1 COMPARISONS OF RMSE AE AB AND KD
NN-1 NN-3
3.47(2) 1.17 (1)
17.68(4) 14.81(2)
-17.68(4) -14.81(2)
KD(u-plot) 0.118(4) 0.083(1)
KD(y-plot) 0.123(3) 0.079(1)


RMSE
AE
AB
GO
5.74(4)
16.29(3)
-16.29(3)
0.091(3)
0.106(2)

DS
11.27(5)
23.90(5)
-23.90(5)
0.263(5)
0.514(5)
IS
3.98(3)
8.70(1)
-8.70(1)
0.085(2)
0.333(4)
Table 2 LONG-TERM PREDICTIONS
NN-1 NN-3
-0.382 -0.372
-0.319 -0.302
-0.287 -0.185
-0.228 -0.165
-0.065 -0.084
-0.084 -0.033
-0.033 -0.027
-0.017 -0.014
GO
-0.285
-0.274
-0.242
-0.194
-0.131
-0.083
-0.056
-0.039
DS
-0.406
-0.350
-0.309
-0.263
-0.208
-0.159
-0.122
-0.094
IS
30
40
50
60
70
80
90
100
-0.127
-0.155
-0.145
-0.116
-0.071
-0.039
-0.025
-0.018
V.CONCLUSION

to software reliability engineering. Although many
researchers think that the neural network approach is a
black-box method, we still can explain the network
networks from a different aspect, that is, from the
mathematical viewpoints of software reliability modeling.
We have shown how to design the elements of neural
network to represent the existing SRGMs in detail.
Furthermore, we use the neural-network-based approaches
In this paper, we apply the neural network approach

Page 6

to achieve a dynamic weighted combinational model. To
validate our proposed models, we use a numerical example
of real software failure data set. We have compared the
performances of the neural network models with some
conventional SRGMs from three aspects: goodness of fit,
prediction ability for short-term prediction and long-term
prediction. From the experimental results, we can confirm
that the neural network models are workable and have
more accuracy with both goodness of fit and the prediction
ability compared to existing conventional models.
ACKNOWLEDGMENT
We would like to express our gratitude for the
support of the National Science Council, Taiwan, under
Grant NSC 94-2213-E-007-087 and also substantially
supported by a grant from the Ministry of Economic
Affairs of Taiwan (Project No. 94-EC-17-A-01- S1-038).

APPENDIX

network should be under some conditions that the used
activation functions are continuous and differentiable
everywhere. In the following, we will give the proof that
the activation functions used in this paper are (1)
continuous and (2) differentiable everywhere.
Using the back-propagation algorithm to train the
Theorem 1. If f is differentiable at a, then f is continuous
at a.
Proof. The proof is omitted.

activation function is differentiable everywhere.
The activation functions that we used are shown below:
1. f(x)=
e−
−
1

2. g(x)=
1
e−
+
3. m(x)=1 (1)
x e−
−+

From theorem 1, we only need to prove that the
x
1
x

x
Proof.
1. For any a
( )
lim
xa
→
since (
from the L’Hospital’s rule, we can derivate the limitation
as:
axx
a
eee
e
xa
−
at a.
Since a is variable, f(x) is differentiable on R.
R
∈
( ) f a
a
−

(1)
x
(1
a
x
)
lim
x
→
→
lim
x
→
as x
xaax
a
0
a
f xeeee
ax
e
x
−−−−
−
−
−
−−
−
−−
−
→
==
,
)
ax
e
−
and (
)0a
−→
a
,
lim
x
→
lim
x
→
1
aa
−−−
−
−
==
, that is, f(x) is differentiable
2. For any a
( )
lim
xa
→
From
( )
lim
xa
→
R
∈
( )g a
a
the
( )g a
a
−

lim(
L’Hospital’s
)(1 )(1)
ax
xa
xa
g xee
xxae
rule,
e
we
x
−
−−
−−
→
−
−
−
=
−++

can get
2
lim(1
)(1)( )(1 )(1)
.
(1)
xaxa
xa
a
a
g xe
xxeeaee
e
e
−−−−
→
−
−
−
=
+++−++
=
+
Since a is variable, g (x) is differentiable on R.
3. For any a
( )
lim
xa
→
From the L’Hospital’s rule, we can get
( )( )
lim
xa
xa
−
R
∈
( )

lim
x
→
axax
a
m xm a
a
eeae
−
xe
xxa
−−−−
−
−
−+−
=

2a
m x m a
a e
−
→
−
=
.
REFERENCES
[1] M. R. Lyu, Handbook of Software Reliability Engineering.
McGraw-Hill, 1996.
[2] J. D. Musa, A. Iannino, and K. Okumoto, Software Reliability,
Measurement, Prediction and Application. McGraw-Hill, 1987.
[3] M. Xie, Software Reliability Modeling. World Scientific Publishing,
1991.
[4] C. Y. Huang, M. R. Lyu, and S. Y. Kuo, “A Unified Scheme of
Some Nonhomogenous Poisson Process Models for Software
Reliability Estimation,” IEEE Trans. Software Eng., Vol. 29, No. 3,
March 2003, pp. 261-269.
[5] Z. Jelinski, and P. B. Moranda, “Software Reliability Research,” in
Statistical Computer Performance Evaluation (ed. Freiberger, W.),
Academic Press, New York, 1972, pp. 465-484.
[6] A. L. Goel, and K. Okumoto, “Time-Dependent error-Detection
Rate Model for Software Reliability and Other Performance
Measures,” IEEE transactions on Reliability Vol. R-28, No. 3,
August 1979, pp. 206-211.
[7] M. Ohba, et al., “S-shaped Software Reliability Growth Curve:
How Good Is It?” COMPSAC’82, 1982, pp.38-44.
[8] S. Yamada, and S. Osaki, “Reliability Growth Models for Hardware
and Software Systems Based on Nonhomogeneous Poisson
Processes: a Survey,” Microelectronics and Reliability, 23, 1983, pp.
91-112.
[9] S. Yamada, and S. Osaki, “S-shaped Software Reliability Growth
Model with Four Types of Software Error Data,” Int. J. Systems
Science, 14, 1983, pp. 683-692.
[10] N. Karunanithi, and Y. K. Malaiya, in M. R. Lyu Handbook of
Software Reliability Engineering, pp. 699-726.
[11] N. Karunanithi, Y. K. Malaiya, and D. Whitley, “Prediction of
Software Reliability Using Neural Networks,” Proc. 1991 IEEE Int,
Symp. on Soft. Rel. Eng., May 1991, pp. 124-130.
[12] N. Karunanithi, D. Whitley, and Y. K. Malaiya, “Using Neural
Networks in Reliability Prediction,” IEEE Software, Vol. 9, No. 4,
July 1992, pp. 53-59.
[13] N. Karunanithi, D. Whitley, and Y. K. Malaiya, “Prediction of
Software Reliability Using Connectionist Models,” IEEE Trans. On
Software Eng., Vol. 18, No. 7, July 1992, pp. 563-574.
[14] T. M. Khoshgoftaar, R. M. Szabo, and P. J. Guasti, “Exploring the
Behavior of Neural-network Software Quality Models,” Software
Eng. J., Vol. 10, no. 3, pp. 89–96, May 1995.
[15] R.? Sitte,?Comparison? of? Software? Reliability? Growth?
Predictions:? Neural? networks? vs.? parametric? recalibration ,?
IEEE?transactions?on?Reliability,?1999,?pp.?285-291.
[16] T. M. Khoshgoftaar, E. B. Allen, J. P. Hudepohl, and S. J. Aud,
“Software Metric-based Neural-network Classification Models of a
Very Large Telecommunications System,” in Applications and
Science of Artificial Neural Networks II, S. K. Rogers and D. W.
Ruck, Eds. Orlando, FL: SPIE-Int. Soc. Opt. Eng., Vol. 2760 of
Proc. SPIE, Apr. 1996, pp. 634–645.
[17] J. P. Hudepohl, S. J. Aud, T. M. Khoshgoftaar, E. B. Allen, and J.
Mayrand, “EMERALD: Software Metrics and Models on the
Desktop,” IEEE Software, vol. 13, no. 5, pp. 56–60, Sept. 1996.
[18] T. Dohi, Y. Nishio, and S. Osaki, “Optimal Software Release
Scheduling Based on Artificial Neural Networks,” Annals of
Software Engineering, Vol. 8, 1999, pp. 167-185.
[19] K. Y. Cai, L. Cai, W. D. Wang, Z. Y. Yu, and D. Zhang, “On the
Neural Network Approach in Software Reliability Modeling”, The
Journal of Systems and Software, 2001, pp. 47-62.
[20] S.?L.?Ho,?M.?Xie,?and?T.?N.?Goh,? A?Study?of?the?Connectionist?
Models? for? Software? Reliability? Prediction ,? Computers? and?
Mathematics?with?Applications,?Vol.?46,?2003,?pp.?1037-1045.
[21] M.R. Lyu and A. Nikora, “Using Software Reliability Models More
Effectively,” in IEEE Software, July 1992, pp.43-53.
[22] K. Kanoun and J. C. Laprie, “Software Reliability Trend Analyses
from Theoretical to Practical Considerations.”, IEEE Transactions
on Software Engineering, Vol. 20, No. 9, Sep. 1994, pp. 740-747.
[23] N. E. Fenton and S. L. Pfleeger, Software Metrics: A Rigorous and
Practical Approach. PWS Publishing Company.