Content uploaded by Alireza Khanteymoori
Author content
All content in this area was uploaded by Alireza Khanteymoori on Jun 18, 2015
Content may be subject to copyright.
American Journal of Scientific Research
ISSN 1450-223X Issue 12 (2010), pp.06-15
© EuroJournals Publishing, Inc. 2010
http://www.eurojournals.com/ajsr.htm
Predict and Approximate Software Quality with Bayesian
Networks and Quality Factors
Homayun Motameni
Department of Computer Sciences, Islamic Azad University, Sari Branch, Sari, Iran
E-mail: Motameni@iausari.ac.ir
Hadi Kamfar
Department of Computer Sciences, Islamic Azad University, Zanjan Branch, Iran
Alireza Khanteimori
Department of Computer Sciences, Amir Kabir University of Technology, Tehran, Iran
Abstract
Software quality is one of important field in software engineering that related to
software satisfaction. Some models and methods have made to calculating quality. Almost
all of these models use quality factors or metrics, but calculate them is another problem
because we can not calculate them exactly or can not determine some of them in some
software project and usually our data about quality factors and metrics are incomplete or
uncertain. Also these models can not predict software quality before calculate all of quality
metrics and factors. Bayesian Networks have become a popular tool for modeling many
kinds of statistical problems over the last decade. In this paper we proposed a model for
software quality with BNs and ISO9126 quality model. ISO9126 is one of best and
complete quality model. This new model doesn't have last quality model's problems and
can predict software quality with incomplete and uncertain data. Also by this model we can
reduce time and cost of calculating software quality.
Keywords: Software Quality Model, Bayesian Networks, ISO9126
Introduction
Software quality is "Conformance to explicitly defined functional and implicit characteristics that are
expected of professionally developed software". Software is now used in many demanding applications
and software defects have caused serious damage and even physical harm. This software can be
software to fly airplanes or to drive automobiles, software to control air traffic, run factories or operate
power plants. People have been killed by defective software. The aim of software quality engineering
is to investigate the relationships among in-process metrics, project characteristics, and end-product
quality. Therefore software quality is very important that can improve software satisfaction. By
determine software quality the manager or stakeholder can decide to using software or improving the
aspects and factors that have low quality. We don't use or sell software with low quality because it
makes much wasted time and cost for both producer and consumer. Software quality has more effect
when we predict or determine quality among the project because we can reduce cost and time with
reviewing or ending the software project.
Predict and Approximate Software Quality with Bayesian Networks and Quality Factors 7
There are many efforts to determining Software quality and have made some models and
methods for calculating it like McCall Model, Boehm Model, FURPS Model, ISO9126 Model and
Dromey Model. Almost all of these models use quality factors or metrics, but calculate these quality
factors and metrics are another problem because we can not calculate them exactly or can not
determine some of them in some software project and usually our data about quality factors and
metrics are incomplete or uncertain. Also these models can not predict software quality before
calculate all of quality metrics and factors.
In this paper we proposed a model for software quality with BNs and ISO9126 quality model.
ISO9126 is one of best and complete quality model. In fact we model ISO9126 with BN. By this work
we can use probability of quality factors and metrics. This new model doesn't have last quality model's
problems and can predict software quality with incomplete and uncertain data. Also by this model we
can reduce time and cost of calculating software quality.
In chapter 2 we introduce Bayesian Network with an example. In chapter 3 we discuss about
software quality model and ISO9126 quality model. In chapter 4 we proposed SWQBN (Software
Quality with Bayesian Network).
Introduction to Bayesian Networks
Bayesian Networks (BN) derive from the convergence of statistical methods that permit one to go from
information (data) to knowledge (probability laws, relationship between variables,…) with Artificial
Intelligence (AI) that permits computers to deal with knowledge (not only information). The main
purpose of BN is to integrate uncertainty into expert systems (SIMON, WEBER & LEVRAT, 2007).
An expert, most of the time, has only approximate knowledge of the system, that he/she formulates in
terms like: A has an influence on B; if B is observed, there is a great chance that C occurs and so on. It
becomes obvious that this tool is well-suited to deal with the problem concerned, as expert knowledge
can be needed to estimate, for instance, the reliability of a component or how components of a
subsystem interact in terms of reliability (Khaddaj & Horgan, 2004).
Bayesian Networks are an increasingly popular formalism for reasoning and decision-making
in problems that involve uncertainty and probabilistic reasoning. A Bayesian Network is an acyclic
directed graph that represents dependencies between variables and their associated probabilities
(Zalewski, Kornecki & Pfister, 2006). Nodes represent variables and arcs represent relationships
between variables. Nodes of a Bayesian network are usually drawn as circles or ovals. The semantics
of a link can be described in the following way: there is a link from X to Y, if X has a direct influence
on Y. Each node has a conditional probability table (CPT) associated that quantifies the effects that the
parents have on the node. A Bayesian network also represents the quantitative relationships among the
modeled variables (Dahl & Gran, 2000).
Numerically, it represents the joint probability distribution among them. This distribution is
described efficiently by exploring the probabilistic independence among the modeled variables. Each
node is described by a probability distribution conditional on its direct predecessors. Nodes with no
predecessors are described by prior probability distributions (Kornecki & Zalewski, 2006).
In figure 1, there is a burglar alarm that can be fired by a burglar or by mistake due to an
earthquake. The probability of happening a burglary (P(B)) is 0.001, and the probability of happening
an earthquake (P(E)) is 0.002. The probability of the alarm going off depends on the nodes Burglary
and Earthquake, which is depicted in the table near the alarm node (Neil & Fenton, 1996).
Supposing that there are two persons that can call when the alarm goes off, John and Mary, the
probability of any of them calling because the alarm went off is also represented in the tables for nodes
John Calls and Mary Calls.
8 Homayun Motameni, Hadi Kamfar and Alireza Khanteimori
Figure 1: An example of a Bayesian network
Inference in a BN is the computation of the posterior probability distribution for the set of
query variables. Given a set of evidence variables for which the exact value is known, an inference
algorithm is used to propagate these values through the BN, according to Bayes rule. There are several
inference algorithms, but basically they fall in two categories: exact inference and approximate
inference.
Exact inference algorithms perform the exact computation of the probability distributions.
Belief propagation in trees is linear, while belief propagation in multiply connected graphs has been
shown to be NP-hard. Approximate inference algorithms try to deal with this difficulty in a more
tractable way, trying to provide an approximation to the probability distribution for query variables
(Kaner & Bond, 2004).
Software Quality Model and Metrics
Quality isn’t about zero defects or measurable improvements in defect rates, nor is it about meeting
documented requirements. Its no more and no less that satisfying customer needs (whether or not the
needs are properly documented). The ultimate measure of design quality is fitness to purpose. This
means that to measure software quality we have to understand what the purposes are for which it is
intended. Note this also means that quality is not a measure of software in isolation; it is a measure of
the relationship between software and its application domain. This means that an assessment of its
quality depends on the context, and if you change the context, the assessment of quality may change –
software that’s good for one purpose may not be so good for another purpose.
Measurement of quality generally starts by identifying a set of critical quality factors, and then
refining these down to measurable attributes of the software (Fenton & Pfleeger, 1997). There is no
agreed way of doing this. Different quality assessors disagree about what various measurable attributes
of software actually tell you about its quality.
No quantitative approach to software quality assurance can be complete without a measurable
definition of software product quality (Mustafa & Khan, 2005). We can never know whether the
quality is satisfactory or improving if we cannot measure it. Moreover, we need quality measures if we
are to improve our resource estimation and productivity measurement. In the case of resource
estimation, higher quality requirements may lead to greater resources. In the case of productivity
measurement, speed of production is meaningless without an accompanying assessment of product
quality. Thus work on resource estimation and productivity assessment inspired software engineers to
develop models of quality which took into account various views of software quality. For example,
Boehm’s advanced COCOMO cost estimation model is tied to a quality model. Similarly, the McCall
quality model is related to productivity (Firesmith, 2003).
Quality Model is the set of characteristics and the relationships between them which provide
the basis for specifying quality requirements and evaluating quality (Gran, 2002). some type of quality
Predict and Approximate Software Quality with Bayesian Networks and Quality Factors 9
model are McCall Model, Boehm Model, FURPS Model, ISO9126 Model (Korb & Nicholson, 2004)
and Dromey Model.
These models are usually constructed in a tree-like fashion. The upper branches hold
important high-level quality factors of software products, such as reliability and usability that we
would like to quantify. Each quality factor is composed of lower-level criteria, such as modularity and
data commonality (Ankush & Ashraf, 2007). The criteria are easier to understand and measure than the
factors; thus, actual measures (metrics) are proposed for the criteria. The tree describes the pertinent
relationships between factors and their dependent criteria, so we can measure the factors in terms of the
dependent criteria measures. Quality models are expected to be used at the specification and design
phase of software quality assurance. The idea is that targets for the high level factors are set, while
assessments of the likeliness of meeting these targets is based on measuring the lower level criteria
during design. We use The ISO9126 Quality Model for predict software quality in this paper that is
more popular than other and canonical.
ISO9126 Model
Based on previous attempts for defining software quality, the International Organization for
Standardization (ISO) and the International Electro technical Commission (IEC) have published the
multipart standard ISO/IEC 9126 which defines a software product quality model, quality
characteristics, and related metrics. These constituents can be used to both evaluate and set goals for
the quality of a software product (ISO/IEC 9126, 2001).
Part 1 of ISO/IEC 9126 contains a two-part quality model: one part of the quality model is
applicable for modeling the internal and external quality of a software product, whereas the other part
is intended to model the quality in use of a software product. These different quality models are needed
to be able to model the quality of a software product at different stages of the software lifecycle
(Fenton & Pfleeger,1997).Typically, internal quality is obtained by reviews of specification
documents, checking models, or by static analysis of source code. External quality refers to properties
of software interacting with its environment. In contrast, quality in use refers to the quality perceived
by an end user who executes a software product in a specific context. These product qualities at the
different stages of development are not completely independent, but influence each other. Thus,
internal metrics may be used to predict the quality of the final product – also in early development
stages.
For modeling internal quality and external quality, ISO/IEC 9126 defines the same model. This
generic quality model can then be instantiated as a model for internal quality or for external quality by
using different sets of metrics. The model itself is based on the six characteristics functionality,
reliability, usability, efficiency, maintainability, and portability. Figure 2 shows the ISO9126 Quality
Model's factors and metrics.
Figure 2: The ISO9126 Quality Model's factors and metrics
Quality
Replaceability
Installability
Co-existence
Adaptability
Usability
Learnability
Understandability
Operability
Attractiveness
Efficiency
Time behavior
Resource
Utilization
Maintainabilit
y
Analyzability
Stability
Changeability
Compliance
Testability
Portability
Maturity
Recoverability
Fault-tolerance
Security
Accuracy
Interoperability
Suitability
Reliability Functionalit
10 Homayun Motameni, Hadi Kamfar and Alireza Khanteimori
As shown in Figure 2, each of these characteristics has further sub-characteristics. ISO9126
Quality Model is an internationally recognized model and being followed by majority of the
organization around the world. During the Requirement Analysis phase, categorize the requirements as
per the characteristic and the sub-characteristic in the ISO9126 Model Quality Factor is obtained by
using a series of formulas (Mustafa & Khan, 2005). The definition of ISO9126 Quality Model's
characteristics are:
Functionality: A set of attributes that bear on the existence of a set of functions and their
specified properties. The functions are those that satisfy stated or implied needs.
Reliability: A set of attributes that bear on the capability of software to maintain its level of
performance under stated conditions for a stated period of time.
Usability: A set of attributes that bear on the effort needed for use, and on the individual
assessment of such use, by a stated or implied set of users.
Efficiency: A set of attributes that bear on the relationship between the level of performance of
he software and the amount of resources used, under stated conditions.
Maintainability: A set of attributes that bear on the effort needed to make specified
modifications.
Portability: A set of attributes that bear on the ability of software to be transferred from one
environment to another.
The model of quality in use is based on the characteristics effectiveness, productivity, safety,
and satisfaction and does not elaborate on further sub-characteristics. In the further parts of ISO/IEC
9126, metrics are defined which are intended to be used to measure the attributes of the
(sub)characteristics defined in Part 1: The provided metrics are quite abstract which makes them
applicable to various kinds of software products, but they cannot be applied without further refinement.
Table 1 show defining the sub-characteristics in the ISO9126 Quality Model.
Table 1: Defining the sub-characteristics in the ISO9126 Quality Model
Characteristic
name
Sub-Characteristic
name Definition
Functionality
Suitability The capability of the software to provide an appropriate set of functions
for specified tasks and user objectives
Accuracy The capability of the software to provide the right or agreed results
Interoperability The capability of the software to interact with one or more specified
systems
Security The capability of the software to protect information and data from
unauthorized access
Reliability
Maturity The capability of the software product to avoid failure as a result of faults
in the software
Fault-tolerance The capability of the software to maintain a specified level of performance
in cases of software faults
Recoverability The capability of the software to re-establish a specified level of
performance and recover the data in case of a failure
Usability
Understandability The capability of the software product to be understand, learned, used and
appear attractive to the user
learn ability The capability of the software to provide appropriate response while
performing its function
Operability The capability of the software to enable the user to operate and control
Attractiveness The capability of the software to be attractive to the user
Efficiency
Time behavior The capability of the software to provide appropriate response while
performing its function
Resource Utilization The capability of the software product to use appropriate amounts and
types of resources
Maintainability
Compliance The capability of the software to follow standards of efficiency
Analyzability The capability of the software to be diagnosed for causes of failures in the
software, or for the parts to be modified to bee identified
Predict and Approximate Software Quality with Bayesian Networks and Quality Factors 11
Changeability The capability of the software to enable a specified modification to be
implemented
Stability The capability of the software to avoid unexpected effects from changes
made in the software
Testability The capability of the software product to enable modified software to be
validated
Portability
Adaptability The capability of the software to be adapted for different environments
Install ability The capability of the software to be installed in a specified environment
Co-existence The capability of the software to co-exist with other independent software
Replace ability The capability of the software to be used in place of another software for
the same purpose
Data Collection
Measurement is the key factor in any software quality assurance program. But effective use of
measurement is dependent on careful data collection, which is difficult, especially when data must be
collected across a diverse set of projects. Thus, data collection is becoming a discipline in itself, where
specialists work to ensure that measures are defined unambiguously, that collection is consistent and
complete, and that data integrity is not at risk. But it is acknowledged that metrics data collection must
be planned and executed in a careful and sensitive manner. Data collection is also essential for
scientific investigation of relationships and trends. Good experiments, surveys and case studies require
carefully-planned data collection, as well as thorough analysis and reporting of the results.
SWQBN: Modeling software quality with BN
We must to obtain or compute every software quality factor to calculate software quality with ISO9126
or any other quality model. But it is impossible to compute all of quality factor in different software for
some reasons:
• Some of quality factor (in different software) can't be computed
• Sometimes cost of compute all of quality factors are very high
• We need long time for compute all of quality factors
• In some cases we need to a lot of people (user, quality manager, member of quality assurance's
team) to compute quality factors
Because of these reasons we can't compute all of software quality factors for all software. Also
we usually have uncertainty in quality factors and metrics. Therefore it is better to find a way to
compute software quality with only some of quality factors. Our new model needs data of quality
factors and metrics from different software and different company.
We can use Bayesian Networks to compute software quality. By this way we don't need to
know all of quality factors or metrics. Bayesian network graph for software quality has three levels.
Quality metrics are in first level and their arcs go to quality factors in second level. Quality factor's arcs
go to third level in quality node.
Each node (quality, quality factor, quality metric) has five states: Very High, High, Medium,
Low and Very Low. The data (probability) for first level (quality metric) is different from one
company or organization to another. For example security in a company is more important than another
one. Therefore the quality experts of each company can fill them in CPT according to their quality
metrics, knowledge, and experience of previous software in Thier Company. Using the previous
knowledge of the similar project and software is one of the most important factors to filling CPT.
Bayesian network graph is shown in figure 3.
12 Homayun Motameni, Hadi Kamfar and Alireza Khanteimori
Figure 3: Modeling software quality with Bayesian Networks
Because of large amount of data for CPT in second and third level and to filling them faster it is
almost impossible to fill CPT by hand. We can use BN programming software like BNT (in Matlab) or
BNJ (in JAVA).
We can use this way to fill CPT of node in second and third level. We generate different state
of CPT by programming algorithm. For example there are 54 = 625 states (row in CPT) for a quality
factor by 4 quality metrics. By table2 we can calculate the amount of each CPT's rows.
Table 2: Degree of quality state
State degree
Very High 5
High 4
Medium 3
Low 2
Very Low 1
Also we can suppose priority for each quality metrics and their affect on quality factor. But
quality experts don't agree on a unique model for it and most of them agree on this model (model with
equal priority of quality metrics). After that we modulate these amounts for each row to domain 0 to 1.
For this task we can use this formula: p=d/(5*n).
In this formula 'd' is degree of each row and 'n' is the number of quality metrics that go to
quality factor. And 'p' is calculated percent of each row. In some BN's software 'p' is in range of 0 to
100. We don’t multiply 100 to calculate 'p' in them and formula
become
ndpndp /*20100*))*5/((
=
⇒
=
. For example 'Reliability' and its three metrics (Maturity,
fault-tolerance, Recoverability) are in table 3 (part of CPT).
Table 3: An example of a row of CPT (Reliability factors)
Maturity Fault-tolerance Recoverability
Very High Low Medium
Predict and Approximate Software Quality with Bayesian Networks and Quality Factors 13
67.663/10*20/*2010325
=
=
=
⇒
=
+
+
=
ndpd
It means we have Reliability in this example by probability of 66.67 percent. We can see the
code of filling CPT for a factor with three metrics like 'Reliability'. Figure 4 shows Code of filling CPT
for a factor with three metrics.
Figure 4: Code of filling CPT for a factor with three metrics
For
i = 1
to
5
For j = 1 to 5
For k = 1 to 5
d = i + j + k;
p = d / (5 * n);
CPT( i , j , k) = p; Set to CPT
End For
End For
End For
This algorithm will implement for all of quality factors and software quality. Some of nested
'For' is equal to some of metrics that effect on each quality factor. Therefore we need 7 nested loops for
filling CPT of Software quality. We can use one set of nested 'For' for the factors with equal amount of
metric. Therefore we can use one set of four nested 'For' for 'Functionality', 'Portability' and 'Usability'.
Figure 5 shows Code of filling CPT for three factors each of them with four metrics. This code
constructs CPTs faster.
Figure 4: Code of filling CPT for a factor with three metrics
For
i = 1
to
5
For j = 1 to 5
For k = 1 to 5
For l = 1 to 5
d = i + j + k + l;
p = d / (5 * n);
CPT( i , j , k , l) = p; Set to 'Functionality'
CPT
CPT( i , j , k , l) = p; Set to 'Usability' CPT
CPT( i , j , k , l) = p; Set to 'Portability' CPT
End For
End For
End For
End For
Profits of SWQBN
Using SWQBN model for predicts software quality has some benefit that different it from other
models. We can predict software quality with a few evidences. In other models for compute quality we
had to gain all of quality factors and metrics. This trait reduces cost and time wasting because if we
determine software quality earlier, we can make a suitable decision to increase software quality earlier.
Therefore we can produce software earlier, cheaper and by higher quality. In other models we had to
14 Homayun Motameni, Hadi Kamfar and Alireza Khanteimori
compute quality by some math relation and asking question from and filling questionnaire by quality
experts, users, manager, stakeholders, members of software quality assurance team and members of
software development team. Therefore SWQBN decreases time and cost of computing software
quality. Also SWQBN need to much fewer people to calculating quality than other models.
By SWQBN we can predict software quality before end of software project (in construction
time). With this ability we can avoid project from failing. Also by these reason we can decrease cost of
quality assurance. One of best advantage of SWQBN is that we can view affect of one or more quality
factors or metrics on software quality. This ability helps us to make a suitable decision among the
project to increasing software quality.
Conclusion
Quality models are methods to calculate software quality. Usually quality models are made of by
quality factors and quality metrics. ISO9126 is one of the newer, standard and more popular of quality
models. This model contains six quality factors. These factors are determined by some quality metrics.
But ISO9126 and other quality models have some limit to compute software quality. The main problem
of them is limitation to calculate all of quality factor in different software. We choose ISO9126 model
because it is more efficient than other quality models. We model software quality with BN and
ISO9126. This model reduces cost and time wasting that is high in other quality model (Bertoa, Troya
& Vallecillo, 2006).
We had to compute all of quality factors and metrics exactly but it is not possible to compute all
of them all the time and in each software project. But this proposed model can work with incomplete
and uncertain data. Also by this model we don't have to compute all of the quality factors and metrics.
We can compute software quality by having some of the quality factors and metrics. Also SWQBN
decreases time and cost of quality assurance and computing software quality. SWQBN can predict
quality among the project therefore we can increase software quality by make a suitable decision. For
this reason SWQBN decrease time and cost of developing software. One of the most important
advantages of using SWQBN is improvement of software quality and customer satisfaction (Yao &
Yang, 2007).
For further work we can implement software to predict quality with our new model. This
software will get default (primary) probability of quality metrics in user's software company from user.
To predict software quality for each software project, the calculated qualities metrics will enter to
quality predict software. Then software calculates probability of other quality factors, metrics and
software quality. Also we can use different effect of quality factors and metrics for software quality.
Project managers, quality experts, stakeholders, software development teams and quality assurance
teams will be the user of this software.
References
[1]
SIMON, C.H., WEBER, P.H., & LEVRAT, E. (2007). Bayesian Networks and Evidence
Theory to Model Complex Systems Reliability, Journal of Computers, 2(1), 1-12.
[2]
Khaddaj, S., & Horgan, G. (2004). The Evaluation of Software Quality Factors in Very Large
Information Systems, Electronic Journal of Information Systems Evaluation, 7(1), 43-48.
[3]
Zalewski, J., Kornecki, A.J., Pfister, H.L. (2006). Numerical Assessment of Software
Development Tools in RealTime Safety Critical Systems Using Bayesian Belief Networks,
Proceedings of the International Multiconference on Computer Science and Information
Technology, pp. 433 – 442.
[4]
Dahl, G., & Gran, B.A. (2000). The Use of Bayesian Belief Nets in Safety Assessment of
Software Based Systems, International Journal of General Systems, 29(2), 205-229.
Predict and Approximate Software Quality with Bayesian Networks and Quality Factors 15
[5]
Kornecki, A., & Zalewski, J. (2006). The Qualification of Software Development Tools From
the DO-178B Certification Perspective. Crosstalk, The Journal of Defense Software
Engineering, 19(4), 19-23.
[6]
Neil, M., & Fenton, N. (1996). Predicting Software Quality using Bayesian Belief Networks,
Proceedings of 21st Annual Software Engineering Workshop NASA/Goddard Space Flight
Centre, December 4-5, pp.1 – 8.
[7]
Kaner, C., & Bond, W.P. (2004). Software Engineering Metrics: What Do They Measure and
How Do We Know? 10th international software metrics symposium, metrics, pp.1 – 12.
[8]
Fenton, N.E., & Pfleeger, S. L. (1997). Software Metrics: A Rigorous and Practical
Approach,2nd Edition Revised ed. Boston: PWS Publishing.
[9]
Mustafa, K., & Khan, R.A. (2005). Quality Metric Development Framework (MDF), Journal of
Computer Science, 1 (3), 437-444.
[10]
Firesmith, D. (2003). Using Quality Models to Engineer Quality Requirements, Journal of
Object Technology, 2(5), 67-75.
[11]
Gran, B.A. (2002). The use of Bayesian Belief Networks for combining disparate sources of
information in the safety assessment of software based systems, Dr.Ing. Thesis, Norwegian
University of Science and Technology, pp.1 – 100.
[12]
Korb, K.B., & Nicholson, A.E. (2004). Bayesian Artificial Intelligence, Chapman & Hall/CRC.
[13]
Ankush, M., & Ashraf, K. (2007). Bayesian Network Technologies: Applications and
Graphical Models, IGI PuBlIshING, New York.
[14]
ISO/IEC 9126. (2001). Software Engineering-Product Quality. Parts 1 to 4.
[15]
Bertoa, M.F., Troya, J.M., & Vallecillo, A. (2006). Measuring the usability of software compo-
nents, Journal of Systems and Software ,79(3), 427-439.
[16]
Yao, L., & Yang, B. (2007).An Approach to Early Prediction of Software Quality, Journal of
Electronic Science and Technology of China, 5(1), 1-6.
