The impact of usability on the cost of quality

Full text

Copyright, American Society for Engineering Management, 2012
THE IMPACT OF USABILITY ON THE COST OF QUALITY
Burchan Aydin, M.A.A., Texas Tech University
Himlona Palikhe, M.S.I.E., Texas Tech University
Mario G. Beruvides, Ph.D., P.E., Texas Tech University
____________________________________________________________________________________________
Abstract
Usability leads to many significant benefits for
organizations such as reduced development cost,
decreased maintenance and support costs, and
increased sales. Increased efficiency, effectiveness,
and user satisfaction are the main benefits for product
end users. There are also hidden costs of ‘not investing
on usability’. These are the missed benefits that
usability offers such as increasing sales. Usability may
decrease the total cost of achieving good quality. Cost
of quality is classified into prevention (P), appraisal
(A), failure (F) costs known as the PAF model. Later,
opportunity (L) cost was added to the PAF model to
consider the intangible costs. This paper looks at the
change in cost of quality before and after the
investments in usability and classifies the cost of not
investing in usability into the PAF model and
opportunity costs. These findings show the effects or
impacts of usability investment on the cost of quality.
Key Words
Usability, Usability costs, Cost of quality, Prevention
cost, Appraisal cost, Failure cost, Opportunity cost
Introduction
“Usability is the ease of use and acceptability of a
system for a specific class of users carrying out
particular tasks in a specific environment” (Holzinger,
1995, 1). The main characteristics of usability are
efficiency, effectiveness, and satisfaction.
Effectiveness is “the accuracy and completeness with
which users achieve specified goals”, while efficiency
is “the use of resources in relation to the accuracy and
completeness with which users achieve goals” (ISO
9241-11, 1998, 2). For instance, on an e-commerce
web site, ‘purchase completion rate’ indicates
effectiveness, and ‘time to complete the purchase’
indicates efficiency. On the other hand, satisfaction is
“the freedom from discomfort, and positive attitudes
towards the use of the product” (ISO 9241-11, 1998,
2). In case of an e-commerce web site, ‘returning
customer rates’ is a good measure to determine the
satisfaction level. These indicate that usability
improves quality and decreases the total cost of
achieving good quality.
Cost of quality is a tool that reveals the cost of not
achieving the best quality possible. There are different
models of cost of quality in the literature. Among
these models, the most widely used is the prevention,
appraisal, and failure cost (PAF) model introduced by
Feigenbaum in 1956. Opportunity cost was added to
the PAF model by Beruvides and Sandoval-Chavez in
1997. The PAF and opportunity costs are described in
the following sections.
Prevention Cost. Costs incurred while preventing
defects from occurring are prevention costs. Quality
costs to avoid substandard products from launching in
the first place fall under prevention cost. Quality
engineering and employee quality training are some of
the examples of prevention costs (Feigenbaum, 1983).
Appraisal Cost. Costs used to sustain quality levels of
the company by proper assessment of product quality
are appraisal costs. Scrutiny, testing, outside
endorsements, quality checks etc. are some of the
examples of appraisal costs (Feigenbaum, 1983).
Failure Cost. Costs acquired due to defective
products, rework etc. are failure costs. It is measured
by internal failure costs and external failure costs.
Costs due to substandard quality inside the
organization such as scrap, waste and rework are
internal failure costs. Whereas, costs due to
substandard quality outside the organization such as
product-performance failure and customer complaints
etc. are external failure costs (Feigenbaum, 1983).
Opportunity Cost. Sandoval-Chavez and Beruvides
(1998) stated that opportunity cost is the “cost of
quality expressed as revenue lost and profit not earned”
(p. 107). Inefficiency in delivery service and material
handling combined with underutilization of
infrastructure is considered as undermining factors.
The losses due to these factors are generally considered
as opportunity cost (Beruvides and Sandoval-Chavez,
1997).
Cost of Quality Before Investments on Usability
The total cost of quality is higher before usability
investments. Literature shows that there are lots of
benefits of usability. In this paper, the benefits of
usability in the case of development organizations is
discussed. Development organizations are
organizations that develop software, user interface of
hardware and e-commerce websites. Exhibit 1 shows,

2
International Annual Conference of the American Society for Engineering Management
Exhibit 1. Costs of Quality before Usability Investments.
the different PAF and opportunity costs in terms of cost
of quality before usability investments in development
organizations. No appraisal costs were found in the
literature before usability investments. Usability
investments decreases the total cost of quality by
decreasing these PAF costs and increasing the
opportunity for benefits. Some of the benefits due to
investments in usability are discussed in the following
section.
Benefits of Usability
Usability offers many benefits to organizations and
end-users. In this section, the main benefits of
usability, to development organizations, will be
explained and illustrated by a few examples.
Increased Sales. Usability is an investment that offers
great revenue growth. Usable sites enhance user
satisfaction, which most likely increases reputation of
the organizations. Satisfied customers extend the
duration that they stay on company websites. This
improves the chances of completed purchases in e-
commerce websites. Moreover, satisfied users return
to the website later on, creating even more sale
opportunities for e-commerce sites. A study of Internet
shoppers by Binary Compass Enterprises showed that
new users at an e-commerce site spent an average of
$127 per purchase, whereas repeat users spent an
average of $251 (Usable Information Technology,
1997). Thus, usability offers revenue growth by
satisfying first time customers, and also by increasing
number of returned customers. Some statistics are
provided below to show cases in the last decade.
• Staples enhanced the usability of staples.com in
1999 working with Human Factors International.
Third-quarter 2000 sales raised 491% compared to the
previous year. Moreover, the new website had 67%
more repeat customers in a quarter fiscal year (HFI,
1999).
• Another success story about usability is that La
Quinta Hotel Chain reached a revenue growth of 83%
in a year by improving the usability of their website
(Peterson, 2007).
• Redesign of a software product called DEC
Rally resulted in 80% increase in revenues for the
developing organization (Wixon and Jones, 1995).
• DELL Computer Inc. redesigned their e-
commerce web site by applying basic usability
principles. Online purchases rose from $1 million per
day in September 1988, to $34 million per day in
March, 2000 (Black, 2002).
Decreased User Errors. Usability increases
effectiveness, which indicates that users can complete
tasks accurately. Accuracy of a task can indicate
completion of a purchase on an e-commerce site, or
finding the information to a customer question on a
support call. If users make less error in e-commerce
sites, they will more likely be willing to stay in the site.
That helps increase revenues for e-commerce.
Moreover, if employees can process tasks in intranets
or any other internal software with fewer errors,
significant time will be saved in organizations. These
time savings can be converted into financials by
considering the hourly rates of those employees.
Following are some examples of decreased error rates
provided by usability investments.
• Usability improvements yielded 90% reduction
in customer errors in an e-commerce website (Karat,
2005).
• The results of an empirical study performed in a
large wireless communicator carrier showed that mean
error rates per screen decreased by 67% after usability
enhancements (Cope and Uliano, 1995).
• The New York Stock Exchange (NYSE)
upgraded their computer system with more usable
equipment, and error rates have fallen by 10% (Gibbs,
1997).
Prevention Cost
Failure Cost
Opportunity Cost
 Training Cost
 Development Cost
 Error
 Failure
 Maintenance Cost
 Support Cost
 Employee Turnover Cost
 Litigation Cost
 Drop off Cost
 Miss the opportunity of the following:
 Increasing Sales
 Decreasing Task Completion Time
 Decreasing Training Time
 Reducing Learning Time
 Reducing Development Time
 Increasing User Satisfaction and Loyalty
 Increasing Traffic
 Increasing Returning Customer

The Impact of Usability on The Cost of Quality
3
Decreased Task Time. Usability increases efficiency,
which indicates that users can complete tasks in a short
time without too much effort. Examples of the impact
of usability on task times can be seen in many cases in
the literature.
• American Express redesigned its computer
interface for bank authorizations, and task times
decreased by 325% from 17 minutes to 4 minutes
(Gibbs, 1997).
• Nielsen and Gilutz (2003) conducted a series of
usability tests on intranets, and the results indicated
that total time to perform 14 tasks decreased from 68.5
minutes to around 36.5 minutes, a 47% drop.
• Performance Bikes redesigned their web site
and time to checkout reduced by 40% (Nielsen and
Gilutz, 2003).
• Sun Microsystems designing Java White Papers
reworked existing pages to enhance usability and
decreased task times (number of seconds users take to
complete three tasks) from 637 seconds to 315 seconds,
an efficiency improvement of 102% (Nielsen and
Gilutz, 2003).
• Breastcancer.org, a non-profit organization to
provide information on breast cancer, consulted
Foraker Design Group to redesign its web site
according to usability principles. As a result, time to
register decreased by 53% from an average of 15
minutes to 7 minutes (Usability First, 2007).
• In a call center, usability lead to savings of 61
seconds per call; indicating savings of $250,000 as a
result of reduced call times (Bond, 2007).
Decreased Training Time. As usability makes
products easier to use, training time for novice users is
significantly shorter for products with high usability.
Eliminating a major part of the training costs is
definitely favorable for organizations. Following
statistics show real cases where training costs were
decreased by investing on usability.
• The computer interface for bank authorizations
of American Express was redesigned with usability
principles. Training times fell from 12 hours to two
hours (Gibbs, 1997).
• It is found in a study by management
information system managers that training time for a
standard computer was 21 hours compared to the
training time of only 11 hours of a more usable
computer (Diagnostic Research, 1990, as cited in
Rohn, 2005).
• Karat (1993b) reported a case study of a product
with high usability requiring an hour of training,
whereas similar product without usability necessitating
a week of training.
Reduced Development and Maintenance Costs.
Usability problems are easy and less costly to be fixed
early in the development cycle. Customers usability-
test the system whether or not your company does
usability testing (Donahue, 2001). That will force the
companies to make late design changes. Therefore, it
is certainly the best decision to consider usability
problems early in the development stage. Not only the
development costs, but also the development time is
reduced by usability. The main reason why
development time decreases is that a product with only
relevant functionality (minimalist design) will be
designed; eliminating time wasting on designing
irrelevant functions for the product (Bevan, 2001).
• Boehm (1981) estimated that fixing usability
problems in the maintenance phase is 40-100 times
more expensive than fixing the same problems in the
development phase.
• Pressman (1992) estimated that if making
changes in definition phase costs 1, it costs 1.5-6 times
more in development phase, and 60-100 times more
after product release.
• Mayhew and Tremaine (2005) used an early-to-
late design change rate of 0.25 as a rule of thumb
together with many researchers such as Cope and
Uliano (1995). Thus, design changes costs four times
more in later stages of development lifecycle compared
to the development stage.
• Pressman (1992) estimated that 80% of the
software lifecycle costs occur in the maintenance stage,
and 80% of the maintenance is because of unmet user
requirements (Martin and McClure as cited in Bevan,
2001)
• American Airlines figured that catching a
usability problem in the design stage may reduce
development costs by 60-90% (Harrison, et al., 1994).
Reduced Support Costs. As usability increases
effectiveness and efficiency, and provides ease of use,
less support is needed to learn and use the product.
• Breastcancer.org redesign resulted in a decrease
of average weekly support requests by 80% from 15 to
3. Moreover, one year after the redesign, help desk
costs decreased by 69% from $932 per month to $293
per month (Usability First, 2007).
• Lucy.com invested on usability to improve
descriptions and images on product pages. Support
calls dropped by 20% (Bevan, 2005).
• Ford Motor Company invested on usability and
reduced the number of support calls from an average of
3 to none. The company saved $100,000 on support
costs (Bevan, 2005).
• A printer manufacturer released a printer driver
without any usability testing. In a month, they received
over 50,000 support calls because installing the driver
was difficult. Responding these calls caused company
around $500,000 that month. In addition, company had
to send a letter of apology and patch diskettes to the

4
International Annual Conference of the American Society for Engineering Management
users. Each of these letter and diskette package cost
the company $3. As a result, company spent around
$900,000 to resolve the problem (Bias and Mayhew,
1994).
Increased Customer Satisfaction and Decreased
Employee Turnover. A major characteristic of
usability is user satisfaction. Usable products satisfy
the users, and prevent frustration. “Systems matching
user needs, dramatically improve satisfaction” (Bevan,
2005, 9). Employees using satisfactory products are
less likely to resign. Thus, usability reduces employee
turnover. Following are some cases from the literature.
• Usability raised user satisfaction ratings for a
system by 40% in a 1992 Gartner Group study
(Harrison, et al., 1994).
• Federal Emergency Management Agency
(FEMA) improved the effectiveness of their web site
by usability testing in 2006. The satisfaction ratings
improved by 71% (Usability First, 2012).
• In La Quinta Hotel case, user satisfaction
increased by 28% (Peterson, 2007).
• Employee turnover cost is 1.5 times an
employee’s yearly salary (Karat, 2005). In a case study
cited in Karat (2005), a 10% reduction in employee
turnover in two divisions of a hotel chain was more
than the total profits made by those divisions.
There are many more benefits of usability
mentioned in the literature, such as litigation
deterrence, reduced supervisory assistance, and
decreased installation, configuration, and deployment
costs. However, the scope of this study is limited to
the benefits that are mostly cited.
Discussion
The benefits from usability discussed above affect the
cost of quality of organizations in a positive way. All
the benefits mentioned above decreases the total cost of
quality of development organizations as usability
investments in itself is a huge prevention cost.
Usability investments not only reduces failure cost
(like error, failure, maintenance and support cost,
employee turnover cost, and litigation cost) but also
reduces prevention cost (like training cost,
development cost) and helps to increase the benefits by
decreasing the opportunity cost. Exhibit 2 summarizes
the findings of this study.
Exhibit 2. Impact of Usability Investment in the Cost
of Quality
Usability
Benefit
Sample Cases
Change in
COQ
Decreased
Training
Time
 American Express
%83
 Diagnostic
Research 48%
Decreases
Prevention
Cost
Reduced
Development
and
Maintenance
Cost
 Boehm (1981)
estimated early-to-
late ratio of 1:40-
100
 American Airlines
60-90%
Decreases
Prevention
and Failure
Cost
Decreased
Error
 90% an e-
commerce web site
(Karat, 2005)
 The New York
Stock Exchange
(NYSE) 10%
Decreases
Failure
Cost
Reduced
Support Cost
 Breastcancer.org
69%
 Ford Motor
Company 100%
 Lucy.com 20%
Decreases
Failure
Cost
Decreased
Employee
Turnover
 10% Karat (2005)
Decreases
Failure
Cost
Increased
Sales
 Staples.com 491%
 La Quinta 83%
 DEC Rally 80%
Decreases
Opportunity
Cost
Decreased
Task Time
 American Express
325%
 Performance Bikes
40%
 Sun Microsystems
51%
Decreases
Opportunity
Cost
Increased
Customer
Satisfaction
 FEMA 71%
 La Quinta 28%
Decreases
Opportunity
Cost
Decreased training time decreases training cost,
which decreases prevention cost and saves time.
Reduced development cost also reduces prevention
cost. Usability investment is a prevention cost which
reduces the total cost of quality by reducing failure
cost, opportunity cost and other prevention costs.
Decrease in user errors reduces failure cost which
is due to improvement in the quality through usability.
Reduced maintenance costs reduce failure cost due to
emergency or failure maintenance. Reduced support
costs also reduce failure cost as support is needed if

The Impact of Usability on The Cost of Quality
5
there is a problem. Decreased employee turnover
decreases failure cost as usability makes employees
satisfied with their job.
Increased sales decreases the opportunity cost
whereby decreasing the cost of quality. Increasing
sales was a lost opportunity before usability
investments. The examples of increased sales in the
literature show the improvements in websites and
software, which improves the overall quality and
decreases the total cost of quality. Decreased task time
also decreases opportunity cost as it was a lost
opportunity before usability investments. Moreover, it
increases the quality in terms of time. Increased
customer satisfaction decreases opportunity cost as it
was a lost opportunity before usability investments.
What is evident from this analysis is that the
research to date shows that the major benefits in
implementing usability is the reduction of costs- a
strategy that is easier to sell to management than
increasing customer satisfaction which is hard to
quantify. Though, customer satisfaction is important
and does result when usability is implemented, the
more practical and measurable, thus saleable approach
for engineering managers is to focus on the cost
reduction benefits.
This paper shows how usability investments can
decrease the total cost of quality and improve the
quality of development organizations. This indicates
that investments on usability play a vital role in the
cost of quality of organizations. Further work can be
done to calculate and compare the total cost of quality
before and after usability investments.
References
Beruvides, Mario G., Diego A. Sandoval-Chavez, “The
Importance of Opportunity Costs in Organizations:
A Cost of Quality Perspective,” Proceedings from
the 1997 Management of Technology VI
Conference, Goteborg, Sweden. (June 25-28,
1997), pp. 1008-1014.
Bevan, N. “Cost benefit analysis. TRUMP report,”
http://www.usability.serco.com, (cited 2001)
Bevan, N., “Cost benefits evidence and case studies,”
http://www.usability.serco.com, (cited 2005)
Bias, Randolph G., and Deborah J. Mayhew, Cost-
justifying usability, Morgan Kaufmann Publishers
(1994).
Black, J., “Usability is Next to Profitability,”
http://www.businessweek.com/technology/content/
dec2002/tc2002124_2181.htm, (cited 2002).
Boehm, Barry W., Software Engineering Economics,
Prentice Hall (1981).
Bond, C., “Time is money: impacts of usability at a call
center” Ergonomics in Design: The Quarterly of
Human Factors Applications, Vol. 15, No. 17
(2007), pp. 17-22.
Cope, Micheal, E., and Kevin C. Uliano, “Cost-
justifying Usability Engineering: A Real World
Example,” Proceedings from the Human Factors
and Ergonomics Society 39th Annual Meeting,
Designing for the Global Village, (October 1995)
263-267.
Donahue, George M., “Usability and the Bottom Line,”
Software, IEEE, Vol. 18, No. 1, pp. 31-37.
Feigenbaum, A. V., Total quality control, Harvard
Business Review, 93-101. (1956, Nov/Dec).
Feigenbaum, A. V., Total Quality Control, Third
edition, McGraw-Hill, Inc., New York (1983).
Harrison, M., Henneman, R., and Blatt, L., “Design of
a human factors cost-justification tool,” In R.G.
Bias and D.J. Mayhew, Eds. Academic Press, New
York, (1994), pp. 203-239.
Holzinger, A., “Usability engineering for software
developers,” Communications of the ACM, Vol.
48, No. 1, pp. 71-74.
Human Factors International (HFI) “HFI Helps
Staples.com Boost Repeat Customers by 67%,”
http://www.humanfactors.com/about/staples.asp
Gibbs, W. Wayt, “Taking Computers to Task,”
Scientific American Digital, Vol. 7, pp. 1-9.
ISO/IEC. 9241-14 Ergonomic requirements for office
work with visual display terminals (VDT)s - Part
14 Menu dialogues, ISO/IEC 9241-14: 1998 (E),
1998.
Karat, C., “Cost-benefit and business case analysis of
usability engineering,” Proceedings from the ACM
SIGCHI Conference on Human Factors in
Computing Systems, Amsterdam, (1993b).
Karat, C., “A business case approach to usability cost
justification for the web,” In R. G. Bias & D. J.
Mayhew (Eds.), Cost Justifying Usability: An
Update for the Information Age, San Francisco,
CA: Morgan Kaufmann (2005), pp. 103-142.
Mayhew, D. J., and Tremaine, M. M., “A basic
framework,” In R. G. Bias & D. J. Mayhew (Eds.),
Cost Justifying Usability: An Update for the
Information Age, San Francisco, CA: Morgan
Kaufmann (2005), pp. 41-102.
Nielsen, J., & Gilutz, S., “Usability return on
investment” Nielsen Norman Group (2003).
Peterson, M. R., “UX Increases Revenue; Two Case
Studies,” User Experience, Vol. 6, No. 2 (2007),
pp. 4-7.
Pressman, Roger S., Software Engineering: A
Practitioner's Approach, third edition, McGraw-
Hill (1992).
Rohn, J. A., “Cost-justifying usability in vendor
companies,” In R. G. Bias & D. J. Mayhew (Eds.),
Cost Justifying Usability: An Update for the
Information Age, San Francisco, CA: Morgan
Kaufmann (2005), pp. 185-214.

6
International Annual Conference of the American Society for Engineering Management
Sandoval-Chavez, Diego A., Mario G. Beruvides,
“Using Opportunity Costs to determine the Costs
of Quality: A Case Study in a Continuous-Process
Industry,” The Engineering Economist, Vol. 43,
No. 2 (1998), pp. 107-124.
Usability First “Usability ROI Case Study:
Breastcancer.org Discussion Forums,”
www.usabilityfirst.com/documents/U1st_BCO_Ca
seStudy.pdf, (cited 2007).
Usability First “Case Studies, FEMA (Federal
Emergency Management Agency)”
http://www.usabilityfirst.com/about-
usability/usability-roi/case-studies, (cited 2012).
Usable Information Technology “Jacob Nielsen’s
Alertbox, Loyalty on the Web”
http://www.useit.com/alertbox/9708a.html, (cited
August 1997)
Wixon, Dennis, Sandy Jones, “Usability for Fun and
Profit: A case study of the design of DEC RALLY
version 2”, Human-computer Interface Design;
Success Stories, Emerging Methods, and Real-
world Context (1995), pp.3-35.
About the Authors
Burchan Aydin is a PhD student in Industrial
Engineering program at Texas Tech University. He
receieved his B.E. in Industrial Engineering from
Middle East Technical University, Ankara, Turkey. He
received his M.A.A degree in organizational
development from the University of the Incarnate
Word, San Antonio, Texas. His research interests
include human factors, ergonomics, usability, and
engineering management. He is currently a teaching
assistant for department of Engineering Technology,
TTU.
Himlona Palikhe., Himlona Palikhe M.S.I.E. is a
graduate student in Systems and Engineering
Management program at Texas Tech University. She
received her M.S. degree in Industrial Engineering
from Texas Tech University and B.E. in Electronics
and Communication from Tribhuvan University,
Kathmandu, Nepal. Her research interests include
engineering management, engineering economics, cost
of quality, quality control and electric utilities. She is
currently a teaching assistant at Texas Tech University.
Mario G. Beruvides Ph.D., P.E. is the AT&T
professor of industrial engineering of Texas Tech
University and Director of the Laboratory for Systems
Solutions. He Received his PH.D from Virgina
Polytechnic Institute @ State University and an MSIE
and BSME from the University of Miami. His major
areas of interest are advanced economic analysis,
management of technology, engineering management,
white-collar/knowledge work, productivity and
performance measurement and systems engineering.