Content uploaded by Layla Hasan
Author content
All content in this area was uploaded by Layla Hasan on May 31, 2016
Content may be subject to copyright.
Managing Information in the Digital Economy: Issues & Solutions
__________________________________________________________________________________________
* Working toward PhD in MIS.
233
Criteria for Evaluating Quality of Websites
Layla R. Hasan*, Zarqa Private University, Zarqa, Jordan, layla@zpu.edu.jo
Emad Abuelrub, Zarqa Private University, Zarqa, Jordan, abuelrub@zpu.edu.jo
Abstract
In the age of information technology and as the
dependency of web technology increases, so does
the need to assess the factors associated with
website success. This paper reviewed the most
recent evaluation criteria methods which were used
in different web services, and it proposes a general
criteria for evaluating the quality of any website
service regardless of the type of service that it
offers. The dimensions of the criteria are content
quality, design quality, organization quality, and
user-friendly quality. These dimensions together
with their comprehensive indicators and check list
can be used by web designers and developers to
create quality websites.
1. Introduction
In the age of information technology and with the
rapid growth of the World Wide Web, the
spectacular growth trend in e-business, that has
been experienced so far, is expected to continue
[21]. Companies seeking to achieve significant
benefits through e-business need to create an
effective, and usable web presence to ensure
successful interaction and communication with
their employees, partners, and customers [6].
In the last decade, we have seen a proliferation
of websites with a tremendous amount of
information either with high quality, or with low
quality, as well as sites that are outright misleading
[11, 15]. The explosion of the web has determined
the need of measurement criteria to evaluate the
aspects related to the quality in use, such as
usability and accessibility of a web application.
Numerous studies have focused on the designs
of websites for general information seeking
purposes and for electronic commerce purposes
[38], since the design and commercial development
of websites are very critical to e-commerce success
[5]. Although, there has been a significant research
on supporting electronic commerce, there is no
standard framework defining website effectiveness
[6].
This paper reviewed the most recent evaluation
criteria methods which were used in different web
services, and proposes a general comprehensive
criteria for evaluating the quality of any website
services regardless of the type of service that it
offers. The remaining of the paper is as follows.
Section 2 gives a brief reviewing of literature.
Section 3 defines the dimensions of the proposed
criteria and its indicators. Section 4 concludes the
paper and suggests some future work.
2. Literature Review
Websites characteristics are important; they have
been a constant concern of research in different
domains and they were widely studied in the e-
commerce literature. This section briefly reviews the
previous studies according to the website service.
Business and commercial websites were
studied from different perspectives. Some researchers
investigated website features or factors that are
critical to e-business success [22, 23, 25, 27]. Other
researchers address key issues of online business
from customer satisfaction perspective [16, 33, 35,
38]. Other researchers developed generic tool for the
assessment of website quality [4, 5, 10, 12, 21, 26,
30]. Some researchers concentrated on some
important features; they either proposed a framework
to measure the important features of the website or
used previous models to find out to which extent e-
business websites incorporate these important
features. Basu [6], Lim [20], and Singh and Fisher
[32] concentrated in their study on website usability,
Heimlich and Wang [16] proposed key issues of
website’s structure, Cao and Zhang [7] examined
factors that affect e-commerce website design.
Heimlich [15] discussed content evaluation of
websites, while Fogg et al. [11] investigated how
different elements of websites affect people’s
perception of credibility.
Educational websites were studied from many
different perspectives. Zhang and Dran [38]
developed a theoretical framework for evaluating
website quality from a user satisfaction perspective.
Others concentrated on some specific features of
websites, for example Lautenbach et al. [19]
developed a framework to measure usability of
websites, while Yoo and Jin [37] investigated and
evaluated the design of university websites.
Banking websites were studied from many
different perspectives using different models. Diniz et
al. [9] and Zhang and Dran [38] proposed a model to
evaluate and build digital business environment from
the user’s point of view. While other researchers
proposed a specific framework to evaluate the
Internet banking websites and the service quality of
Internet banking [2, 34, 36]. Other researchers
adopted through their studies a number of previous
models to evaluate Internet banking websites [3, 14,
29], such as Diniz and Herey Models.
Governmental websites were studied from
different perspectives. Zhang and Dran [38]
developed a theoretical framework for evaluating
website quality from a user satisfaction perspective,
while Krauss [18] identified comprehensive quality
dimensions that can be used to rate website quality
features that are important to e-government websites.
Others concentrated on some features, such as
content, design [17], accessibility, quality and
privacy [8], usability and content accessibility [24],
and accessibility [1].
Managing Information in the Digital Economy: Issues & Solutions
234
3. The Proposed Criteria
After deeply and comprehensive reviewing of
different evaluation methods and their elements that
were used in different services over the Internet, we
propose 4-dimensions’s criteria which is
comprehensive and included all the previous
dimensions and elements. The dimensions of the
proposed criteria are content quality, design quality,
organization quality, and user-friendly quality. Our
criteria includes all main indicators of the previous
studies besides some indicators in which we see
them important from our own experience.
3.1 Dimension 1: Content Quality
It is generally agreed that content quality is an
important dimension which deals with the
characteristics of websites’ information. This
dimension has been addressed by a variety of
researchers over the previous years.
The following indicators and check elements are
the most important relating to the content quality
dimension:
1. Timely: Currency of websites’ information,
how frequently the website is updated, and
when the site was updated [4, 5, 7, 8, 10, 11,
13, 15, 16, 17, 19, 23, 25, 27, 28, 30, 33, 38].
2. Relevant: Comprehensive information,
provide the right level of details [5, 10, 12,
13, 15, 26, 27, 28, 38], informative, meaning,
value added [17, 35], and fit to users’ need
[7, 17, 18, 26, 28, 31]. So, websites include
information about the organization’s
objectives [3, 16, 17, 29], organization’s
history [3, 6], customers [13, 16, 29],
products or services [6, 23, 29], and
photographs of organization’s facilities [6].
3. Multilanguage/Culture: Information is
available in different languages [1, 10, 11,
17, 18, 21, 34], suitable to different cultures
[10, 18], and meets the needs of all
customers.
4. Variety of presentation: Information is
presented in different forms (text, video,
audio, … ) [17, 31, 33].
5. Accuracy: Information is precise; there is no
spelling errors or grammar errors [4, 5, 7, 10,
12, 13, 15, 17, 18, 23, 26, 27, 31, 33, 38],
and the sources of information is identified
[11, 13, 15, 28].
6. Objective: Information is presented in
objective manner without political, cultural,
religious, or institutional biases [13, 15, 28].
7. Authority: The credibility of websites’
information is clearly identified by providing
information about: Organization’s physical
address [11, 17], sponsor(s) [10, 13, 15, 16,
21, 26, 28], manager(s) [10, 15, 16, 28],
specification of sites’ manager(s) [10, 15],
identification of copyright [28], email to
manager of the website [11, 13, 16, 28], and
metadata elements exist [8, 10].
3.2 Dimension 2: Design Quality
This dimension concerns the visual characteristics of
websites’ design that attracts the users and
encourages them to stay longer time viewing the
website and re-enter it. Most previous studies cover
this dimension for its importance, they called it
design, display, or presentation of information.
The following indicators and check elements are
the most important relating to the design quality
dimension:
1. Attractive: The design of the website is
innovative [18], has an aesthetic effect [2, 4-7,
11, 16, 17, 18, 29, 31, 36], and has an
emotional appeal [4-7, 10, 18, 20, 23, 31, 38].
2. Appropriateness: The design of the website is
appropriate to the type of the website [5],
images witin the pages serve their functional
purposes [15, 28], images, colors, and text are
appropriately balanced on each page [16], and a
fewer number of screens in each page [17, 37].
3. Color: Effective use of background and text
color [1, 6, 16, 17, 19, 22, 31, 32]. According to
background color, light colors are preferred to
be used [37], while concerning text color, it
shouldn’t exceed four colors within the same
page [37].
4. Image/Sound/Video: It concerns the non text
elements which are used within the website [2,
6, 15, 16, 17-19, 26, 28, 29, 31, 32]. Few
number of image/sound/video should be used
[30, 37], and size of image/sound/video should
be small since large number of
image/sound/video per page and large size of
image/sound/video will slow downloading the
page. Alternative text should be used for all
non text elements [17, 24, 30].
5. Text: Characteristics of text used within
websites’ pages [6, 19, 32]. There should be
consistency in text; one font size and one font
style except for titles [37], text font should be
chosen among the most readable ones [1, 17,
30] with relative size [16, 30], pages shouldn’t
use all capital letters unless in titles or headings
[30, 37], pages should use white space or
breathing space between page elements [22,
37], different or multiple headings such as
titles, sub titles, sub sub titles is preferred as
appropriate [37], if pages use scrolling text it
shouldn’t hide a large amount of information
[37], and pages should show the text first then
the image(s) [37].
3.3 Dimension 3: Organization Quality
This dimension concerns the logical grouping,
categorization, or structure of websites’ elements in
order to help the user to reach the required
information quickly, navigate easily, feel comfortable
within its layout consistency, and keep informative
that he/she is still in the same website.
The following indicators and check elements are
the most important relating to the organization
quality dimension:
Managing Information in the Digital Economy: Issues & Solutions
235
1. Index: An index or links to all the website’s
pages is available from the main page [10, 16,
36].
2. Mapping: Adequate website map or navigation
bar/menu is available [2, 10, 17, 21, 29, 30], a
user can know the current page that he/she is in
[16, 17, 24].
3. Consistency: A general layout of each page
is consistent [6, 9, 16, 17, 19, 36].
4. Links: Links work properly; it should take
the user where he/she is intended to go [11,
13, 15, 17, 23, 30-32], assistant links
available in each page so that the user can
get back to main page from every section of
the website, the links can help the user to
return to top of the page within the long
pages of the website, user can return to the
original website when he/she follows
external link of any page [6, 10, 16], worthy
links that take user to other related websites
is available [2, 13, 17, 31], no dead links
[30], and the link’s color is changed after the
user has visited it [16].
5. Logo: Organization logo is noticeable in
every page of the website [21].
3.4 Dimension 4: User-friendly Quality
Nearly, all previous studies included this dimension
or one or more of its indicators in their criteria
model because of its importance. It concerns issues
that help users to find the needed information
quickly, to maintain specific level of performance,
and to emphasize the existence of interaction
between user and website.
The following indicators and check elements are
the most important relating to the design user-
friendly dimension:
1. Usability: Website is easy to use, understand,
operate, find information, or navigate [1, 3-6,
9-12, 15, 16, 17-20, 22-24, 27, 29, 30, 32,
33, 36, 38], it is easy to find the website
using external websites [6, 15, 26, 33], and it
is clear to user that new information is added
to the website.
2. Reliability: Website’s address is appropriate
[11, 13, 26, 34] and easy to remember [34],
short download time [8, 11, 17, 26, 27, 29,
31, 33, 35], multi browser support [1, 17,
24], work properly using different screen
settings, few ads are in the website’s pages
[2], there is away to measure its efficiency
[6], and the website is available 7 days/week,
20 hours/day [1, 3, 6, 9, 11, 12, 27, 35].
3. Interactive features: The website has clear
instructions to use different
parts/sections/forms of it [34], help function
and clear error messages are available [2, 13,
15, 22, 23, 30, 31, 33, 36], FAQ is available
[2, 21, 22, 27, 29, 34], effective internal
search tool [2, 3, 6, 7, 9, 11, 14-16, 17, 18,
21, 23, 29, 37, 38], communication channel
and feedback exist between user and website
through email, chat rooms, online
community, or suggestion form [2, 3, 5-7, 9-12,
14, 15-18, 15-18, 21, 22, 25, 26, 27, 29, 31, 34,
35], follow-up service is offered [23], and users
can track their order easily [23].
4. Security/Privacy: Effective mechanisms are
used to keep the transactions secure [2-5, 9, 10,
12, 13, 17, 18, 20-23, 27, 29, 31, 38]. Privacy
of personal information is handled [2, 5, 6, 9,
13, 17, 18, 20-23, 27, 29].
5. Customization: Tailoring the content of the
website according to the needs of specific users
[2-6, 9, 11, 14, 25, 26, 31, 35].
4. Conclusion and Future Work
This paper reviewed the most recent evaluation
methods which were used in different web services,
and it proposes general and comprehensive criteria
for evaluating the quality of any website service
regardless of the type of service that it offers. The
dimensions of the criteria are: content quality, design
quality, organization quality, and user-friendly
quality.
These dimensions with their indicators could be
operationalized and converted into a questionnaire;
the questionnaire could be applied to different
domains. Results from the analyzing of the
questionnaire will help in evaluating these
dimensions and their indicators and make the needed
update on them.
5. References
[1] Abanumy, A. Al-Badi, A. and Mayhew, P. “e-
Government Website Accessibility: In-Depth
Evaluation of Saudi Arabia and Oman,” Electronic
Journal of e-Government (3:3), 2005, pp. 99-106.
[2] Achour, H. and Bensedrine, N. “An Evaluation of
Internet Banking and Online Brokerage in Tunisia,”
in Proceedings of the 1
st
International Conference on
E-Business and E-learning, Jordan, 2005.
[3] Awamleh, R. and Fernandes, C. “Internet
Banking: An Empirical Investigation into the Extent
of Adoption by Banks and the Determinants of
Customer Satisfaction in the United Arab Emirates,”
Journal of Internet Banking and Commerce (10:1),
2005.
[4] Barnes, S. and Vidgen, R. “Assessing the Quality
of Auction Web Sites,” in Proceedings of the 34
th
International Conference on System Sciences,
2001.
[5] Barnes, S. and Vidgen, R. “An Integrative
Approach to the Assessment of e-Commerce
Quality,” Journal of Electronic Commerce Research
(3:3), 2002.
[6] Basu, A. “Context-Driven Assessment of
Commercial Web Sites,” in Proceedings of the 35
th
Hawaii International Conference on System Sciences,
2002.
[7] Cao, M. and Zhang, Q. “Evaluating e-Commerce
Web Site Design: A Customer’s Perspective,” in
Proceedings of Decision Sciences Institute Annual
Meeting, 2002, pp. 1186-1191.
[8] Choudrie, J. Ghinea, G. and Weerakkody, V.
“Evaluating Global e-Government Sites: A View
Managing Information in the Digital Economy: Issues & Solutions
236
Using Web Diagnostic Tools,” Electronic Journal
of e-Government (2:2), 2004, pp. 105-114.
[9] Diniz, E. Porto, M. R. and Adachi, T. “Internet
Banking in Brazil: Evaluation of Functionality,
Reliability, and Usability,” Electronic Journal of
Information Systems Evaluation (8:1), 2005.
[10] Fitzpatrick, R. “Additional Quality Factors for
the World Wide Web,” in Proceedings of the 2
nd
World Congress for Software Quality, Japan, 2000.
[11] Fogg, B. J. Marshall, J. Laraki, O. Osipovich,
A. Varma, C. Fang, N. Paul, J. Rangnekar, A. Shon,
J. Swani, P. and Treinen, M. “What Makes Web
Sites Credible? A Report on a Large Quantitative
Study,” Computer Human Interaction (3:1), 2001,
pp. 61-68.
[12] Gledec, G. “Evaluating Web Site Quality,” in
Proceedings of the 7
th
Internet Users Conference
(CUC2005), Croatia, 2005.
[13] Granath, K. “Evaluating Web Pages,” R
etrieved
2006,
from: http://www.lib.umt.edu/
research/guide/ int_evalweb.htm.
[14] Guru, K. B. Shanmugam, B. Alam, N. and
Perera, J. C. “An Evaluation of Internet Banking
Sites in Islamic Countries,” Journal of Internet
Banking and Commerce (6:1), 2001.
[15] Heimlich, J. “Evaluating the Content of Web
Sites,” Environmental Education and Training
Partnership Resource Library, Ohio State
University Extension, USA, 1999.
[16] Heimlich, J. and Wang, K. “Evaluating the
Structure of Web Sites,” Environmental Education
and Training Partnership Resource Library, Ohio
State University Extension, USA, 1999.
[17] Kokkinaki, I. A. Mylonas, S. and Mina, S. “e-
Government Initiatives in Cyprus,” e-Government
Workshop, Brunel University, UK, 2005.
[18] Krauss, K. “Testing an e-Government Website
Quality Questionnaire: A Pilot Study,” in
Proceedings of the 5
th
Annual Conference on World
Wide Web Applications (WWW2003), 2003.
[19] Lautenbach, M. A. E. Schegget, I. S. Schoute,
A. M. and Witteman, C. L. M. “Evaluating the
Usability of Web Pages: A Case Study,” Retrieved
206, from: http://www.phil.uu.nl/preprints/
ckipreprints/PREPRINTS /preprint011.pdf.
[20] Lim, K. “Security and Motivational Factors of
e-Shopping Web Site Usage,” in Proceedings of
Decision Sciences Institute Annual Meeting, 2002.
[21] Lin, F. Huarng, K. Chen, Y. and Lin, S.
“Quality Evaluation of Web Services,” in
Proceedings of IEEE International Conference on
e-Commerce Technology for Dynamic e-Business,
2004.
[22] Lin, O. and Joyce, D. “Critical Success Factors
for Online Auction Web Sites,” in Proceedings of
the 17
th
NACCQ, 2004.
[23] Liu, C. and Arnett, K. “Exploring the Factors
Associated with Web Site Success in the Context of
Electronic Commerce,” Information and
Management (38), 2000, pp. 23-33.
[24] Ma, H. and Zaphiris, P. “The Usability and
Content Accessibility of the e-Government in the
UK,” in Proceedings of Human Computer
Interaction International Conference, Greece, 2003.
[25] Madeja, N. and Schoder, D. “Designed for
Success: Empirical Evidence on Features of
Corporate Web Pages,” in the Proceedings of the 36
th
Hawaii International Conference on System Sciences,
2003.
[26] Mich, L. Franch, M. and Gaio, L. “Evaluating
and Designing Web Site Quality,” Feature Article,
IEEE Multimedia, IEEE Computer Society, 2003.
[27] Molla, A. and Licker, S. P. “e-Commerce
Systems Success: An Attempt to Extend and
Respecify the Delone and Maclean of IS Success,”
Journal of Electronic Commerce Research (2:4),
2001, pp. 131-141.
[28] Osborne, C. and Rinalducci, J. “Evaluation of
Web Based Resources within the Art History
Discipline,” Technical Report, University of North
Carolina, 2002.
[29] Paynter, J. and Chung, W. “An Evaluation of
Internet Banking in New Zealand,” in Proceedings of
the 35
th
Hawaii International Conference on System
Sciences, 2002.
[30] Signore, O. “A Comprehensive Model for Web
Sites Quality,” in Proceedings of the 7
th
IEEE
International Symposium on Web Site Evolution
(WSE’05), 2005.
[31] Singh, I. and Sook, A. “An Evaluation of the
Usability of South African University Web Sites,” in
Proceedings of the 2002 CITTE Conference, Durban,
South Africa, 2002.
[32] Singh, M. and Fisher, J. “Electronic Commerce
Issues: A Discussion of Two Exploratory Studies,” in
the Proceedings of the Electronic 3
rd
Annual
Conference on Electronic Commerce, Victoria
University, New Zealand, November 1999.
[33] Srivihok, A. “An Assessment Tool for
Electronic Commerce: End User Evaluation of Web
Commerce Sites,” Technical Report, Faculty of
Science, Kasetsart University, Thailand, 2000.
[34] Vijayan, P. and Shanmugam, B. “Service
Quality Evaluation of Internet Banking in Malaysia,”
Journal of Internet Banking and Commerce, (8:1),
2003.
[35] Webpartner, Retrieved June 2005, from:
www.webpartner.com
[36] Wenham, D. and Zaphiris, P. “User Interface
Evaluation Methods for Internet Banking Web Sites:
A Review, Evaluation and Case Study,” In J. Jacko,
C. Stephanidis (Ed.), Human-Computer Interaction,
Theory and Practice, Lawrence Erlbaum, Mehwah,
USA, 2003, pp. 721-725.
[37] Yoo, S. and Jin, J. “Evaluation of the Home
Page of the Top 100 University Web Sites,” Academy
of Information and Management Sciences (8:2),
2004, pp. 57-69.
[38] Zhang, P. and Dran, G. “Expectations and
Ranking of Website Quality Features: Results of Two
Studies on User Perceptions,” in Proceedings of the
34
th
Hawaii International Conference on System
Sciences, 2001.