Content uploaded by Mercy Nkanata
Author content
All content in this area was uploaded by Mercy Nkanata on May 23, 2020
Content may be subject to copyright.
300
Applying DeLone and McLean information systems success
model in the evaluation of e-government initiatives: a literature
review
Mercy Gacheri Nkanata
1
Department of Information Studies, University of Zululand
Mercy.nkanata@gmail.com
Abstract
This paper is a review of existing literature on the use of DeLone and McLean’s
(1992, 2003) information systems success model in evaluating e-government
initiatives. Electronic governments’ practices are universally acknowledged to aid
service delivery to the citizens. The e-government evaluation is considered important
because of the enormous investment by governments to deliver effective services. E-
government evaluation is complex both in theory and practice and the debate
amongst researchers is not only in the evaluation complexity but also about the most
appropriate approach to use. The DeLone and McLean information systems success
model is one of the most widely used for measuring information systems’ success.
Many studies have utilised this model to evaluate the success of information systems
over the years. This paper confirms that the model has been applied in various IS
domains, but in e-government evaluation the usage is low;, e-government is seen as
being at a cross-roads between various research domains such as computer science
and information systems. The analysis established that even when the model is
applied in e-government, some of the dimensions are selected and combined with
others closely related to the study.
Keywords: DeLone and McLean Model, e-government success evaluation, e-
government; information systems success
1. Mercy Gacheri Nkanata is a PhD student in the Department of Information Studies at the University
of Zululand, South Africa.
301
1. Introduction
Governments across the world have embraced the implementation of electronic
government systems with the aim of realising many benefits, for themselves as
providers of public services as well as for the citizens who are the users of the
services (Weerakkody, Irani, Lee, Hindi, & Osman, 2016). E-government is commonly
defined as the use of information and communication technologies (ICTs) by the
government, to deliver services and information to the citizens, the business
community and all other branches of government (Nam, 2014).
E-government initiatives have become effective tools for governance reforms in
confronting public service-related challenges. This has resulted in an increase in
transparency, reduction in corruption, improved efficiency, reduction in administrative
cost, and overall better quality of life for citizens (Ali, Hoque, & Alam, 2018). Deng,
Karunasena, & Xu, (2018) observed that the rapid development of e-Governments
projects all over the world has created an urgent need for continuous evaluation. In
addition, e-government initiatives consume a significant amount of public funds hence
concerned government agencies should be able to justify some form of return on
investment.
It is therefore essential that such major government initiatives undergo post-
implementation assessment. Shan, Wang, Wang, Hao, & Hua (2011:174) assert that,
based on the outcome of the results; government could take necessary and relevant
action. This kind of evaluation can assist government agencies to establish whether
or not they are performing the required tasks and delivering effective and efficient
services to meet the citizens’ expectation.
The evaluation of an information system’s success is considered an important aspect
of the information system field both in practice and research. E-government project
evaluation helps to identify the strengths, weaknesses and best practices for both
local and international integration. However, the approaches used for the evaluation
have changed over the years, as context, purpose and IT impact evolve (Delone &
Mclean, 2016). Although various information success models have been applied in
302
different contexts, in the area of e-government insufficient research has been carried
out to identify the success of e-government measures from citizens’ perspectives.
Belanger & Carter (2012) analysed 30 e-government peer reviewed journals, and
only two used citizen- based data, confirming that the impact of e-government on the
citizens has not been well researched. From the e-government research available,
little is known about the impact and result of e-government projects or their capacity
to bring about real changes in organisation that would improve public service delivery.
Luna-Reyes et al., (2012) acknowledge that efforts have been made to evaluate
different dimensions of electronic government systems. Most of the areas covered
have focused on the impact of e-government on employees (Gable, Sedera, & Chan,
2008; Scott, H., & Golden, 2009 ; Stefanovic, Marjanovic, Delic, Culibrk, & Lalic,
2016) on government-to-citizen systems (Wang & Liao, 2008), and e-government
websites (Teo, Srivastava & Jiang, 2008; Huang & Benyoucef, 2014; Verkijika & De
Wet, 2018)
Scott, Delone, & Golden (2016) point out that more research needs to be done to
ascertain the impact and results of e-government projects. The e-government
success measurements are not well understood by the research communities as well
as by practitioners. More research focusing on the holistic approach of examining e-
government initiatives from the citizens’ perspective, as the users of the system,
needs to be carried out.
Rana, Dwivedi, Williams & Lal, (2015:42) acknowledge that DeLone and McLean’s
IS model has gained much attention among researchers, but point out that not much
research has been done on the evaluation of the success of e-government systems,
using the IS model. This review provides an understanding of the extent to which
DeLone and McLean’s information system success model is being utilised in the
evaluation of e-government initiatives.
2. Purpose
This literature review seeks to answer the following research questions: (1) what is
the extent of the use of DeLone and Mclean’s IS success model in e-government
303
evaluation research? (2) Are the six dimensions of DeLone and McLean’s IS model
being used in e-government evaluation research? (3) What are the challenges of
using the D&M Model in evaluating e-government information systems?
3. Theoretical background
The evaluation of information systems (IS) success has been discussed widely in the
field of information systems. (Shan et al., 2011:176) point out that the evaluation of e-
government projects pinpoints strengths and weaknesses, tracks national progress
and moves toward an inclusive information society. Moreover, the rapid development
in e-government has created an urgent need for continuous performance evaluation
of e-government projects in the world (Alcaide-muñoz & Bolívar, 2015).
Wang, Wang, Lin & Tsai, (2019) note that research in the field of information systems’
(IS) success has been informed by a number of models which include: the original
D&M IS success model, which was a comprehensive review of different IS models
(Delone & Mclean, 1992); the updated D&M IS success model comprising six
dimensions (Delone & Mclean, 2003 ); e- commerce systems’ success model
adopted from a D&M updated model (Wang, 2008); model of e-government systems
success based on the updated D&M model IS success model (Wang & Liao, 2008);
modified D&M IS success model introduced educational equality instead of net
benefit (Zheng & Liang, 2017); and modified D&M IS success model for the
assessment of cloud e-bookcase system (Chiu, Chao, Kao, Pu, & Huang, 2016).
This review is mainly informed by the use of DeLone & McLean's (2003) updated
information success model which was introduced by DeLone and McLean in 1992. In
their search for information systems’ success measures, they found a wide variety of
measures in different studies. However, after comprehensively reviewing literature in
180 empirical studies, they grouped the IS success dimensions into six main
categories: (1) systems quality (2) information quality (3) use (4) user satisfaction (5)
individual impact and (6) organisational impact (Delone & Mclean, 1992). Later, the
model received considerable criticism by Seddon (1997). Among other things, the
304
author claimed that the model is confusing because it mixes processes and casual
explanations of information systems.
Ten years later, DeLone and McLean published an updated version of the IS success
model, which added service quality as a new dimension of measuring IS success,
and they merged organisational and individual impacts into a single impact variable
called “net benefit” (Stefanovic et al., 2016). The emphasis of the DeLone & McLean,
(2003) updated model is on the importance of measuring the success of information
systems. The model comprises six dimensions of success. Ondego & Moturi ( 2016)
state that the model can be used to show a causal relationship: how ICT project
implementation affects IS quality, and in turn affects the perceived benefits.
Stefanovic et al. (2016:19) point out that the D&M model is the most widely used
model in evaluating IS success, and that it has been used to measure the success of
various information systems. The model has been used in the assessment of e-
government projects, as e-government systems are considered to be an aspect of
information systems. Within the e-government context, citizens use web-based
applications to search for government information, and to conduct transactions. This
web-based application is an information system phenomenon that is best evaluated
using the IS success model. According to DeLone & McLean's (2003) model, there
are six attributes of successful information systems, which are multidimensional and
closely interrelated, as illustrated in Figure 1 below:
305
System quality in an e-government system is about the performance of the system in
terms of its ease of use and learning integration. System quality is a key determining
factor of e-government system use (Teo et al., 2008): how flexible and reliable the e-
government system is to the users; and how friendly and usable the system is in
terms of accessing government information (Wang & Liao, 2008). In an e-government
setup, system quality symbolises the perceptions of citizens on the technical
performance of the information system, in terms of information retrieval and delivery.
Information quality is defined as the quality of e-government output, and it is
associated with the routine requirements of the user (Petter, Delone, & Mclean,
2013). Increased information quality has an impact on the level of openness and
transparency citizens have with the government (Grimsley, Meehan & Tan, 2007).
The desirable factors that are associated with quality in an e-government system are:
continuous access to government information and services, provision of accurate,
relevant and updated information, and the provision of efficient and effective services
to the citizens (Ondego & Moturi, 2016).
Service quality can be described as the overall support the users of an e-government
system receive from the service provider (DeLone & McLean, 2003). This construct
measures the general perspective of e-government systems, from the perspective of
how ready the staff is to provide the required service, and examines the accessibility
Figure 1: DeLone and McLean, (2003:24)
306
of the system, as well as the safety of transactions undertaken. The availability of the
system to users, provision of individualised attention by the IT personnel and the
specific needs of users are evaluated under this construct (Stefanovic et al., 2016).
The focus of this construct is to measure how well the e-government services are
delivered, and whether or not they match the expectations of the users.
The intention to use/use is the degree to which customers and staff use the
capabilities of an e-government system. Petter, Delone, & Mclean, (2008) assert that
the use and intention to use construct can be used alternatively, depending on
whether the context of usage is voluntary or mandatory. For the e-government
system, use is more relevant since citizens’ utilisation of the system is voluntary. The
“use” construct evaluates the attitude to, and the general satisfaction of, users with
the e-government system.
User satisfaction is the means of measuring customer’s opinion of the information
system; it should capture the entire customer experience, in terms of the reports
produced and the support services provided to the users (DeLone & McLean, 2003).
Net benefit can be described as the extent to which e-government information
systems contribute to the success of clients and organisations using the system such
as greater efficiency, improved decision-making, and improved productivity (DeLone
& McLean, 2003).
DeLone and McLean (2003) model as more widely used information
systems success model
The revised DeLone and McLean IS success model has been used for various
aspects of information systems success, and it is considered widely used in the
evaluation of IS success (Khayun, Ractham, & Firpo, 2012; Stefanovic et al., 2016).
E-government is considered to be an aspect of an information system and the D&M
Model can be applied to assess its effectiveness. The key primary purpose with which
the two scholars, DeLone and McLean, came up with the model was to synthesise the
previous information systems research into more coherent knowledge, which can be
used as a guide for future researchers. Although the model has not been empirically
307
tested, it has guided many researchers in assessing the success of information
systems (Khayun et al., 2012) Lowry, Karuga, & Richardson's ( 2007) fifteen-year
scientometric analysis of articles from three premier IS journals between 1990 – 2004
indicate that DeLone and Mclean are among the most highly cited authors in the field
of information systems. They used the citation analysis to demonstrate the impact of
articles on individual authors in the field of information systems.
Hussein, Karim, Mohamed, & Ahlan, (2007:2) opine that DeLone and McLean’s
model was a major breakthrough in the IS field, as the model has become universal
and instrumental in evaluating information systems performance. A number of
studies have used the model to evaluate different aspects of the e-government
system: Wang & Liao (2008) successfully used the model to study on citizens’
perspectives of e-government system. In another study, Teo et al. (2008) examined
the role of trust in e-government success using 214 Singaporeans e-government
website users. In a related study, Connolly, Bannister, & Kearney (2010) evaluated
the quality of Irish revenue online e-government system.
According to Schaupp (2010:47), the D&M model is the most predominant in
information systems literature, and it is a widely cited framework that providing a
comprehensively review the success of IS. It has been cited in over 300 referenced
journals as the basis of measuring different valuables in information systems
research. The author explains that users make use of an information system and then
evaluates it on the basis of either being satisfied, or not, with the outcome.
Floropoulos, Spathis, Halvatzis, & Tsipouridou, ( 2010) used the model to measure
the success of Greek e-government taxation information system, from the employees
perspectives.
In a similar study, Hsu & Chen (2007) assessed the e-government model in Taiwan,
on user behaviour. Chiu et al., (2016:240) explain that evaluating information system’
success is difficult, but based on DeLone and McLean’s strong foundation and
extensive literature review, the model has a concrete definition of success, and the
capability to explain the success factors that influence both individual and
organisational users of information systems.
308
Some recent publications on the satisfaction of e-government services (Rana,
Dwivedi, Williams, & Lal, 2015; Floropoulos et al., 2010; Formunyuy & De Wet, 2018;
Rana, Dwivedi, Williams, & Weerakkody, 2015; Veeramootoo, Nunkoo, & Dwivedi,
2018) have used different approaches of the D&M model to evaluate the success of
information systems. Rana, Williams, Dwivedi, & Williams, (2012:42) assert that
DeLone and McLean’s (1992, 2003) IS success model has been instrumental in
establishing the primary factors that influence the use and acceptance of e-
government services by citizens.
Critiques of D&M model
The main critic of the Delone & Mclean, (1992) model was Seddon, who claimed that
“the inclusion of both variance and process interpretation in the model leads to so
many potentially confusing meanings” (Seddon, 1997:240). Another critic observed
that the measures of IS effectiveness focus more on the product than the service of
IS. The argument was that IS researchers will not determine IS effectiveness if
service quality is not included in the assessment package (Pitt, Watson, & Kavan,
1995:173).
DeLone and McLean addressed the criticism by proposing an updated model (Delone
& Mclean, 2003), which grouped all the “impact” measures into “net benefit ” and also
added another measure “service quality” as a new dimension. Seddon, (1997) further
claimed that the use construct in D&M model is a not a success measure, but that of
behavior. The argument was that non-users of a system may not indicate that it is not
useful, but that potential users are engaged with other tasks.
Other researchers have mentioned the difficulties of applying the D&M model in the
area of defining and implementing the IS model in some specific research contexts.
Jiang & Klein (1999) claim that users have different preferences of measures,
depending on the type of system being evaluated. Despite the criticisms, the D&M
model makes important contributions to understanding IS success. It provides a
scheme for categorising various IS success measurements that are used in literature,
and suggests a model of casual dependencies between the categories (Seddon,
1997).
309
4. Approach /method
To conduct this review, Webster & Watson's (2002) approach was used. The
approach recommends the following guidelines to be considered when conducting
research on systematic literature review, especially in information systems related
research; (1) to select and use the right keyword for searching the databases (2) to
select the most relevant publications with matching criteria (3) in-depth reading to
identify irrelevant publications. The literature search for this study concentrated on the
period 1992-2018, the choice of 1992 as the baseline is because that was when the
first DeLone and McLean (D&M) model appeared. To retrieve relevant updated
literature for this review, EBSCO Discovery Service and SCOPUS databases were
used. A number of search strings were used to conduct the searches and these
include: D&M IS success model, e-government success evaluation and information
systems success.
A ten-year analysis of D&M Model application was conducted by carrying out a
search on two online databases. The focus of this analysis is to explore the
application of the model in e-government research. The literature selection process is
presented in Figure 2:
310
Step one: The search was carried out using these two reliable and reputable online
journal databases. The databases are updated daily, have diverse field coverage and
contain peer reviewed articles. The two databases allows the use of delimiters to
refine the search .The databases of choice for the literature analysis are
1. Ebsco Host Discovery (https://ebsco .com)
2. Scopus (http://www.scopus.com)
Step two: The study was restricted to articles published from 2009 to 2019
Step three: To extract the articles, the study search included only open access
articles published in English language, that are peer reviewed and full text between
2009 and 2019. The advanced search strategy was performed on the databases
using the Boolean operators, ‘AND’ and’ OR’ to filter the search. To be included for
review, both DeLeon & McLean model or IS success AND e-government must feature
in the title, keywords by author or in the abstract. Some of the articles were irrelevant
Figure 2: Flow chart of the literature selection process
(Sampson et al. 2009:948)
311
and were further filtered. A summary of the inclusion and exclusion criteria is
presented below in Table 1.
Step four: The remaining full text articles were then analysed to determine whether
they met the stipulated criteria. Articles that met these criteria were the ones included
for review in the study. Data extraction
Table 2 below presents the result of data extracted from the two databases.
For the eligibility, all extracted data was screened as presented in Table 3.
Table 1 Inclusion and exclusion criteria
• Inclusion criteria:
• Open access articles about
DeLone & McLean model
• Articles about
• information systems model
• Articles written in English
• E-government articles that
applied D&M IS success model
in the study
• Articles that can answer the
research question
• Articles published from 2009 -
2019
• Exclusion criteria:
• Duplicate articles
• Articles that are not written in
English
• Articles that are not full text
• Articles that are not dealing
with e-government
• Articles that did not apply D&M
IS success model in the study
• Unpublished articles
Table 2 Number of studies in the two selected sources
Source Direct search results Selected for the study
Ebsco Host Discovery 86 23
Scopus 69 11
312
5. Findings and discussions
The analysis findings are presented according to the three research questions the
study sought to answer.
5.1 What is the extent of use of DeLone and Mclean IS success
model in e-government evaluation- research?
The study findings as indicated in Figure 2 show that of 100 articles eligible for the
study, 66 were excluded because they were not related to e-government evaluation
.This analysis show that the model has not been applied in e-government like in other
areas dealing with IS. The analysis also show the highest number of the articles (16)
that discuss various aspects of D&M model application in e-government originated
from Asia; followed by Europe, then North America. The least are from Africa.
This is not surprising, because according to the 2018 e-government survey (United
Nations, 2018) Asia has a noticeable improvement in e-government development, 37
out of 47 countries in Asia scored above the world average e-Government
Development Index (EGDI). In the same report, countries in the African regional’s
average EGDI scores were significantly lower than the world average. Africa is
making progress, but can learn from the countries with high EGDI. The highest
number of the papers reviewed, dealt with the subject of e-government preformation
and user satisfaction, which are key indicators of an e-government system success
as presented in Table 6.
Table 3 Screening of extracted data
Screening label Number of
studies
A Directly eligible articles 34
B Model discussed but not applied in the study 66
C Mentioned in passing 18
D Not relevant 27
E Duplicate 10
TOTAL 155
313
5.2 Are the six dimensions of DeLone and McLean IS model being
used in e-government research?
The results of the analysis reveal that the six dimensions of the model were not
applied in the studies reviewed as presented by DeLone and McLean. Studies which
used all the dimensions of the model had to make some modification to suit the IS
being evaluated. Information quality, service quality and user satisfaction constructs
were highly used in the reviewed articles.
The results of this review show that most e-government evaluation studies in Africa
use some of the dimensions of D&M model and add others that complement the IS
being assessed. Both the Ebsco Discovery Service and Scopus review results
showed that no study had adopted the D&M model for e-government performance
evaluation in Kenya.
5.3 What are the challenges of using D&M Model in evaluating e-
government information system?
The evaluation of e-government initiatives like all other information systems has
proven to be important and complex, both in theory and practice. The complexity
arises because of the many perspectives involved; the difficulties in identifying and
quantifying benefits; not being familiar with evaluation techniques; and the difficulties
Table 6 Geographical & subject distribution of articles of study
Subject Africa Asia Europe North
America
Totals
E- Filing 1 1
E-government systems Adoption 2 2
E-government performance /
satisfaction
2 6 4 1 13
Transparency in government 2 1 3
Online public grievance redressal
system
2 2
Internet in public administration 2 2 4
Website evaluation 1 1 2 1 5
Information storage & retrieval
system
2 2 4
Totals for each region 3 16 11 4 34
314
in the interpretation of results from the data collected (Grimsley & Meehan, 2007;
Alshawi & Alalwany, 2009 ; Shan et al. 2011)
In the e-government context, determining stakeholders is complex because of the
diversity (government officials, elected representatives, public and private
organisations). This requires the researcher to be clear on who the stakeholders for
the study are. The context of measuring net benefit needs to be clear for each study.
Confusion arises as to what constitutes net benefit, whether it should be looked at
from the individual perspective, the government or that of business community or
citizens. Different stakeholders may have varying opinions on what constitutes net
benefit to them.
Table 4 presents the key findings for each article extracted from Ebsco Host
Discovery.
315
F
)+
6&!''7!'7(
4$5;8!#2$
.-
/
& P
4$ A$
' &
-
(/C 2>!
$#
!
$
>!;
4!
$#!$
>!=
!
>!'-
-6
#! >
'
( 3,
4!
D!-
3!
(/C 3 $ !
;
A $
!
#
*
$1!
-
@
E
%
(/7 +
!G
E
! !
!
!
! !#
$-
5
-
&
(/7 B$#
6 !
;!
4!
$
!'A!#
6-
!
F ,' (/7
&
)
&#
!!
'
6-
316
!!
#
6!
0
3!
-3
(/< &
;
%
)=
!'=#
>$!
#$
!-
< &)
&
!-
+
(/< &!
!
!
)=;
#
#
%
#)=
!'
?#
##!
-
7 +
'
(/0 &'8)#
!G
!!
A8)=
# !
$>!
66-
!-$
#'
C -
D
(/0 & !
6;!!
&#>!
!!-
6
#'A!#
!
6 !'
/
%
(/0 &!
!;
2> !
='
4-
> !
317
B
8-
2
(/0 & !
%;
$#
!
)=
!
'-
#)
'
( &!!
-
(/0 !
!
A!
'-
-
6$
@
?-
?
(/0 4 !
;
4>!
4
>!$$!
$4&-
5
B
#
+-
(/F 2>!
$!
;
>
4>
$!
)=
!6
-
F A-P!
(/F
!
6;
!
A$!
!
6 !
!-
6'
0 B
+-
(/F B!
!
!
6
(<
!
!6!
-
4&
< B
#
(/F !!
!;
4!
318
:LOOLDPV
:HHUDNNRG\
YDOLGDWLRQRIDQLQWHJUDWHG,6
VXFFHVVPRGHO
V\VWHPIURP,QGLDQFLWL]HQV¶
SHUVSHFWLYHVXVLQJ'0,6VXFFHVV
PRGHOWRDVFHUWDLQWKHVXFFHVV
'ZLYHGL
:DVWHOO
/DXPHU
+HQULNVHQ
5HVHDUFKRQLQIRUPDWLRQ
V\VWHPVIDLOXUHDQG
VXFFHVVHVVWDWXVXSGDWHDQG
IDLOXUH
7KHIRFXVRIWKHVWXG\ZDVWREULQJRXW
WKHQHHGIRUQHZSHUVSHFWLYHDQGWR
SURYLGHLQVLJKWVRQWKHIDFWRUVWKDW
HQDEOH,6VXFFHVVDVZHOODVWKHIDFWRUV
WRFRQVLGHULQRUGHUWRDYRLG,6IDLOXUH
9HQNDWHVK
+RHKOH
$OMDIDUL
$XVDELOLW\HYDOXDWLRQRI
2EDPDFDUHZHEVLWH
,QYHVWLJDWHVWKHXVDELOLW\DQG
VDWLVIDFWLRQRI86FLWL]HQVZLWKWKH
2EDPDFDUHZHEVLWH7KHDSSURDFK
XVHGKDVLWVWKHRUHWLFDOEDVLVRQ'0
PRGHOHVSHFLDOO\RQWKHTXDOLW\RI
LQIRUPDWLRQUHFHLYHGE\FLWL]HQV
-XNLF9LQWDU
%HQFLQD
([DQWHHYDOXDWLRQWRZDUGVDQ
DVVHVVPHQWPRGHORILWV
LPSDFWRQWKHVXFFHVVRIH
JRYHUQPHQWSURMHFWV
7KHUHVHDUFKWULHVWRILOOWKHJDS
EHWZHHQWKHWKHRUHWLFDODQGHPSLULFDO
DSSURDFKRQLVVXHVRIH[DQWHH
JRYHUQPHQWHYDOXDWLRQXVLQJVRPHRI
'0VXFFHVVFRQVWUXFWVVHUYLFH
TXDOLW\XVHXVHUVDWLVIDFWLRQ5HVXOWV
SURYHWKDWWKHH[DQWHHYDOXDWLRQRIH
JRYHUQPHQWSURMHFWVKDVLPSDFWRQ
SURMHFWVXFFHVV
$ODZQHK$O
UHIDL
%DWLKD
0HDVXULQJXVHUVDWLVIDFWLRQ
IURPHJRYHUQPHQWVHUYLFHV
OHVVRQVIURP-RUGDQ
$UHYLHZWRLGHQWLI\NH\IDFWRUVZKLFK
GHWHUPLQHWKHVDWLVIDFWLRQRIFLWL]HQV
ZLWK-RUGDQ¶VHJRYHUQPHQWVHUYLFH
SRUWDOV,QIRUPDWLRQDQGVHUYLFHTXDOLW\
'0PRGHOFRQVWUXFWVZHUHFRQVLGHUHG
IRUWKHUHYLHZ
5DQDHWDO 7KHRULHVDQGWKHRUHWLFDO
PRGHOVIRUH[DPLQLQJWKH
DGRSWLRQRIHJRYHUQPHQW
VHUYLFHV
7KHVWXG\V\VWHPDWLFDOO\H[SORUHV
H[LVWLQJERG\RIOLWHUDWXUHLQH
JRYHUQPHQW7KHXVHRI'0PRGHO
DQDO\VLVLQGLFDWHGWKDWV\VWHPTXDOLW\
ZDVQRWXVHGWRPHDVXUHLQWHQWLRQRI
XVH
319
.DLVDUD
3DWKHU
7KHHJRYHUQPHQWHYDOXDWLRQ
FKDOOHQJHD6RXWK$IULFDQ
%DWKR3HOHDOLJQHGVHUYLFH
TXDOLW\DSSURDFK
7KHSDSHUIRFXVHVRQHYDOXDWLQJH
JRYHUQPHQWVHUYLFHTXDOLW\LQ6RXWK
$IULFDHJRYHUQPHQWZHEVLWHV
,QIRUPDWLRQTXDOLW\D'0GLPHQVLRQ
KDVEHHQXVHGWRHYDOXDWHWKHTXDOLW\RI
LQIRUPDWLRQEHLQJDFFHVVHGE\WKH
FLWL]HQV
&RPSHQGLXPRIUHYLHZHG'00RGHODSSOLFDWLRQLQHJRYHUQPHQWDV
UHIOHFWHGE\6FRSXV
7DEOH.H\ILQGLQJVIURPWKHUHYLHZHG6FRSXVDUWLFOHV
6
1
$XWKRU <HDU 7LWOHRI3XEOLFDWLRQ )RFXVRI3XEOLFDWLRQ
7HJHWKRII
6DQWD
*UDQREOHV
+R\RV
'RHVWUXVWKDYHLPSDFWRQ
V\VWHPDQGRSHUDWLRQDO
HIIHFWLYHQHVV"7KH
LPSOHPHQWDWLRQRIHJRYHUQPHQW
LQ&RORPELD
7KHIRFXVLVRQLQYHVWLJDWLQJWUXVWLQH
JRYHUQPHQWV\VWHPDIWHU
LPSOHPHQWDWLRQ7KUHH'0PRGHO
FRQVWUXFWLQIRUPDWLRQV\VWHPDQG
VHUYLFHTXDOLW\ZHUHXVHGLQWKH
HYDOXDWLRQ)URPWKHILQGLQJVV\VWHP
DQGVHUYLFHTXDOLW\KDGQRLPSDFWRQ
RSHUDWLRQDOHIIHFWLYHQHVV
6DQWD
0DFGRQDOG
)HUUHU
7KHUROHRIWUXVWLQHJRYHUQPHQW
HIIHFWLYHQHVVRSHUDWLRQDO
HIIHFWLYHQHVVDQGXVHU
VDWLVIDFWLRQOHVVRQIURP6DXGL
$UDELDLQHJRYHUQPHQWWR
EXVLQHVV
([SORUHVWKHHIIHFWRIWUXVWRQRQOLQHH
JRYHUQPHQWV\VWHPVDWLVIDFWLRQE\WKH
XVHUV,QIRUPDWLRQV\VWHPVDQG
VHUYLFHTXDOLW\RIWKH'0,6VXFFHVV
PRGHOPHDVXUHVZHUHFRQVLGHUHGDV
NH\GULYHUVRIXVHUVDWLVIDFWLRQ
,GRXJKL
$EGHOKDNLP
'HYHORSLQJFRXQWULHVH
JRYHUQPHQWVHUYLFHVHYDOXDWLRQ
LGHQWLI\LQJDQGWHVWLQJ
DQWHFHGHQWVRIVDWLVIDFWLRQFDVH
RI$OJHULD
7KHLPSDFWRIV\VWHPVLQIRUPDWLRQ
DQGVHUYLFHTXDOLW\'0PRGHO
FRQVWUXFWVZHUHFRQVLGHUHGNH\IDFWRUV
LQHVWDEOLVKLQJXVHUV¶VDWLVIDFWLRQZLWK
WKHHJRYHUQPHQWV\VWHPLQ$OJHULD
320
!-
&$
&!
$!=#
!&!'
5
+)-8) (/7 !;
6)
!'
!
$#
'-;
6#
6#
!'
F
8#
D,-
3#
(/7
&
8#
# !'
4!
!
#$!
$'-
#
#=
$#$#
'
0
&'
(/7
2
;
=
>
'-
6#
<
+
,
-:
(/0
2>!6
!
!=
!;
2
'
&!
6
#
!'
6#$
7
B
#-
+
(/@
2!
!;
A
=
!$-
=CC<.
321
Conclusion
It is evident that none of the articles had used all the six dimensions of D&M
ISsuccess model. In some studies, the D&M constructs were replaced, or added, with
other dimensions from other models that are closely related to the IS under
evaluation. The findings showed Asia leading in the application of D&M Model in e-
government research, while Africa showed the least utilisation of the model. The
analysis review shows the highest number of papers dealt with the subject of e-
government performance and user satisfaction.
DeLone & Mclean (1992) suggest that the success of information systems evaluation
requires the performance of all the six D&M model dimensions. From this study, it
appeared that most of the e-government evaluation research works have not adhered
to this suggestion; as most of them used few of the D&M dimensions and added their
!
%$B'
A-#
'
C
3-
!
(/(
3) !
#
&)
!
!
-
6
/
B
'
(/(
4#
!
!;!
!
4
# !'
*+!
1#
'
-'
,$!
!I-
A
(//C
!>
!
!
'
4
$# !
!
! !:
322
own, closely related dimensions to their studies. The review show that two of D&M
model constructs highly used in the e-government evaluations were information
quality (15 articles ;44%) and Service quality (13 articles; 38%).
From the review, 100 articles (64.5%) used D&M model for IS evaluation in various
fields, while only 34% of the articles were in e-government, and made use of the
dimensions of D&M model for IS evaluation. This corroborated the submission that
D&M is a widely used model in evaluating IS success (Stefanovic et al., 2016) though
in e-government the utilisation is still very low. The study analysis shows that
information systems, service quality and user satisfaction construct of D & M model
were highly utilised in e-government research and not the complete set of the six
constructs. This implies that there is partial application of the six dimensions of D&M
model in e-government IS success research. According to Delone & Mclean, (1992)
information systems success evaluation is multidimensional and requires the
performance of all the six construct of D&M model. The results of such an evaluation
would provide the policy makers with comprehensive review that would enhance e-
government service delivery to the citizens.
This study limits the search to only two databases; it is suggested that future studies
consider a more comprehensive review by using more databases. Also, there is a
need for more empirical studies on the application of D&M IS success model in e-
government research in Africa.
Acknowledgement
I wish to acknowledge the support I received from my PHD supervisors, Prof. D.N.
Ocholla and Dr. Neil Evans, towards this publication as well as useful remarks I
received from two anonymous reviewers.
References
Agrifoglio, R., Metallo, C., & Lepore, L. (2016). Success Factors for Using Case
Management System in Italian Courts. Information Systems Management,
33(1), 42–54.
Akram, M. S., Malik, A., Shareef, A. M., & Goraya, Shakir, A. m. (2019). Exploring the
interrelationships between technological predictors and behavioral mediators
323
in online tax filing: The moderating role of perceived risk. Government
Information Quarterly, 36, 237–251. Retrieved from https://doi.org/10.1016/
j.giq.2018.12.007
Alawneh, A., Al-refai, H., & Batiha, K. (2013). Measuring user satisfaction from e-
Government services: Lessons from Jordan. Government Information
Quarterly Journal, 30, 277–288.
Alcaide-muñoz, L., & Bolívar, M. P. . (2015). Understanding e-government library
science field of knowledge. Internet Research, 25(4), 633–673. https://doi.org/
10.1108/IntR-12-2013-0259
Ali, M. A., Hoque, R., & Alam, K. (2018). An empirical investigation of the relationship
between e-government development and the digital economy: the case of
Asian countries. Journal of Knowledge Management.Retrieved from https://
doi.org/10.1108/JKM-10-2017-0477
Alshawi, S., & Alalwany, H. (2009). E-government evaluation: Citizen ’ s perspective
in developing countries. InformationTechnology for Development, 15(3), 193–
208.
Alzahrani, L., Al-karaghouli, W., & Weerakkody, V. (2017). Analysing the critical
factors in fl uencing trust in e-government adoption from citizens ’ perspective:
A systematic review and a conceptual framework. Internatoinal Business
Review, 26, 164–175.
Aminah, S., Ditari, Y., Kumaralalita, L., Hidayanto, A. N., Phusavat, K., &
Anussornnitisarn, P. (2018). E-procurement system success factors and their
impact on transparency perceptions: perspectives from the supplier side.
Electronic Government, an International Journal, 14(7), 177–199.
Belanger, F., & Carter, L. (2012). Digitizing Government Interactions with
Constituents: An Historical Review of E-Government Research in Information
Systems. Journal of the Association for Information Systems, 13, 363–394.
Chen, Y. C., & Zhang, J. C. (2012). Citizen-centric e-government performance:
satisfaction with e-information. Electronic Government, an International
Journal, 9(4), 388–402.
Chen, Y., Hu, L., Tseng, K., Juang, W., & Chang, C. (2019). Cross-boundary e-
government systems: Determinants of performance. Government Information
Quarterly, 36, 449–459.
Chiu, P. S., Chao, I. C., Kao, C. C., Pu, Y. H., & Huang, Y. M. (2016). Implementation
and evaluation of mobile e-books in a cloud bookcase using the information
system success model. Library Hi Tech, 34(2), 207–223.
Connolly, R., Bannister, F., & Kearney, A. (2010). Government website service
quality: A study of the Irish revenue online service. European Journal of
Information Systems, 19(6), 649–667.
Delone, W. H., & Mclean, E. R. (1992). Information Systems Success: The Quest for
the Dependent Variable. Information Systems Research, 3(1), 60–95.
Delone, W. H., & Mclean, E. R. (2003). The DeLone and McLean Model of
Information Systems Success: A Ten-Year Update. Journal of Management
Information Systems, 19(4), 9–30.
324
Delone, W. H., & Mclean, E. R. (2016). Information Systems Success Measurement.
Foundations and Trends in Information Systems, Vol. 2, pp. 1–32.
Deng, H., Karunasena, K., & Xu, W. (2018). Evaluating the performance of e-
government in developing countries. Internet Research, 28(1), 169–190.
Retrieved from http://10.0.4.84/IntR-10-2016-0296%0Ahttp://
search.ebscohost.com/
login.aspx?direct=true&db=llf&AN=127846179&site=ehost-live
Dwivedi, Y. K., Wastell, D., Laumer, S., & Henriksen, H. Z. (2015). Research on
information systems failures and successes: Status update and future
directions. Inf Syst Front, 17, 143–157.
Fan, J., & Yang, W. (2015). Study on E-Government Services Quality: The Integration
of Online and Offline Services. 8(3), 693–718.
Floropoulos, J., Spathis, C., Halvatzis, D., & Tsipouridou, M. (2010). Measuring the
success of the Greek Taxation Information System. International Journal of
Information Management, 30, 47–56.
Gable, G. G., Sedera, D., & Chan, T. (2008). Re-conceptualizing Information System
Success: The IS-Impact Measurement Model. Journal of the Association for
Information Systems, 9(7), 377–408.
Grimsley, M., & Meehan, A. (2007). E-Government information systems: Evaluation-
led design for public value and client trust. European Journal of Information
Systems, 16(2), 134–148.
Grimsley, M., Meehan, A., & Tan, A. (2007). Evaluative design of e-government
projects A community development perspective. Transforming Government
People Process and Policy, 1(2), 174–193.
Hasan, M., Maarop, N., Naswir, R. Y., Samy, G. N., Magalingam, P., Yaacob, S., &
Daud, S. . (2018). A proposed Conceptual Success Model of Citizen- Centric
Digital Government in Malaysia. Journal of Fundamental and Applied
Sciences, 10(2), 35–46.
Helbig, N., Gil-garcía, J. R., & Ferro, E. (2009). Understanding the complexity of
electronic government: Implications from the digital divide literature.
Government Information Quarterly, 26, 89–97.
Hsu, F., & Chen, T. (2007). Understanding Information Systems Usage Behavior in E-
Government: The Role of Context and Perceived Value. /PACIS 2007/
Proceeding41, 41. Association for Information Sytems.
Huang, Z., & Benyoucef, M. (2014). Usability and credibility of e-government
websites. Government Information Quarterly, 31(4), 584–595.
Hussein, R., Karim, N. S. A., Mohamed, N., & Ahlan, A. R. (2007). The Influence Of
Organisational F actors On Information Systems Success In E-government
Agencies in Malaysia. The Electronic Journal on Information Sysystems in
Developing Countries, 29(1), 1–17.
Idoughi, D., & Abdelhakim, D. (2018). Developing Countries E-Government Services
Evaluation Identifying and Testing Antecedents of Satisfaction Case of Algeria.
International Journal of Electronic Government Research, 14(63).
325
Jiang, J. J., & Klein, G. (1999). Supervisor Support and Career Anchor Impact on the
Career Satisfaction of the Entry-Level Information Systems Professional.
Journalof Managemeent Information Sytems, 16(3).
Jukic, T., Vintar, M., & Bencina, J. (2013). Ex-ante evaluation: Towards an
assessment model of its impact on the success of e-government projects.
Information Policy, 18, 343–361.
Kaisara, G., & Pather, S. (2011). The e-Government evaluation challenge: A South
African Batho Pele-aligned service quality approach. Government Information
Quarterly, 28(2), 211–221.
Khayun, V., Ractham, P. & Firpo, D. (2012). Assessing E-Excise Sucess with Delone
and McLean ’ s Model. Journal of Computer Information Systems, 52(3), 31–
40.
Lowry, P. B., Karuga, G. G., & Richardson, V. J. (2007). Assessing Leading
Institutions , Faculty , and Articles in Premier Information Systems Research
Journals. Premier Information Systems Research Journals," Communications
of the Association for Information Systems, 20(16).
Luna-Reyes, L. F., Gil-Garcia, J. R., & Romero, G. (2012). Towards a
multidimensional model for evaluating electronic government: Proposing a
more comprehensive and integrative perspective. Government Information
Quarterly, 29(3), 324–334.
Mellouli, M., Bentahar, O., & Bidan, M. (2016). Trust and e government acceptance:
The case of Tunisian on line tax filing. The Electronic Journal Information
Systems Evaluation Volume, 19(3), 197–213.
Mirchandani, D. A., Kathawala, Y. A., Jr, J. H. J., Hayes, J. P., & Chawla, S. (2018). A
comparison of perspectives of Kuwaiti and Indonesian residents towards e-
government. Electronic Government, an International Journal, 14(2), 134–159.
Nam, T. (2014). Determining the type of e-government use. Government Information
Quarterly, 31, 211–220.
Ondego, B., & Moturi, C. (2016). Evaluation of the Implementation of the e-Citizen in
Kenya. International Journal of Applied Information Systems, 10(4), 13–21.
Petter, S., Delone, W., & Mclean, E. (2008). Measuring information systems success:
models , dimensions , measures , and interrelationships. European Journal of
Information Systems, 17(3), 236–263.
Petter, S., Delone, W., & Mclean, E. R. (2013). Information Systems Success: The
Quest for the Independent Variables. Journal of Management InformationS
Ystems, 29(4), 7–62.
Pitt, L. F., Watson, R. T., & Kavan, C. B. (1995). Service quality: A measure of
information systems effectiveness. MIS Quarterly: Management Information
Systems, 19(2), 173–185.
Rana, N., Dwivedi, Y. K., Williams, M. D., & Lal, B. (2015). Examining the Success of
the Online Public Grievance Redressal Systems: An Extension of the IS
Success Model. Information Systems Management, 32(1), 39–59
Rana, N. P., Dwivedi, Y. K., & Williams, M. D. (2013). Evaluating the validity of IS
success models for the electronic government research: An empirical test and
326
integrated model. International Journal of Electronic Government Research
(IJEGR), 9(3), 1–22.
Rana, Nripendra P, Dwivedi, Y. K., Williams, M. D., & Weerakkody, V. (2015).
Investigating success of an e-government initiative: Validation of an integrated
IS success model. Springer Science, 17, 127–142.
Rana, Nripendra P, Williams, M. D., Dwivedi, Y. K., & Williams, J. (2012). Theories
andTheoretical Models For Examining the E-government Service. E-Service
Journal, 8(26–57).
Rehman, M., Kamal, M. M., & Esichaikul, V. (2016). Adoption of e-Government
Services in Pakistan: A Comparative Study Between Online and Offline Users.
Information Systems Management, 33(3), 248–267.
Ritchi, H., Wahyudi, I., & Susanto, A. (2015). Research Program on Key Factors of e-
government and Their Impact on Accounting Information Quality. Procedia -
Social and Behavioral Sciences, 211, 673–680.
Sampson, M., Mcgowan, J., Cogo, E., Grimshaw, J., Moher, D., & Lefebvre, C.
(2009). An evidence-based practice guideline for the peer review of electronic
search strategies. Journal of Clinical Epidemiology, 62, 944–952.
Santa, R., Macdonald, J. B., & Ferrer, M. (2019). The role of trust in e -Government
effectiveness , operational effectiveness and user satisfaction: Lessons from
Saudi Arabia in e-G2B. Government Information Quarterly, 36, 39–50.
Schaupp, L. C. (2010). Web Site Success: Antecedents of Web Site Satisfaction and
Re-use. Journal of Internet Commerce, 9(1), 42–64.
Scott, M., Delone, W., & Golden, W. (2016). Measuring e-government success: A
public value approach. European Journal of Information Systems, 25(3), 187–
208.
Scott, M., H., D. W., & Golden, W. (2009). Understanding Net Benefits: a Citizen -
based Perspective on Egovernment Sucess. Thirtieth International Conference
on Information Systems, 1–12.
Seddon, P. B. (1997). A Respecification and extension of the DeLone and McLean
Model of IS Success. Information Systems Research, Vol. 8, pp. 240–253.
Shan, S., Wang, L., Wang, J., Hao, Y., & Hua, F. (2011). Research on e-Government
evaluation model based on the principal component analysis.
InformationTechnology Management, 12, 173–185.
Sharma, R., & Jayasimha, K. R. (2016). Assessment of e-government service quality:
An emerging market perspective. The Marketing Review, 16(1), 63–78.
Stefanovic, D., Marjanovic, U., Delic, M., Culibrk, D., & Lalic, B. (2016). Assessing the
success of e-government systems: An employee perspective. Information and
Management, 53, 717–726.
Suh, H., Chung, S., & Choi, J. (2017). An empirical analysis of a maturity model to
assess information system success: a firm-level perspective. Behaviour &
Information Technology, 36(8), 792–808.
Tegethoff, T. M., Santa, R., Granobles, D. F. M., & Hoyos, J. C. V. (2019). Does trust
have an impact on system and operational effectiveness? The implementation
of e-government in Colombia. Electronic. Electronic Government, an
International Journal, 15(3), 241–260.
327
Teo, T. S. H., Srivastava, S. C., & Jiang, L. (2008). Trust and Electronic Government
Success: An Empirical Study. Journal of Management Information Systems,
25(3), 99–132.
United Nations. (2018). E-Government Survey 2018: gearing E-Govrnment to
Support Transformation Towards Sustanable and Resilient Societies. New
York.
Veeramootoo, N., Nunkoo, R., & Dwivedi, Y. K. (2018). What determines success of
an e-government service? Validation of an integrative model of e-filing
continuance usage. Government Information Quarterly, 35(2), 161–174.
Venkatesh, V., Hoehle, H., & Aljafari, R. (2014). A usability evaluation of the
Obamacare website. Government Information Quarterly, 31, 669–680.
Verkijika, S. F., & De Wet, L. (2018). A usability assessment of e-government
websites in Sub-Saharan Africa. International Journal of Information
Management, 39, 20–29.
Wang, Y.-S., & Liao, Y.-W. (2008). Assessing eGovernment systems success: A
validation of the DeLone and McLean model of information systems success.
Government Information Quarterly, 25(4), 717–733.
Wang, Yi-shun. (2008). Assessing e-commerce systemsuccess: a respecification and
validation of the Delone McLean model of IS sucess. Info System Journal, 18,
529–557.
Wang, Yi-shun, & Liao, Y. (2008). Assessing eGovernment systems success: A
validation of the DeLone and McLean model of information systems success.
Governemt Information Quartely Information Quartely, 25, 717–733.
Wang, Yu-yin, Wang, Y., Lin, H., & Tsai, T. (2019). Developing and validating a model
for assessing paid mobile learning app success. Interative Learning
Environment, 27(4), 458–477.
Webster, J., & Watson, R. T. (2002). Analyzing the Past to Prepare for the Future:
Writing a Literature Review. MIS Quarterly, 26(2).
Weerakkody, V., Irani, Z., Lee, H., Hindi, N., & Osman, I. (2016). Are U.K. Citizens
Satisfied With E-Government Services? Identifying and Testing Antecedents of
Satisfaction. Information Systems Management, 33(4), 331–343.
Widiyanto, N., Sandhyaduhiti, P. I., Hidayanto, A. N., & Munajat, Q. (2016). Exploring
information quality dimensions of government agency ’ s information services
through social media: a case of the Ministry of Education and Culture in
Indonesia. Electronic Government, An International Journa, 12(3), 256–278.
Wirtz, B. W., & Kurtz, O. T. (2018). Local e-government services: quality aspects and
citizen usage preferences. Electronic Government, an International Journal,
14(2), 160–176.
Zaidi, F., Muneerah, N., & Aminah, S. (2018). Relationship between Service Quality
and e- Government Acceptance: The Role of Gender as a Moderator. Global
Business and Management Research, 10(3), 859–866.
Zheng, Q., & Liang, C. Y. (2017). The path of new information technology affecting
educational equality in the new digital divide-based on information system
success model. Eurasia Journal of Mathematics, Science and Technology
Education, 13(7), 3587–3597.