Content uploaded by Felix Brünker
Author content
All content in this area was uploaded by Felix Brünker on Dec 03, 2020
Content may be subject to copyright.
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
1
Design Requirements for AI-based Services Enriching
Legacy Information Systems in Enterprises: A Managerial
Perspective
Completed research paper
Nicholas R. J. Frick
Professional Communication in Electronic Media/Social Media
University of Duisburg-Essen
Duisburg, Germany
Email: nicholas.frick@uni-due.de
Felix Brünker
Professional Communication in Electronic Media/Social Media
University of Duisburg-Essen
Duisburg, Germany
Email: felix.bruenker@uni-due.de
Björn Ross
School of Informatics
University of Edinburgh
Edinburgh, United Kingdom
Email: b.ross@ed.ac.uk
Stefan Stieglitz
Professional Communication in Electronic Media/Social Media
University of Duisburg-Essen
Duisburg, Germany
Email: stefan.stieglitz@uni-due.de
Abstract
Information systems (IS) have been introduced in enterprises for decades to generate business value.
Historically systems that are deeply integrated into business processes and not replaced remain vital
assets, and thus become legacy IS (LISs). To secure the future success, enterprises invest in innovative
technologies such as artificial intelligence-based services (AIBSs), enriching LISs and assisting
employees in the execution of work-related tasks. This study develops design requirements from a
managerial perspective by following a mixed-method approach. First, we conducted ten interviews to
formulate requirements to design AIBSs. Second, we evaluated their business value using an online
survey (N = 101). The results indicate that executives consider design requirements as relevant that
create strategic advancements in the short term. With the help of our findings, researchers can better
understand where further in-depth studies are needed to refine the requirements. Practitioners can
learn how AIBSs generate business value when enriching LISs.
Keywords Artificial intelligence, AI-based services, legacy information systems, design requirements,
enterprises
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
2
1 Introduction
Information systems (IS) have been used for decades to generate business value by gaining advantages
over competitors in almost every part of organisational environments. Systems are used by individuals
to process and produce data (Aram and Neumann 2015), to speed up business processes (Neumann et
al. 2014), regulate the informational, material, and human resources as well as enhance efficiency,
effectiveness and productivity (Xu and Topi 2017). Large parts of systems have been instituted over
years in enterprises and thus can be described as legacy information systems (LISs). They are
considered to be the “backbone of an organisation’s information flow and the main vehicle for
consolidating business information” (Bisbal et al. 1999). Ensuring the ongoing operation is therefore
mandatory for organisations as LISs are strongly linked to the strategic business goals (Robertson
1997). However, enterprises still need to be able to exploit novel and innovative trends to secure their
future success. The continuous development of technology paired with the lack of time to replace LISs
(Hasselbring 2000) requires the adaptation of existing applications. A currently popular group of
technologies that enhance LISs in organisations is artificial intelligence (AI) (Frick et al. 2019a).
The term AI is used to describe a wide range of technologies with self-learning abilities which are
possibly able to achieve superior performance compared to humans (Coombs et al. 2020). AI can have
strong economic potential and generate strategic business advancements as it can take over repetitive
tasks and relieves employees from unwanted duties (Siau and Wang 2018). When AI is used to enrich
LISs in enterprises, we use the term AI-based services (AIBSs), which are “components enriching IS in
organisations with the main objective of collaborating with employees and assisting in the execution
of work-related tasks” (Frick et al. 2019a). AIBSs are applied in enterprises to support employees in
the decision-making process (Brachten et al. 2020), accelerate internal support processes (Frick et al.
2019b) or facilitate strategic decisions on an organisational level (Aversa et al. 2018).
Despite the fact that AIBSs are increasingly being used in businesses (Dwivedi et al. 2019), there is an
urgent demand to formulate requirements that should be considered when designing systems
enhancing LISs. Most existing AIBSs adapted to business processes are considered to be narrowed
down to a specified task (Batin et al. 2017), where the majority focuses on the short-term creation of
added value while less attention is paid to design aspects. Research here needs to generate theoretical
guidance to “create ideal AI systems for human decision makers” (Duan et al. 2019) in contrast with
current literature that mainly targets technological aspects (Mikalef et al. 2018). Addressing the
pressing need to do more research in this area, this study aims at proposing suitable recommendations
and is thus guided by the following research question:
RQ1: What are the requirements that need to be considered to design AI-based services enriching
legacy information systems in enterprises?
RQ2: To what extent do the identified design requirements for AI-based services enriching legacy
information systems contribute to business value in enterprises?
This study makes a first foray into the examination of AIBSs from a business perspective following a
mixed-method approach. We conducted semi-structured expert interviews with ten executives from
multiple enterprises to derive design requirements. Preliminary results from these interviews were
previously reported in a research-in-progress paper (Frick et al. 2019a). In this article, we additionally
report on our quantitative evaluation of the findings using an online survey with N = 101 managers to
verify which requirements create business value when enriching LISs.
Researchers and practitioners find the requirements helpful to consider important aspects before the
actual introduction of AIBSs. From a theoretical perspective, this research gives an overview of design
requirements when deploying AIBSs in enterprises and outlines an orientation for further in-depth
research. From a practical point of view, practitioners can understand how AIBSs generate business
value when enriching LISs. Hence, this article extends the IS literature by broadening our knowledge
on how to design, implement and deploy AIBSs for enriching LISs. We believe this study is valuable to
researchers and practitioners equally for understanding and overcoming difficulties when dealing with
the introduction of AIBSs in enterprises.
2 Theoretical Background
Implementing IS in organisations aims to enhance business performance. IS ensure the effectiveness
and efficiency of the organisation (Hevner et al. 2004) as well as supporting collaboration by fulfilling
the role as a communication and coordination system (Aram and Neumann 2015). Within an
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
3
organisation, IS as support systems can be characterised by three central functions. First, supporting
the company’s business operations. Second, supporting managerial decision making. Third,
supporting the achievement of strategic competitive advantages (Susanto and Meiryani 2019).
Therefore, IS improving the organisation’s business performance by ensuring these three functions
can be considered as business IS (Aram and Neumann 2015) and is described as “a collection of
various information that has unity between one and the other aimed at business interests” (Susanto
and Meiryani 2019). IS in organisations consist of various information technologies (Orlikowski and
Iacono 2001). These information technologies fulfil functions such as transmitting, processing, or
storing information (Piccoli 2008). By doing this, IS in organisations help to process large amounts of
information and to solve upcoming decision-making problems (Leavitt and Whisler 1958). Due to the
generation of numerous benefits, organisations have been using IS to generate business value for
decades. However, systems not coping with modern requirements or are not modifiable for business
purposes (Robertson 1997) slowly turn into LISs but remain vital assets for organisations (Bianchi et
al. 2003). The major problem with LISs is that they are deeply integrated into the running of a
business (Robertson 1997) and that there is simply no rational reason for replacing them (Hasselbring
2000), thus organisations remain dependent (Robertson 1997). Nevertheless, enterprises are regularly
required to invest in innovative technologies to generate or maintain advantages over competitors.
Thereby, applications need to be adaptable to retain LISs as reasonably as possible (Bianchi et al.
2003) to enrich existing solutions and to assist employees in their daily work. Related to the dynamic
development of new technologies, organisations need to consider ongoing improvements of LISs to
ensure nascent business requirements, emphasizing the impact of information technologies on
business operations (Bjerknes et al. 1991).
A concept which becomes increasingly relevant for the aligning organisational strategies is AI. There is
no uniform definition, but AI can be considered as “the ability of a machine to perform cognitive
functions that we associate with human minds, such as perceiving, reasoning, learning, interacting
with the environment, problem solving, decision-making, and even demonstrating creativity” (Rai et
al. 2019). AI is believed to fundamentally change the future of business across industries, generating
advantages over competitors and maximizing the market share (Benbya and Leidner 2018; Wang and
Siau 2019). The potential benefits cannot be overlooked causing organisations to invest heavily
(Schuetzler et al. 2018). When AI is applied as component enriching existing IS, it can be considered as
AIBS (Frick et al. 2019a). They are typically implemented using machine learning algorithms (Kersting
2018) and are turning into a key element for enterprises (Dwivedi et al. 2019). In a recent study (Frick
et al. 2019b) we demonstrated that AIBSs can be integrated into existing internal support workflows.
The authors indicated that the categorisation and distribution of incoming customer inquiries are
heavily accelerated. Another example (Pessach et al. 2020) evaluated the application of an AIBS to
support human resource employees with the recruitment and placement of professionals. The results
showed that insights might have been overlooked by internal recruiters who were using conventional
methods. The examples illustrate that AIBSs promise great potential for organisations, including those
which still rely on a multitude of LISs. Although AIBSs are becoming more ubiquitous within LISs,
there are no properly validated requirements that need to be considered to design services enriching
existing IS in enterprises.
3 Research Design
In order to examine which requirements need to be considered to design AIBSs enriching legacy IS in
enterprises and likewise generate business value, we selected a mixed-method approach. This design
strategy equips researchers with an effective technique in dealing with evolving situations and complex
improvements while being able to achieve contributions for theory and practice (Venkatesh et al.
2013). Mixed-methods are capable of simultaneously addressing confirmatory and exploratory issues,
provide greater insights compared to single methods and help to analyse divergent and/or
complementary findings (Teddlie and Tashakkori 2003, 2009). The approach at hand is an
exploratory sequential procedure combining qualitative and quantitative research to validate whether
assumptions based on a small sample size can be generalized for a larger population (Creswell and
Creswell 2018). We conduct qualitative research to identify core issues and obtain knowledge within a
less explored domain (Kelle 2006), and use the subsequent quantitative phase to validate our findings
with a larger population (Creswell and Creswell 2018).
3.1 Expert Interviews
Expert interviews established themselves and grown in popularity as an efficient and concentrated
method to collect relevant data (Bogner et al. 2009). We chose this method to 1) give the interviewees
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
4
enough space to elaborate on issues, 2) provide selective assistance by the researchers and, 3) ensure
capturing all relevant aspects to generate comparable responses to simplify the subsequent coding
process. The term expert describes an individual with advanced knowledge in the investigated field of
research (Meuser and Nagel 2009). In this study (see also Frick et al. 2019a), the experts are
employees working at management level. Furthermore, experts needed to be familiar with AI and have
knowledge of where AIBSs can be applied to improve business performance and further they needed to
have a minimum of three years of experience to be well acquainted with the company and the sector.
We also defined that the companies in which the experts worked should have applied AIBSs to enrich
LISs and be planning future adoptions. In terms of sample size, we follow Creswell and Creswell
(2018) who recommend using between three and ten individuals. Based on these factors, a large
German retail holding organisation was selected which owns equity interests in further companies.
Here, we chose companies focusing on various areas within the holding organisation: agricultural
trade (C1), animal husbandry advisory (C2), energy product consulting (C3), animal feed advisory
(C4), construction services (C5), wholesale e-commerce (C6) and agricultural machinery distribution
(C7). We acquired two project managers (E1/C2 [male, 28 years old, tenure of 8 years], E2/C1 [f, 25,
8]), three managing directors (E3/C3 [m, 35, 10], E7/C6 [m, 40, 21], E8/C1 [m, 43, 19]), three heads of
divisions (E4/C4 [f, 40, 9], E9/C3 [f, 30, 4], E10/C5 [m, 43, 18]), and finally two managers (E5/C7 [m,
57, 5], E6/C7 [m, 47, 15]). The interviews were conducted in person at the workplace of the
interviewees. Participants were 39 years old on average, with three female and seven male experts and
a mean tenure at the company of 11.7 years.
Conducting semi-structured expert interviews implies creating “questioning guided by identified
themes in a consistent and systematic manner” (Qu and Dumay 2011). Therefore, a guideline with
central questions on AI, AIBSs, LISs and business value was developed in advance, divided into the
following 9 parts: 1) Introduction of the interviewer and brief summary of the purpose of the research,
as the participants had already received relevant information when they were recruited. 2) Self-
introduction of the interviewee, including career development, current responsibilities in the company
as well as demographic data. 3) Definition of AIBSs and prior experience, followed by the authors’
explanation of AIBSs to ensure the same level of knowledge among all participants. 4) Areas in which
AIBSs are applied in organisations and which (L)IS are enriched. 5) Adoption and acceptance of AIBSs
and which barriers might arise when enriching (L)IS with AIBSs. 6) Advantages, disadvantages and
dangers when using AIBSs in (L)IS. 7) How AIBSs need to be developed to use them daily in (L)IS. 8)
Responsibility for an implementation and what an introduction looks like. 9) Conclusion of the
interview: Possibility for the interviewee to ask further inquiries followed by a debriefing.
Interpreting what respondents mean in their answers to questions assumes that researchers have
extensive knowledge in the subject matter (Campbell et al. 2013). Following this requirement, the
authors have a strong background on IS, LIS, AI and AIBSs as well as its utilization in enterprises. We
used content analysis as the most precise method to analyse qualitatively collected material (Mayring
2014). The research data was coded according to certain, empirically and theoretically reasonable
points, enabling a structured description of the material (Mayring 2014). Codes represent words or
short phrases for attributes of language-based or visual data (Saldaña 2009) aiming at reducing the
intricacy of vocabulary and identifying core categories. The coding was collaboratively done by two
researchers to distribute the effort of the coding process and to get different perspectives on the
qualitative data. A list of general codes was created coding two interviews in front and collected inside
a codebook. One of the researchers maintained the codebook as editor and was responsible for
updating, revising and maintaining the list of codes during the research process (Guest and MacQueen
2008). Respecting the codes-to-theory model (Saldaña 2009), the analytic process is not linear but
rather cyclical. It is divided into two cycles: an initial coding of the data, followed by pattern coding for
the categorization of coded data. The first cycle was used to structure the data and assigning codes. In
the second cycle, categories were created. In summary, we created 379 codes with 10 categories using
MAXQDA (version 18). We finally validated the intercoder reliability using Krippendorff’s alpha,
resulting in a value of .823 which is above the threshold of .800 (Hayes and Krippendorff 2007).
3.2 Online survey
To validate the design requirements, we conducted an online survey. As a precondition for
participation in our study, participants had to speak English or German fluently, as the survey was
designed in both languages. In addition, individuals had to work in a company within the management
level to ensure an understanding of the business perspective. Various German organisations were
approached directly by the researchers, plus, participants were recruited via Prolific, a platform
designed to acquire subjects for surveys (Palan and Schitter 2018). The study started with a
standardised briefing about anonymization and research purposes, followed by a detailed explanation
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
5
about AIBSs, why they are already introduced in organisations and what they are capable of. To verify
each design requirement, we adopted and modified constructs from previously validated instruments
to ensure the accuracy of the measurements. However, since the results provide new insights, we were
not able to identify items for every design requirement, resulting in the development of own
constructs. We used a combination of already existing as well as self-developed items which were
validated as part of the evaluation. Besides items for the design requirements, we further measured the
business value when using AIBSs to enrich LISs in enterprises. All items were measured on a 10-point
numeric scale and questions starting with the phrase “how relevant are the following
aspects/statements regarding AI-based services”. Example items are “the utilization of AI-based
services is a good idea” or “the strategy regarding the utilization of AI-based services is congruent with
the business strategy of organisations”. Participants had to answer 74 questions in total, excluding
information about their demographic data. To ensure that the attendees were aware of the definition of
AIBSs throughout the survey, they had to answer 2 questions with “yes” or “no” about the main
intention as the last step: 1) “The main objective of AIBSs is to collaborate with employees and assist
them in their daily tasks” and 2) “The purpose of this survey is to evaluate requirements to develop
AIBSs”. The survey was designed using LimeSurvey and took about 15 minutes to complete, the
analysis was conducted using jamovi (version 1.1.9.0).
4 Results
4.1 Requirements
The Strategic Orientation of an enterprise controls the actual use of AIBSs within an organisation.
However, AIBSs must not be implemented in a sweeping way but rather specifically to enhance
distinct functions. This may reduce costs and the need for resources of an organisation, which is a key
aspect of common strategic orientations (Cao 2002; Johnson 2018). One expert explained “The
management and executive board have to support that. We have to achieve additional benefits for
ourselves as well as for our customers”
1
(E8). Furthermore, an activity within an organisation has a
strategic value when it contributes to the organisational success (Barney 1991). Likewise, the
deployment of services such as AIBSs needs to align along the strategic orientation of the organisation
in order to fulfil the overarching organisational strategy and vice versa (Henderson and Venkatraman
1999). In this context, such services have to provide concrete advantages with respect to employees
and customers of the organisation (Luse et al. 2013).
Process Organisation describes the actual process of coherent and individual operations. This step
aims to support the existing IT processes in order to improve their velocity by reducing non automated
work steps. One respondent emphasised “This is a whole process, fast, effective and customer-
friendly. The system thinks and acts in a processual way” (E2). Organisations supporting new
technology investments such as new business models and new business processes will get superior
returns comparing to other competitors that do not invest (Susanto and Meiryani 2019). Therefore, the
process organisation is integrally tied to the Strategic Organisation. Enriching LISs with AIBSs enables
the organisation to digitize the individual processes and functions (Luse et al. 2013).
Before the actual and continual interaction with the system, the Acceptance and Adoption of AIBSs
by users must be achieved. Experts point out that new technology in general has an acceptance
problem in organisations. However, to interact with it at all, perceived usefulness and perceived ease
are the major aspects that must be taken into account. One participant pointed out that “AIBSs should
not dictate how to act in specific situations, that would only create unnecessary barriers” (E2). “User
acceptance and confidence are crucial for the development of any new technology” (Taherdoost
2017). With the growing interest in AIBSs, organisations need to investigate the challenges referring to
adoption (Alsheibani et al. 2018). Research has developed various models explaining individual
technology adoption (Venkatesh et al. 2003) which have been continuously revised especially in an
organisational context. However, adoption is not only an important issue for technology in general but
also for AIBSs.
Authenticity, Trust and Transparency describes that the interaction with an AIBSs should
preferably be perceived as authentic. In addition, trust in AIBSs and their transparency has to be as
high as possible. One respondent stated “[Understanding the outcome is] very important! On the one
hand, users can understand how the system came up with the decision, on the other hand, the users’
level of knowledge is adjusted” (E1). The decision-making process has to be as transparent as possible,
1
Excerpts from the German interviews have been translated into English for the reader’s convenience.
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
6
given that transparency is identified as crucial for trust building (Wünderlich et al. 2013) and leading
to an increase of authenticity as well as use intention. “The perception of authenticity is critical for a
user’s evaluation of the service as valuable and satisfactory and for establishing trust” (Wünderlich
and Paluch 2017). In addition, the interaction quality leads to trust and to usage intention (Nasirian et
al. 2017).
A frequently mentioned topic in the interviews was the experts’ concerns about Security, Privacy
and Ethics during the interaction process. In the opinion of the experts, it is most important to clarify
which (personal) data is processed, where, how and by whom and that no ethically reprehensible
decisions are made. One interviewee said “so, there will be a lot of scepticism, because everyone is
afraid that personal data will be published. So, everyone has quite a bit of respect” (E7). Another
underlined that “[The artificial intelligence system] may ask for things that may not be relevant in
the context. Also, sensitive topics, there is a lot of sensitive [personal] data which the user does not
want to reveal” (E9). Informational self-determination refers to “the right or ability of individuals to
exercise personal control over the collection, use and disclosure of their personal data by others”
(Cavoukian 2008). The interviewees indicated that adequate communication had to take place before
the introduction of AIBSs and that the legal basis had to be clarified in advance. AIBSs are trained by
developers and thus might contain considerable human bias (Rothenberger et al. 2019). Ethical
concerns are a major challenge (Duan et al. 2019), thus AIBSs must be implemented with caution
(Wang and Siau 2018).
An often mention requirement for AIBSs can be summarised as Task Support and Service
Features. Experts point out that interaction with AIBSs has the main goal to support employees in
their daily work in order to fulfil tasks more quickly and thus save time and therefore money,
summarised as increased effectiveness and efficiency. AIBSs can particularly adopt repetitive tasks for
which no cognitive abilities are needed. One expert specified AIBSs as “An expedient” and further “A
way that it makes my work easier and I can take care of what I enjoy” (E3). Service Features can be
seen as a comparison between what the employee feels should be offered and what is provided, in
other words, the discrepancy between perceptions and expectations (Pitt et al. 1995). In this case,
users should have the overall opinion that AIBSs increase the effectiveness and efficiency at work.
A major aspect when designing AIBSs are System Characteristics. This requirement essentially
describes the technical characteristics of the system. Experts point out that AIBSs have to be user-
friendly, reliable and extensible as well as provide a quick response. Combined, the ease of use and
learnability of AIBSs must be guaranteed. One expert depicted “It has to be simple and practicable
and it has to deliver additional benefits right from the start” and more “It has to be easy to use” (E6).
Technology needs to be understandable and usable, delightful and enjoyable, with the goal to actually
fulfil human needs (Norman 2013). In addition to paying attention on engineering, manufacturing and
ergonomics, aesthetics of form and the quality of interaction must be taken into account (Norman
2004). The system acceptability must always be guaranteed in order to be utilised by users.
The requirement of Implementation and Deployment describes how systems are developed and
how they are introduced within the organisation and to employees. The development of systems
should not only be done by IT experts but integrate users who provide functional know-how. An
introduction needs to involve affected employees to minimize the resistance against AIBSs.
Furthermore, introduced systems should be reviewed regularly to ensure their functionality. One
respondent explains that “I need someone who is able to exploit the possibilities together with me in
order to reach maximal benefits” (E5). Barki and Hartwick (Barki and Hartwick 1989) describe
participation as “the behaviours and activities that users or their representatives perform in the
system development process” with the overall responsibility as a key dimension (Hartwick and Barki
1994). In the development of IS, it is considered an important factor for achieving system success and
is commonly mentioned in research (Mann and Watson 1984).
Another significant point that has been mentioned are concerns about the Connectivity and
Collaboration of AIBSs. On the one hand, enriching LIS with AIBSs means systems being
interconnected and complementing each other and employees. On the other hand, through the
collaboration with AIBSs, communication between departments and locations as well as between
employees will be promoted. One attendee said that “As an advantage I see the fact, that AIBSs can
generally promote collaboration between centralised and decentralised units” (E2). Systems can be
designed to serve as knowledgeable collaborators of employees, helping to accomplish goals while
ensuring to remain in control (Xu and Topi 2017), thus the collaboration is becoming a partner
relationship (Oberquelle 1984; Oberquelle et al. 1983). AIBSs greatly enhance collaboration of people
and resources in organisations (Tang and Sivaramakrishnan 2003). By using AIBSs, employees are
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
7
able to process all available and relevant data to mitigate unintentional bias in human decisions (Elson
et al. 2018). In addition, systems can be joined into groups working together.
Another requirement for AIBSs is that users should learn through the interaction with such systems.
Through the interaction with AIBSs users can, for example, prepare better for upcoming appointments
and pay attention to matters they have previously disregarded. Therefore, Enhanced User
Performance and Service Training means enhancing people's (cognitive) performances by
learning from AIBSs as well as challenging employees’ cognitive abilities. One interviewee clarified
that “It is also increasingly important that I as a user quickly have a value. I think this factor should
not be underestimated” (E7). Learning from AIBSs and thereby enhancing the performance of
employees should have a positive effect on the organisation: the technology should be an instrument to
enhance people's performance. Interaction with AIBSs helps to boost performance at work (Siddike et
al. 2018) and helps to overcome human limitations and enhance human abilities (Rouse et al. 2009).
4.2 Validation
In total, 150 participants took part in the study, 124 of whom completed it. After excluding data sets
with a short completion time (below 5 minutes) and answers with significant similarities as well as
analysing the two validation questions as the last step of the survey, we resulted in N = 101
participants. In terms of gender, 48 (47,5%) were female, 53 (52,5%) were male with a minimum age
of 22 and a maximum of 64 (M = 37, SD = 9.79). Regarding the level of education, 12 (11,9%) have an
apprenticeship, 8 (7,9%) a secondary school degree, 15 (14,7%) a high school degree, 60 (59,5%) a
university degree and final 6 participants have a PhD (6%). Participants live in a variety of different
countries, with the majority in the United Kingdom (35/34,7%) and Germany (26/25,8%), working in
full-time employment (75/74,3%), part-time employment (18/17,8%), self-employed (6/5,9%), mainly
in companies of a size between 1 and 49 employees (21/20,8%), 5.000 and 9.999 employees
(22/21,7%) and 1.000 and 4.999 employees (18/17,82%). In terms of industries, most participants
work in IT (23/22,8%), consumer goods (12/11,9%) and energy (12/11,9%). 34 companies (33,7%) are
using AI and 67 (66,3%) are not. For validating which design requirements are relevant for enriching
LISs, we assessed the correlation coefficient of the constructs using the Pearson correlation (Zhou et
al. 2016). We found significant correlations between design requirements and business value (cf. Table
1): 1) Acceptance and Adoption (r = .324, p < .001), 2) Task Support and Service Features (r = .353, p <
.001), 3) System Characteristics (r = .329, p < .001) and finally 4) Enhanced User Performance and
Training (r = .477, p < .001).
Requirement
Pearson's r
p
Strategic Orientation
.079
.434
Process Organisation
.146
.145
Acceptance and Adoption
.324***
< .001
Authenticity, Trust and Transparency
.172
.086
Security, Privacy and Ethics
.085
.397
Task Support and Service Features
.353***
< .001
System Characteristics
.329***
< .001
Implementation and Deployment
.156
.120
Connectivity and Collaboration
.154
.125
Enhanced User Performance and Training
.477***
< .001
Table 1. Pearson's r between requirements and business value (*** significant at < 0.001 (2-tailed))
5 Discussion and Implications
IS used in organisations to improve business performance (Neumann et al. 2014), regulate resources
and enhance efficiency, effectiveness and productivity (Xu and Topi 2017). However, the steady
progression in improving technologies and the need of securing future success of organisations lead to
new requirements especially for LISs, systems that have been used for decades but are not easy to
replace (Hasselbring 2000) and remain vital assets for organisations (Bianchi et al. 2003). Thereby,
the integration of AIBSs enriching LISs provides great potential for organisations. The goal of this
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
8
study was to identify design requirements from a business perspective that need to be considered to
design AIBSs. Overall, managers assess 4 of 10 requirements as vital to generate business value (cf.
Figure 1).
Figure 1. Significant requirements generating business value for AIBSs enriching LIS in enterprises
Acceptance and Adoption is considered significant. This goes in line with earlier research identifying
acceptance as crucial for any technological advancement (Taherdoost 2017). We interpret this to the
mean that managers have a great interest in employees accepting AIBSs in order to really use them in
the workplace. Furthermore, Task Support and Service Features assists employees in their daily work
to perform tasks more efficiently, thus saving time and money. Our results show that managers
consider this to be relevant for the future success of enterprises. We understand that the introduction
of AIBSs enriching LISs must always create benefits for the applying organisation. The same might be
valid for Enhanced User Performance and Training. Previous research explains that AIBSs can boost
performance at work (Siddike et al. 2018) and enhance human capabilities (Rouse et al. 2009). We
argue that manager might perceive AIBSs as a suitable method to educate employees and train them
faster for a certain task. By improving the skills of employees, the organisational revenue might be
increased. System Characteristics, as the last significant requirement, explains that the ease of use and
learnability of AIBSs must be guaranteed. This was confirmed by previous research (Norman 2013).
We thus state that managers sense quick values for enterprises when AIBSs are easier to use.
Surprisingly, according to the survey of managers, many of the requirements were not significantly
related to business value. In case of Strategic Orientation and Process Organisation, we appreciate that
LISs are probably not necessarily aligned with the overall business strategy but rather have an end of
life. Therefore, AIBSs enhancing existing systems temporarily are less aligned with the strategy or new
processes and are thus intended to create benefits in the short term. We interpret the missing
correlation of Authenticity, Trust and Transparency that managers may not care whether employees
understand the results of AIBSs. Even though the decision-making process needs to be transparent for
trust building (Wünderlich et al. 2013) it is not necessarily essential if employees are forced to use a
system. This might also apply for Security, Privacy and Ethics and further, doubtful decisions are not
in the spotlight of the industry but profit maximisation. Regarding Implementation and Deployment,
managers might simply not be interested in how services are developed but focus on the overall
strategic outcome. Finally, Connectivity and Collaboration might miss any correlation as managers do
not exactly know what AIBSs are capable of. This this is also shown by our study as only 34 companies
(33,7%) are using AI, thus the application in enterprises is not yet very widespread.
This research is not free of limitations. First, we derive design requirements from a limited group of
ten experts. Although a number of experts was involved, they can each only cover their own
perspective. In addition, interviews were conducted in just one holding organisation, although the
experts represent very different departments. We describe results from on online survey which is
based on a sample size of N = 101. Although the spread of AIBSs in organisations is increasing
(Dwivedi et al. 2019), it was still difficult to acquire more participants from the management level who
were already familiar with AI. Last, our design requirements do not cover personal user characteristics
and innovativeness as major moderating effects on the intention to use such systems (Rzepka and
Berger 2018).
The contribution of this paper is interesting for researchers and practitioners equally to design,
implement and deploy AIBSs in enterprises to enrich LISs. From a theoretical point of view, this paper
gives an overview of requirements to design AIBSs and provide insights for areas where future in-
depth research is needed. IS researchers can better understand AIBSs’ targeted characteristics which
are fruitful to positively influence business value in enterprises. From a practical point of view,
organisations appreciate the business value which can be generated by using our requirements.
Enterprises following the recommendations are more likely to generate advantages over competitors.
Practitioners further understand which design requirements are relevant for existing IS. Future
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
9
research should dive deeper into the design requirements as many of the requirements are rather
broad. Therefore, studies need to conduct in-depth research working out the individual conditions for
each requirement. In addition, IS scholars might also be interested in using the requirements to design
an AIBS and evaluate a prototype in a real-world scenario possibly refining the design requirements.
6 Conclusion
Enterprises have been using IS for decades, however, many of those slowly became LISs. Nevertheless,
organisations still need to adapt to ongoing technological advancements such as AIBSs. In this study,
we have presented and discussed requirements for AIBSs from a managerial perspective. It became
clear that executives consider design requirements as relevant that create business value in the short
term. However, we argue that our requirements are still valid for AIBSs enriching IS in general.
Admittedly, the picture painted by our research is far from clear as the requirements might have been
formulated to broad. Researcher and practitioners need to watch future developments closely to
understand how enterprises create and maintain business value using AIBSs.
7 References
Alsheibani, S., Cheung, Y., and Messom, C. 2018. “Artificial Intelligence Adoption: AI-Readiness at
Firm-Level,” in Twenty-Second Pacific Asia Conference on Information Systems.
Aram, M., and Neumann, G. 2015. “Multilayered Analysis of Co-Development of Business Information
Systems,” Journal of Internet Services and Applications (6:1), Journal of Internet Services and
Applications. (https://doi.org/10.1186/s13174-015-0030-8).
Aversa, P., Cabantous, L., and Haefliger, S. 2018. “When Decision Support Systems Fail: Insights for
Strategic Information Systems from Formula 1,” Journal of Strategic Information Systems
(27:3), pp. 221–236. (https://doi.org/10.1016/j.jsis.2018.03.002).
Barki, H., and Hartwick, J. 1989. “Rethinking the Concept of User Involvement,” MIS Quarterly (13:1),
pp. 53–63. (https://doi.org/10.2307/248700).
Barney, J. 1991. “Firm Resources and Sustained Competitive Advantage,” Journal of Management
(17:1), pp. 99–120.
Batin, M., Turchin, A., Markov, S., and Zhila, A. 2017. “Artificial Intelligence in Life Extension: From
Deep Learning to Superintelligence.,” Informatica (41:401–417).
Benbya, H., and Leidner, D. E. 2018. “How Allianz UK Used an Idea Management Platform to Harness
Employee Innovation,” MIS Quarterly Executive (17:2), pp. 139–155.
Bianchi, A., Caivano, D., Marengo, V., and Visaggio, G. 2003. “Iterative Reengineering of Legacy
Systems,” IEEE Transactions on Software Engineering (29:3), pp. 225–241.
(https://doi.org/10.1109/TSE.2003.1183932).
Bisbal, J., Lawless, D., Wu, B., and Grimson, J. 1999. “Legacy Information Systems: Issues and
Directions,” IEEE Software (16:5), pp. 103–111. (https://doi.org/10.1109/52.795108).
Bjerknes, G., Bratteteig, T., and Espeseth, T. 1991. “Evolution of Finished Computer Systems,”
Scandinavian Journal of Information Systems (3:1), pp. 25–45.
Bogner, A., Littig, B., and Menz, W. 2009. “Introduction: Expert Interviews - An Introduction to a New
Methodological Debate,” Interviewing Experts, pp. 1–13.
(https://doi.org/10.1057/9780230244276_1).
Brachten, F., Brünker, F., Frick, N. R. J., Ross, B., and Stieglitz, S. 2020. “On the Ability of Virtual
Agents to Decrease Cognitive Load: An Experimental Study,” Information Systems and E-
Business Management. (https://doi.org/10.1007/s10257-020-00471-7).
Campbell, J. L., Quincy, C., Osserman, J., and Pedersen, O. K. 2013. “Coding In-Depth Semistructured
Interviews,” Sociological Methods & Research (42:3), pp. 294–320.
(https://doi.org/10.1177/0049124113500475).
Cao, Q. 2002. “The Relationship of Strategy, Fit , and Business Performance in an E- Commerce
Setting: An Empirical Study,” in Eighth Americas Conference on Information Systems.
Cavoukian, A. 2008. “Privacy in the Clouds,” Identity in the Information Society (1:1), pp. 89–108.
(https://doi.org/10.1007/s12394-008-0005-z).
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
10
Coombs, C., Hislop, D., Taneva, S. K., and Barnard, S. 2020. “The Strategic Impacts of Intelligent
Automation for Knowledge and Service Work: An Interdisciplinary Review,” The Journal of
Strategic Information Systems (July 2017), Elsevier, p. 101600.
(https://doi.org/10.1016/j.jsis.2020.101600).
Creswell, J. W., and Creswell, D. J. 2018. Research Design: Qualitative, Quantitative, and Mixed
Methods, SAGE Publications.
Duan, Y., Edwards, J. S., and Dwivedi, Y. K. 2019. “Artificial Intelligence for Decision Making in the
Era of Big Data – Evolution, Challenges and Research Agenda,” International Journal of
Information Management (48:January), Elsevier, pp. 63–71.
(https://doi.org/10.1016/j.ijinfomgt.2019.01.021).
Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., Duan, Y., Dwivedi, R.,
Edwards, J., Eirug, A., Galanos, V., Ilavarasan, P. V., Janssen, M., Jones, P., Kar, A. K., Kizgin,
H., Kronemann, B., Lal, B., Lucini, B., Medaglia, R., Le Meunier-FitzHugh, K., Le Meunier-
FitzHugh, L. C., Misra, S., Mogaji, E., Sharma, S. K., Singh, J. B., Raghavan, V., Raman, R., Rana,
N. P., Samothrakis, S., Spencer, J., Tamilmani, K., Tubadji, A., Walton, P., and Williams, M. D.
2019. “Artificial Intelligence (AI): Multidisciplinary Perspectives on Emerging Challenges,
Opportunities, and Agenda for Research, Practice and Policy,” International Journal of
Information Management (July), Elsevier. (https://doi.org/10.1016/j.ijinfomgt.2019.08.002).
Elson, J. S., Derrick, D. C., and Ligon, G. S. 2018. “Examining Trust and Reliance in Collaborations
between Humans and Automated Agents,” in 51st Hawaii International Conference on System
Sciences, pp. 430–438.
Frick, N. R. J., Brünker, F., Ross, B., and Stieglitz, S. 2019a. “Towards Successful Collaboration:
Design Guidelines for AI-Based Services Enriching Information Systems in Organisations,”
Australasian Conference on Information Systems, pp. 355–361.
Frick, N. R. J., Brünker, F., Ross, B., and Stieglitz, S. 2019b. “The Utilization of Artificial Intelligence
for Improving Incident Management,” HMD (56:2), pp. 357–369.
(https://doi.org/10.1365/s40702-019-00505-w).
Guest, G., and MacQueen, K. M. 2008. Handbook for Team-Based Qualitative Research, AltaMira.
Hartwick, J., and Barki, H. 1994. “Explaining the Role of User Participation in Information System
Use,” Management Science (40:4), pp. 440–465.
(https://doi.org/https://www.jstor.org/stable/2632752).
Hasselbring, W. 2000. “Information System Integration,” Communications of the ACM (43:6), pp.
33–38.
Hayes, A. F., and Krippendorff, K. 2007. “Answering the Call for a Standard Reliability Measure for
Coding Data,” Communication Methods and Measures (1:1), pp. 77–89.
(https://doi.org/10.1080/19312450709336664).
Henderson, J., and Venkatraman, N. 1999. “Strategic Alignment: Leveraging Information Technology
for Transforming Organizations,” IBM Systems Journal (32:1), pp. 4–16.
Hevner, A. R., March, S. T., Park, J., and Ram, S. 2004. “Design Science in Information Systems
Research,” MIS Quarterly (28:1), pp. 75–105. (https://doi.org/10.2307/25148625).
Johnson, A. M. 2018. “The Strategic Value of Participating in Information Security Research :
Evidence from the Finance , Healthcare , and Insurance Industries,” The Journal of the Southern
Association for Information Systems (5:1).
Kelle, U. 2006. “Combining Qualitative and Quantitative Methods in Research Practice: Purposes and
Advantages,” Qualitative Research in Psychology (3:4), pp. 293–311.
(https://doi.org/10.1177/1478088706070839).
Kersting, K. 2018. “Machine Learning and Artificial Intelligence: Two Fellow Travelers on the Quest
for Intelligent Behavior in Machines,” Frontiers in Big Data (1), pp. 1–4.
(https://doi.org/10.3389/fdata.2018.00006).
Leavitt, H. J., and Whisler, T. L. 1958. “Management in the 1980s,” Harvard Business Review (36:6),
pp. 41–48.
Luse, A., Townsend, A., Mennecke, B., and Demarie, S. 2013. “Strategic Information Systems Security:
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
11
Definition and Theoretical Model,” in Nineteenth Americas Conference on Information Systems.
Mann, R. I., and Watson, H. J. 1984. “A Contingency Model for User Involvement in DSS
Development,” MIS Quarterly (8:1), pp. 27–38. (https://doi.org/10.2307/249242).
Mayring, P. 2014. Qualitative Content Analysis: Theoretical Foundation, Basic Procedures and
Software Solution, Klagenfurt: Beltz Verlag.
Meuser, M., and Nagel, U. 2009. Interviewing Experts, London: Palgrave Macmillan.
Mikalef, P., Pappas, I. O., Krogstie, J., and Giannakos, M. 2018. “Big Data Analytics Capabilities: A
Systematic Literature Review and Research Agenda,” Information Systems and E-Business
Management (16:3), Springer Berlin Heidelberg, pp. 547–578. (https://doi.org/10.1007/s10257-
017-0362-y).
Nasirian, F., Ahmadian, M., and Lee, O.-K. 2017. “AI-Based Voice Assistant Systems: Evaluating from
the Interaction and Trust Perspectives,” in Twenty-Third Americas Conference on Information
Systems.
Neumann, G., Sobernig, S., and Aram, M. 2014. “Evolutionary Business Information Systems:
Perspectives and Challenges of an Emerging Class of Information Systems,” Business and
Information Systems Engineering (6:1), pp. 33–38. (https://doi.org/10.1007/s12599-013-0305-
1).
Norman, D. A. 2004. Emotional Design: Why We Love (or Hate) Everyday Things, New York: Basic
Books.
Norman, D. A. 2013. The Design of Everyday Things: Revised and Expanded Edition, New York:
Basic Books.
Oberquelle, H. 1984. “On Models and Modelling in Human-Computer Co-Operation,” in European
Conference on Readings on Cognitive Ergonomics, pp. 26–43.
Oberquelle, H., Kupka, I., and Maass, S. 1983. “A View of Human—Machine Communication and
Cooperation,” International Journal of Man-Machine Studies (19:4), pp. 309–333.
Onwuegbuzie, A. J., and Leech, N. L. 2005. “Taking the ‘Q’ Out of Research: Teaching Research
Methodology Courses Without the Divide Between Quantitative and Qualitative Paradigms,”
Quality & Quantity (39:3), pp. 267–295. (https://doi.org/10.1007/s11135-004-1670-0).
Orlikowski, W. J., and Iacono, C. S. 2001. “Research Commentary: Desperately Seeking the ‘IT’ in IT
Research—A Call to Theorizing the IT Artifact,” Information Systems Research (12:2), pp. 121–
134. (https://doi.org/10.1287/isre.12.2.121.9700).
Palan, S., and Schitter, C. 2018. “Prolific.Ac—A Subject Pool for Online Experiments,” Journal of
Behavioral and Experimental Finance (17), pp. 22–27.
(https://doi.org/10.1016/j.jbef.2017.12.004).
Pessach, D., Singer, G., Avrahami, D., Chalutz Ben-Gal, H., Shmueli, E., and Ben-Gal, I. 2020.
“Employees Recruitment: A Prescriptive Analytics Approach via Machine Learning and
Mathematical Programming,” Decision Support Systems (134:August 2019), Elsevier, p. 113290.
(https://doi.org/10.1016/j.dss.2020.113290).
Piccoli, G. 2008. Information Systems for Managers: Texts & Cases, New York: John Wiley & Sons.
Pitt, L. F., Watson, R. T., and Kavan, C. B. 1995. “Service Quality: A Measure of Information Systems
Effectiveness,” MIS Quarterly (19:2), pp. 173–187. (https://doi.org/10.2307/249687).
Qu, S. Q., and Dumay, J. 2011. “The Qualitative Research Interview,” Qualitative Research in
Accounting & Management (8:3), pp. 238–264. (https://doi.org/10.1108/11766091111162070).
Rai, A., Constantinides, P., and Sarker, S. 2019. “Next-Generation Digital Platforms: Toward Human-
AI Hybrids,” MIS Quarterly (43:1), iii–ix.
Robertson, P. 1997. “Integrating Legacy Systems with Modern Corporate Applications,”
Communications of the ACM (40:5), pp. 39–46. (https://doi.org/10.1145/253769.253785).
Rothenberger, L., Fabian, B., and Arunov, E. 2019. “Relevance of Ethical Guidelines for Artificial
Intelligence - a Survey and Evaluation,” in Twenty-Seventh European Conference on
Information Systems.
Australasian Conference on Information Systems Frick et al.
2020, Wellington AI-based Services: A Managerial Perspective
12
Rouse, W. B., Geddes, N. D., and Curry, R. E. 2009. “An Architecture for Intelligent Interfaces: Outline
of an Approach to Supporting Operators of Complex Systems,” Human-Computer Interaction
(3), pp. 87–122.
Rzepka, C., and Berger, B. 2018. “User Interaction with AI-Enabled Systems: A Systematic Review of
IS Research,” in Thirty Ninth International Conference on Information Systems.
Saldaña, J. 2009. The Coding Manual for Qualitative Research, (1st ed.), London: SAGE Publications.
Schuetzler, R. M., Grimes, G. M., and Giboney, J. S. 2018. “An Investigation of Conversational Agent
Relevance, Presence, and Engagement,” in Twenty-Fourth Americas Conference on Information
Systems.
Siau, K., and Wang, W. 2018. “Building Trust in Artificial Intelligence, Machine Learning, and
Robotics,” Cutter Business Technology Journal (31), pp. 47–53.
Siddike, M. A. K., Spohrer, J., Demirkan, H., and Kohda, Y. 2018. “People’s Interactions with Cognitive
Assistants for Enhanced Performances,” in 51st Hawaii International Conference on System
Sciences.
Susanto, A., and Meiryani. 2019. “Information Systems In Current Business Activities,” International
Journal of Scientific & Technology (8:01).
Taherdoost, H. 2017. “A Review of Technology Acceptance and Adoption Models and Theories,” in
International Conference Interdisciplinarity in Engineering (Vol. 22), pp. 960–967.
(https://doi.org/10.1016/j.promfg.2018.03.137).
Tang, Z., and Sivaramakrishnan, S. 2003. “Creating an Intelligence Infrastructure for Intelligent
Organizations,” in 9th Americas Conference on Information Systems, pp. 2745–2750.
Teddlie, C., and Tashakkori, A. 2009. Foundations of Mixed Methods Research, Thousand Oaks, CA:
SAGE Publications.
Venkatesh, V., Brown, S. A., and Bala, H. 2013. “Bridging the Qualitative-Quantitative Divide:
Guidelines for Conducting Mixed Methods Research in Information Systems,” MIS Quarterly
(37:1), pp. 21–54. (https://doi.org/10.25300/MISQ/2013/37.1.02).
Venkatesh, V., Morris, M. G., Davis, G. B., and Davis, F. D. 2003. “User Acceptance of Information
Technology: Toward a Unified View,” MIS Quarterly (27:3), pp. 425–478.
(https://doi.org/10.1017/CBO9781107415324.004).
Wang, W., and Siau, K. 2018. “Ethical and Moral Issues with AI -- A Case Study on Healthcare
Robots,” in Twenty-Fourth Americas Conference on Information Systems, pp. 1–5.
Wang, W., and Siau, K. 2019. “Artificial Intelligence, Machine Learning, Automation, Robotics, Future
of Work and Future of Humanity: A Review and Research Agenda,” Journal of Database
Management (30:1), pp. 61–79. (https://doi.org/10.4018/JDM.2019010104).
Wünderlich, N. V., and Paluch, S. 2017. “A Nice and Friendly Chat With a Bot: User Perceptions of AI-
Based Service Agents,” in Thirty Eighth International Conference on Information Systems.
Wünderlich, N. V., Wangenheim, F. V., and Bitner, M. J. 2013. “High Tech and High Touch: A
Framework for Understanding User Attitudes and Behaviors Related to Smart Interactive
Services,” Journal of Service Research (16:1), pp. 3–20.
Xu, J. J., and Topi, H. 2017. “A Conceptual Model for User-System Collaboration: Enhancing Usability
of Complex Information Systems,” Communications of the Association for Information Systems
(41:1), pp. 775–800. (https://doi.org/10.17705/1CAIS.04131).
Yan, K., Leidner, D., and Benbya, H. 2018. “Differential Innovativeness Outcomes of User and
Employee Participation in Online User Innovation Communities,” Journal of Management
Information Systems (35:3), pp. 1–34.
Zhou, H., Deng, Z., Xia, Y., and Fu, M. 2016. “A New Sampling Method in Particle Filter Based on
Pearson Correlation Coefficient,” Neurocomputing (216), pp. 208–215.
Copyright © 2020 Nicholas R. J. Frick, Felix Brünker, Björn Ross, Stefan Stieglitz. This is an open-
access article licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand,
which permits non-commercial use, distribution, and reproduction in any medium, provided the
original author and ACIS are credited.
