Conference PaperPDF Available

www.litbaskets.io, an IT Artifact Supporting Exploratory Literature Searches for Information Systems Research

Authors:

Abstract and Figures

Information Systems (IS) researchers currently lack an obvious place to start their literature searches. Existing tools suffer from being either too narrow in their coverage of existing research, leading to an insufficiency effect (low recall); or they are too encompassing, leading to an impracticality effect (low precision). From 11 listings of IS-related journals, we identify a set of 1,042 journals receptive to IS research. We introduce a web interface that allows searching for literature across most of these journals. The search tool enables researchers to narrow or widen the focus of searches, thus allowing researchers to optimise the precision-recall trade-off of their literature searches. We provide an evaluation of our artifact and discuss the relevance of our artifact for exploratory literature searches. Our artifact seeks to facilitate knowledge claims in IS research based on a shared body of knowledge beyond the AIS basket of eight journals.
Content may be subject to copyright.
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
www.litbaskets.io, an IT Artifact Supporting Exploratory
Literature Searches for Information Systems Research
Sebastian Boell
Discipline of Business Information Systems
The University of Sydney Business School
Sydney, Australia
Email: sebastian.boell@sydney.edu.au
Blair Wang
School of Information Systems and Technology Management
UNSW Business School
Sydney, Australia
Email: blair.wang@unsw.edu.au
Abstract
Information Systems (IS) researchers currently lack an obvious place to start their literature searches.
Existing tools suffer from being either too narrow in their coverage of existing research, leading to an
insufficiency effect (low recall); or they are too encompassing, leading to an impracticality effect (low
precision). From 11 listings of IS-related journals, we identify a set of 1,042 journals receptive to IS
research. We introduce a web interface that allows searching for literature across most of these
journals. The search tool enables researchers to narrow or widen the focus of searches, thus allowing
researchers to optimise the precision-recall trade-off of their literature searches. We provide an
evaluation of our artifact and discuss the relevance of our artifact for exploratory literature searches.
Our artifact seeks to facilitate knowledge claims in IS research based on a shared body of knowledge
beyond the AIS basket of eight journals.
Keywords: Literature Search, IS Journals, Literature Reference Database, Bibliometrics, Seniors
Basket of Journals
1
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
1 Introduction
Identifying relevant literature to understand the current state of knowledge is an important endeavour
for any research project in order to successfully argue for the relevance of its contribution (Boell and
Hovorka 2019; Hovorka and Boell 2015). Moreover, stand-alone literature reviews have received
considerable attention in IS in recent years (Boell and Cecez-Kecmanovic 2014; Paré et al. 2015; Tate
et al. 2015; vom Brocke et al. 2015; Webster and Watson 2002). Searches in databases such as the
AIS e-Library (AISeL), Scopus, Web of Science (WoS), Google Scholar – are one important means for
identifying earlier research to be included in literature reviews (Greenhalgh and Peacock, 2005).
Good academic practice involves the documentation of completed literature searches. However, it is
not currently clear where IS scholars should start looking for literature. While there is the AISeL, it
only covers few IS journals, currently not even allowing a search through what is published in the AIS
Senior Scholars’ Basket of Eight journals (Bo8). In contrast, many disciplines have established
literature reference databases that serve as a starting point when looking for existing research. For
instance, Psychology researchers generally begin looking for literature in Psycinfo, indexing around
2,500 journals; Medical researchers would start in Medline, indexing around 5,200 journals; and in
Information Science, LISTA selectively indexes around 600 journals. By comparison, the AISeL covers
15 journals, only 2 of which are included in the Bo8. Our research is thus motivated by the lack of a
simple search tool that provides a starting point for IS researchers when they embark on their journey
of identifying existing research of relevance to IS.
We develop an artifact that supports IS researchers in undertaking exploratory literature searches
(Bates 1989; Marchionini 2006; Savolainen 2017). While the Bo8 serves well as an aspiration for
publication, limiting the literature identification process to only what is published in the Bo8 ignores
many relevant articles published outside of it. This is what we, in this paper, call the insufficiency
effect. Besides a narrow search in the Bo8, IS researchers frequently use large-scale multidisciplinary
literature reference databases for their searches such as Google Scholar, Scopus or WoS. These
databases, however, generally cover tens of thousands of journals, many of which publish research of
very little relevance to IS researchers. Researchers are therefore often overwhelmed by a huge number
of so called ‘false positives’ (Larsen et al. 2019). This makes it difficult to make sense of the huge
volume of retrieved documents in a feasible amount of time, something we call in this paper the
impracticality effect. This paper engages with the need to balance between the insufficiency effect and
the impracticality effect.
Examining journals appearing across eleven listings of IS journals, we develop an extensive set of
1,042 journals whose at least partial relevance to IS research has been previously established. Ranking
these journals by how often they are included in multiple journal listings, we identify a means through
which literature searches can be conducted by narrowing and widening the focus of searches. Setting
up an open source project, we build an IT artifact for the IS community that enables searching though
these journals using the Scopus database. In the spirit of open source we propose that the literature
search artifact available at www.litbaskets.io be refined by the collective effort of the IS community.
2 Literature Reviews and Literature Searches
As the IS discipline has matured, recognition of the need to stock of existing IS research has also
increased (Boell and Cecez-Kecmanovic 2014; Tate et al. 2015; vom Brocke et al. 2015; Webster and
Watson 2002). There are different types of literature reviews that serve different purposes including
theory development, providing an overview over a research area, and detailed analysis of a particular
effect in a meta-analysis (Paré et al. 2015). Such literature reviews use various approaches for
identifying relevant publications to be included in the review, ranging from keyword searches in
databases, citation tracking of relevant papers, analysis of bibliographies, to serendipitous encounters
and suggestions and advice by colleagues (Greenhalgh and Peacock, 2005).
In recent years in particular so called systematic or structured approaches have gained popularity in IS
(Boell and Cecez-Kecmanovic 2015). As part of this trend, we see an increase in literature reviews that
concentrate their efforts in identifying earlier IS research on the Bo8 and leading conferences in the
field: e.g. “we restrict our scope to the eight journals included in the AIS Senior Scholars' Basket of
Journals” (Schuster et al. 2018, p. 6); “we focused on the widely acknowledged AIS Senior Scholars’
Basket of Journals” (Winkler and Wessel 2018, p. 4); “we focused on articles published in the basket of
eight” (Diederich et al. 2019, p. 1553). However, focusing on only few leading outlets for literature
identification is problematic as Larsen et al. (2019) highlight:
2
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
“Like the blind men, we may focus on the voices of manuscripts at an arbitrary set of ‘top
journals’, which often eliminate manuscripts that challenge dominant theory, have unsupported
hypotheses or nonsignificant findings and thus did not clear the hurdles of the small set of
editors and reviewers at top journals.” (Larsen et al. 2019, p. 888)
Accordingly, much relevant existing knowledge may not be published in top tier journals such as the
Bo8. Nonetheless, findings of such research are relevant and should inform the generation of new
knowledge. While much is to be learned from an exceptional piece of research published in a top tier
journal, other journals also make important contributions to the knowledge of the field. One may even
go as far as arguing that an inward gaze and a sole focus on a few leading outlets can, in the long run,
be detrimental to the intellectual development of the discipline as controversial and unusual ideas are
sidelined in the academic discourse (Boell and Hovorka 2019; Larsen et al. 2019). However, such ideas
can drive innovative approaches and new thinking about phenomena (Alvesson and Sandberg 2014).
Besides the problem of literature reviews taking a too narrow focus, researchers also learn more about
the existing literature as they engage in the literature review process (Boell and Cecez-Kecmanovic
2014). Bates (1989), Marchionini (2006), Savolainen (2017) and others have thus emphasised the
exploratory nature of literature searches. As He et al. (2019) put it
“Searchers pick up pieces of information as they navigate through an information space. In this
evolving process, searchers acquire new knowledge and change their perception of search tasks
through their interaction with the information space” (p. 844).
For instance, as researchers engage with the existing literature they will discover different theories,
concepts, methodologies, frameworks, models, or constructs, associated with specialised vocabulary or
phrases that then can be used to further refine the literature identification process (Boell and Cecez-
Kecmanovic 2014). However initially, when researchers delve into the literature discovery process,
they are often not familiar with such specific vocabulary useful for locating highly specialised
literature. At these points, researchers often need to use general vocabulary as search terms in order to
describe the phenomena of interest to them. However, when searching with general vocabulary in a
large database often many irrelevant documents will be retrieved. Our research is thus motivated by
two aspects:
(1) The first motivation of our research is to addressing the problem of literature reviews that
are taking a too narrow focus when considering earlier published research, which we call the
insufficiency effect in the coverage of existing literature.
(2) The second motivation for our research is to address the need of researchers to interactively
manage the amount of search results retrieved in response to a query in order to deal with what
we call the impracticality effect.
3 What is an IS Journal? — Identifying a Set of IS Journals
IS as a discipline has many interests thus making it often unclear where IS research begins and where
it ends. This, however, can make it difficult to identify what journals outside of the Bo8 are publishing
research of relevance to IS. Before building a search artifact we thus first needed to identify a suitable
set of sources through which a literature search should be conducted. We decided to focus on journal
articles as a starting point since they are generally considered the primary research output of
completed IS research projects. We further believe that existing listings of IS journals offer a suitable
starting point for our purpose.
In order to identify journals that are of current interest to IS we decided to use any listing or ranking of
IS journals published in 2008 or later. For those interested in earlier journal rankings see, for
instance, Gillenson and Stefford (2008) or Walstrom and Hardgrave (2001). Overall, we found eleven
such journal lists from which we identified journals (Table 1). Importantly, our focus here is not on the
alleged quality of journals (Lewis et al. 2007; Lowry et al. 2013) or the diversity of editorial boards
(Burgess es al. 2017; Cabanac 2012), but simply on the journals considered.
One well-known journal list is the one published by the Association for Information Systems (AIS
2011). In addition to the Bo8, the AIS listing also includes journals nominated by various special
interest groups (SIGs) of the AIS. For instance, Information and Organization and Decision Support
Systems are nominated by multiple SIGs as one of their top 5 journals. All together the AIS thus
introduces a list of 55 unique journals receptive to IS research (AIS 2011). Beyond the AIS list there are
other reputable lists of IS journals. Some of these listings are used for research assessment purposes,
such as the Academic Journal Guide by the UK-based Chartered Association of Business Schools
3
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
(CABS 2018) and the listing of 177 IS journals by the Australian Business Deans Council (ABDC 2016).
In addition, the London School of Economics (LSE) has published an IS journal list (Willcocks et al.
2008); Holsapple (2009) provides a list of 43 journals based on publication behaviour at US public
universities; Chan et al. (2015) used a citation based analysis for identifying IS journals; and Stewart et
al. (2017) build a list of IS journals based on the WoS journal category for Computer
Science/Information Systems. Furthermore, Scopus also provides three different listings of IS
journals. Finally, John Lamp (2004) maintains a listing of IS journals on his website that is regularly
updated since 1995 and well recognised within the IS community. For the purpose of our research we
only focused on journals listed by Lamp (2004) as active.
Table 1. Overview of IS journal lists.
Listing Description # of journals
(% unique)
(1) ABS –
Journal
Ranking
Ranking journals from 4* (best) to 1. Aims to show “where the best work in
their field tends to be clustered”. Based upon peer review, editorial and
expert judgements and citation statistics.
93
(7%)
(2) ABDC /
ACPHIS
Ranking journals from A* (best) to C. The listing for IS is identical to the
listing provided by the Australian Council of Professors and Heads of
Information Systems (ACPHIS).
177
(10%)
(3) AIS Journals nominated by the AIS “college of senior scholars” including
journals nominated by special interest groups (SIGs) as “worthy of broader
AIS recognition extending beyond the basket of 8” (AIS 2011).
55
(11%)
(4) WoS
based
Based on the WoS category ‘Computer Science and Information Systems’,
extended with additional IS journals in order to identify “a large enough
sample to reflect the diversity of IS journals” (Stewart et al 2017, p. 549).
98
(30%)
(5) Citation
based
Citation-based analysis with the aim to develop “an objective method to
identify a set of core journals” (Chan et al. 2015, p. 239).
35
(0%)
(6) LSE LSE’s ranking to inform promotions decisions and to “advise junior staff and
students on where they should publish.” (Willcocks et al. 2008, p. 163)
46
(26%)
(7) US Public
Universities
Based on the publications of “106 full-time, tenured IS faculty members at
[...] top public research universities” in the US (Holsapple 2009, p. 122).
43
(7%)
(8) Scopus IS
List
In addition, we added three Scopus “subject areas” designated IS: First,
‘Computer Science – Information Systems’, oriented towards technical
perspectives on Information Systems.
309
(52%)
(9) Scopus
MIS List
As above, for: ‘Business – Management Information Systems’, oriented
towards the use of Information Systems for business management.
99
(33%)
(10) Scopus
ISM List
As above, for: ‘Decision Sciences – Information Systems and Management’,
oriented towards the use of IS to assist decision-making.
108
(32%)
(11) John
Lamp’s List
Since 1995 John Lamp actively maintains an Index of Information Systems
Journal. The aim of the index is to be exhaustive in its coverage. For our
purpose we chose to include all journals listed as active (‘current’).
6811
(61%)
Taken together, the eleven lists summarised in Table 1 identify 1,042 different journals. Out of the
eleven lists, ten contain journals that are unique to only one list leading to a list of 714 journals unique
to a single journal listing (Table 1). Thus, when assessing journals as to how often they appear across
different journal lists, we see a strong concentration pattern as per Table 2 below. This skewed
distribution can be expected as it is in line with what can be observed for most bibliometric
phenomena (Wilson 1999): skewed distribution patterns are commonly observed when looking at
authors publishing on a topic, publications on a given topic across journals or citations across
publications. A possible explanation for the skewed distribution of journals across lists is that they are
published by bodies with differing expertise and research orientation, using various methodologies for
1 John Lamp’s list has 684 entries marked ‘current’; however, of these, 3 are internally duplicated:
Information, Communication and Society (appears twice); International Journal of Information
Communication Technologies and Human Development (appears twice); International Journal
of Technology Diffusion (appears twice).
4
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
identifying IS journals. Thus, while there is agreement on core IS journals, IS as a reference discipline
(Baskerville and Myers 2002) is connected to the interests of diverse related communities. These
communities will be reflected differently across journal lists and hence the composition of journals in
different listings will vary. For instance, Burgess et al. (2017) argue for the existence of sub-
communities within IS such as E-commerce, Human-Computer Interaction, Management IS, among
others. Likewise, the range of SIGs within the AIS and the range of journals they nominate suggest that
there is a range of journals receptive to different types of IS research.
Table 2. Appearance of journals across 11 IS journal listings and their coverage in Scopus.
# of lists
journal is
listed in
11 10 9 8 7 6 5 4 3 2 1 Total
# journals
in this set 5 1 1 5 4 13 22 37 72 168 714 1,042
% of total 0.5% 0.1% 0.1% 0.5% 0.4% 1.2% 2.1% 3.6% 6.9% 16% 69% 100%
# indexed
by Scopus 5 1 1 5 4 13 22 36 67 148 545 847
% of set
indexed ALL ALL ALL ALL ALL ALL ALL 97% 93% 88% 76% 81%
Based on the skewed appearance of journals across listings, we see the emergence of a ‘core’ of IS
journals appearing across the majority of lists. For instance, besides the Bo8 only four journals are
listed in 8 or more rankings (see figure 1 below). We therefore argue that a list of 51 journals appearing
in five or more lists would constitute a suitable set for embarking on initial exploratory literature
searches. As the review process progresses or if insufficient number of results are retrieved IS
researchers may want to ‘cast a wider net’, hence literature searches can be further expanded into
journals appearing across fewer journal lists.
Figure 1. IS journals appearing in 5 or more lists including years covered in Scopus.
5
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
4 Building a Search Artifact
To realise the vision of an IT artifact for literature searches, we needed to connect to an existing
literature database covering the majority of the journals identified as publishing research of relevance
to IS. Our first attempt, based on Google Scholar, was of limited success. While Google Scholar allows
users to manually specify the journal to target for a particular search it does not allow this journal filter
to compound. We would therefore need to generate a different results page for each journal searched.
As many research universities provide their staff with access to Scopus we thus saw an opportunity to
leverage the existing Scopus advanced search functionality for building a tool that enables targeted
searches for IS. Advanced searches in Scopus allow the use of a retrieval language which adds
compounding conditions to searches beyond keywords. For instance, all resources indexed in Scopus
have a unique SOURCE-ID. Thus, an advanced search for ‘SOURCE-ID(12402)’ will identify all
articles published in MIS Quarterly that are indexed by Scopus. When the SOURCE-ID is combined
with keywords one is thus able to search through the title, keywords and abstracts of articles published
in MIS Quarterly. Furthermore, additional conditions can be included in searches such as document
types in order to search only for literature review articles or editorials or year of publication. The
ability to search for review articles and editorials are particularly useful for exploratory searches.
Literature reviews enable the identification of publications that offer an overview of earlier research. A
search through editorials is useful for identifying special issues on particular topics.
To proceed, we first matched the complete set of all journals identified by us (see Tables 1, 2) to their
respective SOURCE-ID registered in Scopus. In the case of John Lamp’s list a SCOPUS-ID was already
provided for most journals for other lists used identifying features such as ISSN or journal titles. This
process was facilitated by a relational database system (mySQL) in which we captured other useful
attributes such as Scopus coverage years and journal URL, where applicable. Of the 1,042 unique
journals identified during the discovery phase, 847 are indexed by Scopus (81%, see Table 2). As
shown in Figure 1, all 51 journals included in 5 or more lists are currently indexed by Scopus with
coverage in many cases is also going back several decades. Using the relational database of SOURCE-
ID for IS journals we developed a web-based graphical user interface accessible at www.litbaskets.io in
which users can generate search strings for Scopus’ advanced search and customise the selection of
journals. IS researchers can thus use our tool to generate elaborated search strings for Scopus
advanced search that enables simultaneous search in up to 847 journals publishing research or interest
to IS. All users have to do is copy the search string into Scopus’ advances search window (Figure 2).
Figure 2. Transferring Scopus Query String into Scopus Advanced Search.
Furthermore, if users are accessing Scopus from within an organisational network that has access to
Scopus (e.g. on campus or via VPN) search queries can be directly performed in Scopus. Since we
established our own relational database containing Scopus SOURCE-IDs, we can directly generate
predictive URLs for specific Scopus searches. As URL length are capped, direct searches work only for
sets of ~52 journals. However, as our artifact generates complete search strings using the correct
6
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
syntax for Scopus’ advanced search, simultaneous searching across all 847 journals indexed by Scopus
is possible by copying the generated search string directly into Scopus (Figure 2).
Our codebase is open-source and available on GitHub2 to facilitate the development of our search
artifact by the IS community. Our search artifact is tentatively named Litbaskets (short for Literature
Baskets) and deployed at www.litbaskets.io. It has the following features:
‘Search’: Landing page with a search box and instructions. The instructions encourage the user
to think about the nature of their search, and on that basis, make adjustments using the
configuration pages (see below). If the user chooses to proceed with the search without
customising the settings, the search is performed on our default core set of 51 journals with the
user-entered search terms matched against the title, keywords and abstract fields in Scopus.
Upon submitting their search and if fewer than 52 journals are searched, the user (assuming
they have access to Scopus) is redirected to a Scopus search results page in a new browser tab.
In addition, a message appears on www.litbaskets.io with the relevant query string that can be
pasted directly into Scopus’ advanced search if needed (see Figure 2), and options to execute
the same search directly on AISeL or the dblp.org bibliographic library. (Although dblp.org
relates to Computer Science and not IS, we provide the option for the cases where the research
topic is sufficiently technical to make use of Computer Science literature.)
‘1. Choose Litbasket’: Configuration page that allows the user to select a ‘litbasket’, which is
simply a predefined set of journals that the search can be performed on. The default litbasket
is our ‘core’ set of 51 journals, which we call ‘L5’ (mentioned in 5 or more of our reference lists,
as per Figure 1). Other litbaskets based on list counts range from the largest, ‘L1’ (all 847
journals indexed by Scopus) to the smallest, ‘L7’ (only the 16 journals that appear in 7 or more
of our reference lists, as per Figure 1).We also include an even smaller litbasket than L7 which
is the ‘Bo8’ option so that users can, if they deem appropriate, still simply just search through
the AIS Basket of Eight.
‘2. Customise Sources’: Configuration page that allows the user to select or deselect individual
journals to make adjustments to the ‘litbasket’ that they have selected. To assist the user to
find a specific journal, this configuration page includes all reference lists from Table 1, as well
as the aggregate of the journals from all lists combined. Thus, users can customise a particular
journal set from all 847 journals based on the specific needs imposed by their research topic.
‘3. Apply Filters’: Configuration page that allows the user to use additional Scopus search
features. For example, in order to return only current research, there is a default filter that
limits search results to only retrieve articles published from 2013 onwards. If older results are
desired, researchers can adjust this filter. In addition, Scopus (unlike Google Scholar) indexes
articles into document types. This enables filtered searches: either only for literature review
articles (for an overview of earlier research and how studies are related to each other); or only
for editorials (for identifying special issues on particular topics).
5 Assessing the Artifact
Two crucial measures for assessing any search system are recall and precision (Salton and McGill
1983). Underpinning both measures is the criterion of relevance, since items retrieved in a search can
either be described as addressing (‘relevant’), or not addressing (‘irrelevant’), what is sought. Recall is
a measure describing how many of all potentially relevant items available in a search system are
retrieved by a search. Ideally, a retrieval system would aim for a recall of 100% to identify all relevant
items available. A search system that always reproduces all items available in response to a query
would, by definition, have a perfect recall. Nonetheless, such a system would be largely unusable due
to the aforementioned impracticality effect.
Therefore, precision is established as a second measure of the exactness of retrieval systems. This
measure assesses how many of the results retrieved by the system are actually relevant. Ideally, a
perfect retrieval system would aim for a precision of 100% if, in response to a query, the system only
showed users documents that are relevant to them. However, in practice, attempts to achieve high
precision result in too few results, as per the aforementioned insufficiency effect. This is the recall-
precision trade-off relationship (Buckland and Gey 1994), wherein an increase in recall generally
comes at the expense of precision, and searches with high precision will generally yield a lower recall.
Figure 3 visualises the recall-precision trade-off, and how it leads to the impracticality effect and the
insufficiency effect.
2 GitHub repository: https://github.com/blairw/litbaskets
7
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
Figure 3. How www.litbaskets.io manages the recall-precision trade-off.
To exemplify the usefulness of www.litbaskets.io for dealing with both the insufficiency effect (when
too little literature is found) and the impracticality effect (when researchers are overwhelmed with too
many results), we conduct some exemplary searches. Since the actual judgement of whether a paper is
relevant will depend on what a researcher seeks to achieve, we chose to use the number of documents
retrieved as a proxy for judging how encompassing the results retrieved by our artifact are in
comparison to existing alternatives. To do this, we selected four broad search terms that we believe are
representative for what may be initially used by researchers conducting exploratory searches (Table 3).
Table 3. Number of results for different search tools using example searches based on title,
abstract and keywords and publication year > 2013.
Search
Term
AIS
Bo8 AISeL Litbaskets.io
Core Set (‘L5’)
Litbaskets.io
Full Set (‘L1’) WoS Google
Scholar
blockchain 4 154 51 692 2,974 ~24,100
crowdfunding 25 144 77 204 1,302 ~20,400
e-health 2 135 104 830 6,187 ~78,900
outsourcing 63 243 338 1,452 18,338 ~84,300
*Searches in Google Scholar could not be limited to abstracts, titles, keywords only.
As aforementioned, relying on a set of ‘top journals’ is problematic when conducting literature reviews.
Further looking at the number of documents retrieved using titles, abstracts and keywords for the AIS
Bo8, it is seems that for many searches too few documents are retrieved for a comprehensive literature
review. When limiting the search to titles, abstracts and subjects in the AISeL the number of
documents retrieved are more promising. However, looking more closely at the results almost all of
them are conference papers. We thus believe that the AISeL can offer a good additional resource to
litbaskets.io for identifying conference papers. In contrast, the number of search results retrieved from
Web of Science (WoS) and Google Scholar indicate an impracticality effect as researchers would have
to traverse through thousands of documents of which many will be irrelevant to the research topic.
While the usefulness of the ability for IS researchers to search through a specific set of IS journals
should be evident, the number of results for www.litbaskets.io further exemplifies this. As can be seen
in Table 3, users are able to interactively manage the focus of their literature searches and thereby
control the number of documents retrieved. Our core set (‘L5’) in Table 3 is for searches across
journals listed in Figure 2, i.e., those appearing in the majority of all eleven of our reference lists. As
the range of journals through which a search is conducted increases, so does the number of documents
retrieved. This is illustrated by the increase in the number of documents retrieved for the full set of all
847 journals indexed by Scopus. Thus our artifact allows researchers to manage the recall-precision
8
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
trade-off as searchers can increase the number of journals if the recall in response to their query is too
low. However, too many documents are retrieved for an initial engagement with the literature
researchers can easily apply filters such as review articles only’ or reduce the number of journals
through which a search is conducted.
6 Discussion
6.1 Contributions
The contributions of our research are related to the bibliometric work underpinning the creation of our
list of IS journals as well as the IT artifact we created. Firstly, our artifact contributes an important
new means for discovering IS knowledge. Many established disciplines have a primary resource for
discovering publications, yet no comparable resource exists for IS. Our tool thus addresses an
important need as discipline-specific literature reference databases fulfil notable functions: (1) They
make literature accessible through searches; (2) They support researchers in making knowledge claims
based on a disciplines discourse by ensure good coverage of established knowledge; (3) They select
specific sources to filter out predatory journals and any sources not central to disciplinary knowledge.
Moreover, by enabling a search through an extensive set of journals in which research of relevance to
IS may be published, we enable IS researchers to improve the effectiveness of their literature searches.
Recall and precision and therefore the effects of insufficiency as well as the impracticality are related to
both the range of sources considered for a literature search as well as the search terms chosen for the
search. We manage and optimise precision by reducing the sources considered for searching, from the
broad coverage offered by large multidisciplinary sources such as Google Scholar or WoS to a more
targeted and specific set of journals. Furthermore, Scopus further supports the literature identification
process as it allows researchers to employ elaborate search strategies such as successive fractions or
snowballing through its ‘refine search’ function (Boell and Cecez-Kecmanovic 2014). Hence,
researchers are less likely to be overwhelmed by huge numbers of irrelevant documents. We also
manage and optimise recall, as searches through a more targeted set of journals enable a more
thorough search within this set, for instance, by using single word searches or common language in
addition to specialised terminology. Hence, researchers are less likely to miss relevant publications as
they can use more inclusive search terms in their searches without risking that too many irrelevant
documents (‘false positives’) are retrieved.
Our tool therefore is useful to IS researcher in three ways: (1) By enabling targeted searches and filters,
we empower researchers to quickly identify a highly relevant article to be added on top of their reading
list. Based on the reading of such an article, researchers discover further relevant terminology,
constructs, concepts, theories, models, etc. that then inform further literature searches (Boell and
Cecez-Kecmanovic 2014); (2) By using our tool, researchers can also compile lists of relevant articles
related to a topic that provide them with an overview of what is well known and researched within the
wider IS community, thus better frame the contribution of their work to the disciplinary discourse
(Boell and Hovorka 2019); (3) By enabling a targeted search, our tool supports editors by facilitating
identification of potential reviewers.
Finally, the methodology we used for creating the journal list demonstrates that skewed distributions
underpinning bibliometric phenomena also apply to how frequently journals are mentioned across
journal listings. In other words, inclusion of journals in journal lists follows a similar distribution
pattern as, for instance, citations to articles. This finding has implications for journal analysis in
bibliometric studies more widely as it questions the validity of studies underpinned by a single journal
list. For instance, Stewart et al. (2017) make important observation about the cohesion of the IS
discipline. Yet when looking at the set of journals used for the analysis, 30% of the 98 journals did not
appear in any of the other ten journal listings identified by us. Likewise, Burgess et al. (2017)
exploration of the diversity of editorial boards in IS journals was based on the CABS journal listing.
However, the CABS ranking not only includes 6 journals not mentioned in any of the other journal
lists, it also excludes most journals of our journal set of 51 journals appearing in the majority of all
journal lists. Hence our research encourages caution when looking at findings from earlier bibliometric
studies not only for IS, but for bibliometric studies related to journals in general.
6.2 Limitations and Next Steps
Although our search tool enables most IS researchers to begin comprehensive literature reviews on
most IS topics, it is not perfect and has some limitations. Firstly, users must have access to Scopus.
Secondly, while we identified an extensive set of IS journals, depending on the research undertaken
9
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
additional journals should be searched. Thirdly, our search tool enables searching though journal
article and excludes other publications such as conference proceedings or book chapters. These
limitations do, however, have mitigating factors. Firstly, Scopus access is very common. We inspected
the institutional login page for Scopus and found that, for example, every university that has ever
hosted the Australasian Conference on Information Systems (ACIS) currently has institutional access
to Scopus (Scopus 2019). Secondly, we argue that by starting with journal papers – i.e., documentation
of completed research projects in their first literature review iteration, IS researchers can have a
better appreciation for conference papers that they discover in later iterations. Furthermore our
artifact can be paired with AISeL that offers much better coverage of IS conference, but lacks coverage
of IS journals. We thus recognise the value of potentially including additional databases in future to
reduce the high level of reliance on Scopus, and we are currently investigating the technical feasibility
of connecting to APIs available for dblp.org and Google Scholar.
One of the reasons for making our codebase entirely open-source is to encourage other IS academics to
contribute, such that the project as it is ‘by the community, for the community’. We take inspiration
from other significant open-source projects started in academia and a view that the IS community has
a critical mass of members who possess the programming skills required to participate. For instance,
we would welcome an initiative to extend the coverage of www.litbaskets.io to also include the
proceedings of leading IS conferences in its searches.
7 Conclusion
To make a meaningful contribution, research needs to be framed in relation to the prevailing discourse
of an academic community (Boell and Hovorka 2019; Hovorka and Boell 2015). The literature
identification process is central for understanding share knowledge and hence for expressing new
knowledge claims. However, unlike Medline for Medicine or Psycinfo for Psychology, there is currently
no database that IS researchers can go to as a starting point for conducting exploratory literature
searches, leading to either too little literature to review (the insufficiency effect) or too much literature
to review (the impracticality effect). Our study addresses this deficit by identifying a set of journals
that can be considered as receptive to IS research. By building www.litbaskets.io as an interface for
searching through these journals we hope to not only make literature searches more effective for IS
researchers, but also to facilitate the IS discipline uniting around a core of shared knowledge.
8 References
Alvesson, M., and Sandberg, J. 2014. “Habitat and Habitus: Boxed-in versus Box-Breaking Research”,
Organization Studies (35:7), pp. 967–987.
Association for Information Systems (AIS) 2011. “Senior Scholars’ Basket of Journals” [online].
https://aisnet.org/general/custom.asp?page=SeniorScholarBasket
Australian Business Deans Council (ABDC) 2016. “ABDC Journal Quality List” [online].
https://abdc.edu.au/research/abdc-journal-list/
Baskerville, R.L., and Myers, M.D. 2002. “Information Systems as a Reference Discipline”, MIS
Quarterly (26:1), pp. 1–14.
Bates, M.J. 1989. “The design of browsing and berrypicking techniques for the online search interface”,
Online Review (13:5), pp. 407–424.
Boell, S.K., and Cecez-Kecmanovic, D. 2014. “A Hermeneutic Approach for Conducting Literature
Reviews and Literature Searches”, Communications of the AIS (34:1), pp. 257–286.
Boell, S.K., and Cecez-Kecmanovic, D. 2015. “On Being ‘systematic’ in Literature Reviews in IS”,
Journal of Information Technology (30:2), pp. 161–173.
Boell, S.K. and Hovorka, D.S. 2019. “Writing, Arguing, Contributing - A Cogent Argumentation
Framework for Identifying, Specifying, and Evaluating Research Contribution”, Australasian
Journal of Information Systems (23). 10.3127/ajis.v23i0.1857.
Buckland, M., and Gey, F. 1994. “The relationship between Recall and Precision”, Journal of the
American Society for Information Science (45:1), pp. 12–19.
Burgess, T.F., Grimshaw, P., and Shaw, N.E. 2017. “Research Commentary—Diversity of the
Information Systems Research Field: A Journal Governance Perspective”, Information Systems
Research (28:1), pp. 5–21.
Cabanac, G. 2012. “Shaping the Landscape of Research in Information Systems From the Perspective
of Editorial Boards: A Scientometric Study of 77 Leading Journals”, Journal of the American
Society for Information Science and Technology (63:5), pp. 977–996.
Chan, H.C., Guness, V., and Kim, H.-W. 2015. “A Method for Identifying Journals in a Discipline: An
Application to Information Systems”, Information and Management (52:2), pp. 239–246.
10
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
Chartered Association of Business Schools (CABS) 2018. “Academic Journal Guide 2018” [online].
https://charteredabs.org/academic-journal-guide-2018/
Diederich, S., Brendel, A.B., and Kolbe L.M. 2019. “On Conversational Agents in Information Systems
Research: Analyzing the Past to Guide Future Work”, Proceedings of the 14th International
Conference on Wirtschaftsinformatik, p. 1550-1564.
Gillenson, M., and Stafford, T. 2008. “Journal Rankings 2008: A Synthesis of Studies”, Americas
Conference on Information Systems (AMCIS), Paper 58.
Greenhalgh, T., and Peacock, R. 2005. “Effectiveness and efficiency of search methods in systematic
reviews of complex evidence: audit of primary sources”, British Medical Journal (331:7524), pp.
1064–1065.
He, J., Ping, Q., Lou, W., and Chen, C. 2019. “PaperPoles: Facilitating Adaptive Visual Exploration of
Scientific Publications by Citation Links,” Journal of the Association for Information Science
and Technology (70:8), pp. 843-857.
Holsapple, C.W. 2009. “A new map for knowledge dissemination channels”, Communications of the
ACM (52:3), pp. 117–125.
Hovorka, D.S., and Boell, S.K. 2015. “Cogency and Contribution in IS Research”, International
Conference on Information Systems, Fort Worth (USA), Paper 1574.
Lamp, J. W. 2004. Index of Information Systems Journals. Geelong, Deakin University [online].
http://lamp.infosys.deakin.edu.au/journals/
Larsen, K.R., Hovorka, D.D., Dennis, A.R., and West, J.D. 2019. “Understanding the Elephant: The
Discourse Approach to Boundary Identification and Corpus Construction for Theory Review
Articles”, Journal of the AIS (20:7), pp. 887-916.
Lewis, B., Templeton, G., and Luo, X. 2007. “A Scientometric Investigation into the Validity of IS
Journal Quality Measures”, Journal of the AIS (8:12), pp. 619–633.
Lowry, P.B., Moody, G.D., Gaskin, J., Galletta, D.F., Humpherys, S.L., Barlow, J.B., and Wilson, D.
2013. “Evaluating Journal Quality and the Association for Information Systems Senior Scholars’
Journal Basket Via Bibliometric Measures: Do Expert Journal Assessments Add Value?”, MIS
Quarterly (37:4), pp. 993–1012.
Marchionini, G. 2006. “Exploratory search: From finding to understanding”, Communications of the
ACM, (49:4), pp. 41–46.
Paré, G., Trudel, M.-C.C., Jaana, M., and Kitsiou, S. 2015. “Synthesizing Information Systems
Knowledge: A Typology of Literature Reviews,” Information and Management (52:2), pp. 183–
199.
Salton, G., and McGill, M.J. 1983. Introduction to modern information retrieval, New York: McGraw-
Hill.
Savolainen, R. 2017. “Berrypicking and information foraging: Comparison of two theoretical
frameworks for studying exploratory search”, Journal of Information Science (43:3), pp. 1–14.
Scopus 2019. Login via your Institution [online], https://www.scopus.com/checkaccess.uri
Schuster, R., Wagner, G., and Schryen, G. 2018. “Information Systems Design Science Research and
Cumulative Knowledge Development: An Exploratory Study”, International Conference on
Information Systems, San Francisco (USA).
Stewart, A., Cotton, J. L., and Adya, M. 2017. “Information Systems: A House Divided?”,
Communications of the AIS (41:24), pp. 544–586.
Tate, M., Furtmueller, E., Evermann, J., and Bandara, W. 2015. “Introduction to the special issue: The
literature review in information systems”, Communications of the AIS (37), Paper 5.
vom Brocke, J., Simons, A., Riemer, K., Niehaves, B., Plattfaut, R., and Cleven, A. 2015. Standing on
the shoulders of giants: Challenges and recommendations of literature search in information
systems research, Communications of the AIS (37), Paper 9.
Walstrom, K.A., and Hardgrave, B.C. 2001. “Forums for information systems scholars: III,”
Information and Management (39:2), pp. 117–124.
Webster, J., and Watson, R.T. 2002. Analyzing the past to prepare for the future: Writing a literature
review. MIS Quarterly (26:2), pp. 13-23.
Willcocks, L., Whitley, E. A., and Avgerou, C. 2008. “The ranking of top IS journals: A perspective
from the London School of Economics”, European Journal of Information Systems (17:2), pp.
163–168.
Wilson, C.S. 1999. “Informetrics”, Annual Review of Information Science and Technology (34), pp.
107–247.
Winkler, T. J., and Wessel, M. 2018. “A Primer on Decision Rights in Information Systems: Review
and Recommendations”, International Conference on Information Systems, San Francisco
(USA).
11
Australasian Conference on Information Systems Boell & Wang
2019, Perth Supporting Exploratory IS Literature Searches
Copyright
Copyright: © 2019 Boell & Wang. This is an open-access article distributed under the terms of the
Creative Commons Attribution-NonCommercial 3.0 Australia License, which permits non-commercial
use, distribution, and reproduction in any medium, provided the original author and ACIS are
credited.
12
... an IT artefact developed to assist scholars in retrieving relevant literature from the most recognized databases within the IS discipline. 53 Litbaskets.io converts the given search string to be used by Scopus's advanced search via ISSN numbers of selected outlets, thus providing the possibility to search across indexed scientific sources. 53 Scholars can further apply different filters and restrictions, that is, selecting specific outlets or limiting by a year, to narrow down the search results based on certain criteria. ...
... 53 Litbaskets.io converts the given search string to be used by Scopus's advanced search via ISSN numbers of selected outlets, thus providing the possibility to search across indexed scientific sources. 53 Scholars can further apply different filters and restrictions, that is, selecting specific outlets or limiting by a year, to narrow down the search results based on certain criteria. 53 Since we aimed at obtaining a comprehensive overview of the current research on ethics in the context of AI and digital health, we deliberately selected the largest filter containing 847 outlets (for a full list of the included outlets, please see https://www.litbaskets.io/). ...
... 53 Scholars can further apply different filters and restrictions, that is, selecting specific outlets or limiting by a year, to narrow down the search results based on certain criteria. 53 Since we aimed at obtaining a comprehensive overview of the current research on ethics in the context of AI and digital health, we deliberately selected the largest filter containing 847 outlets (for a full list of the included outlets, please see https://www.litbaskets.io/). Furthermore, we did not limit our search starting with a specific year. ...
Article
Full-text available
The application of artificial intelligence (AI) not only yields in advantages for healthcare but raises several ethical questions. Extant research on ethical considerations of AI in digital health is quite sparse and a holistic overview is lacking. A systematic literature review searching across 853 peer-reviewed journals and conferences yielded in 50 relevant articles categorized in five major ethical principles: beneficence, non-maleficence, autonomy, justice, and explicability. The ethical landscape of AI in digital health is portrayed including a snapshot guiding future development. The status quo highlights potential areas with little empirical but required research. Less explored areas with remaining ethical questions are validated and guide scholars’ efforts by outlining an overview of addressed ethical principles and intensity of studies including correlations. Practitioners understand novel questions AI raises eventually leading to properly regulated implementations and further comprehend that society is on its way from supporting technologies to autonomous decision-making systems.
... After (Boell andWang, 2019, Wagner et al., 2021) and for leading IS conference proceedings in AISeL (AMCIS, ECIS, HICSS, ICIS, PACIS, WI). Litbaskets.io provides an artifact to search for IS journal publications that are ranked in multiple, reputable lists (Boell and Wang, 2019). ...
... After (Boell andWang, 2019, Wagner et al., 2021) and for leading IS conference proceedings in AISeL (AMCIS, ECIS, HICSS, ICIS, PACIS, WI). Litbaskets.io provides an artifact to search for IS journal publications that are ranked in multiple, reputable lists (Boell and Wang, 2019). Table 2 displays an overview of our search and the screening process. ...
Conference Paper
Full-text available
Over the last years, configurational research has become increasingly popular in the Information Systems (IS) discipline. Researchers value configurational methods like Qualitative Comparative Analysis (QCA) as their application contributes to a better understanding of complex phenomena. QCA helps to uncover interrelations of conditions that lead to an outcome, building on the principles of equifinality, conjunctural causation, and asymmetry. More recently, IS researchers have started to analyze qualitative data, like case study data, with QCA. However, there is a lack of methodological guidance on how to calibrate qualitative data into set membership values for QCA. Therefore, this paper structures methodological steps and the associated options to calibrate qualitative data from an interdisciplinary perspective and critically reviews the observed methodological choices in IS research. This paper also gives recommendations for calibrating qualitative data to support informed methodological choices for future research.
... We used the tool litbaskets.io to construct a search string which allowed us to search the title, abstract, and keywords of the top 302 IS publications in the Scopus database [38]. We searched for all publications that had been published until December 2021. ...
Conference Paper
When done right, the use of low code development promises a significant competitive advantage in the software development process for organizations. Thus, multiple vendors have created low code development platforms to ease the use of low code development. However, current research on low code development platforms mainly focuses on the technological aspects of the platforms but not on their adoption. Hence, it remains unclear what drives and inhibits the adoption of low code development platforms. We conducted a literature review and identified thirteen factors that inhibit the adoption and seven factors that drive it. We structure these factors along with the diffusion of innovation framework that helps to disentangle drivers and inhibitors. As a result, we provide an initial explanation of the adoption of low code development platforms. Nevertheless, we conclude that existing research on the adoption of low code development platforms is not specific enough to understand the phenomenon substantially. Further, for some factors (e.g., cost), there is a disagreement in the academic literature on whether they are drivers or inhibitors. Hence, we identify gaps and derive avenues for future research.
... The sample is compliant to the sampling model introduced by Charan and Biswas (2013). In addition, the sampling approach allows generalisation of findings as recommended by Boell and Wang (2019). ...
Conference Paper
Full-text available
This paper analysed 30 NGOs based in Tanzania so as to establish the Digital Knowledge Management gap that exists to their systems. In addition, the paper used the literature of the African origin to determine challenges facing organisations in the implementation of Digital Knowledge Management Systems; this was followed by highlighting opportunities for enhancing the use of Digital Knowledge Management Systems within the continent. The study was descriptive in nature. Results confirmed that Knowledge Management Systems owned by organisations do not meet minimum technical criteria. In addition, there were challenges associated with how to work with the information, there were challenges on how to manage Knowledge Systems, and challenges on how to deal with the technology. The following are the opportunities for enhancing Digital Knowledge Management-the use of Open Sources, engaging higher learning institutions in knowledge management issues, and researching for contents relevant to NGOs.
... Litbaskets is an information technology artifact supporting exploratory literature searches for information systems research(Boell & Wang, 2019). ...
Article
The ethical dimensions of Artificial Intelligence (AI) constitute a salient topic in information systems (IS) research and beyond. There is an increasing number of journal and conference articles on how AI should be designed and used. For this, IS research offers and curates knowledge not only on the ethical dimensions of information technologies, but also on their acceptance and impact. However, the current discourse on the ethical dimensions of AI is highly unstructured and seeks clarity. As conventional systematic literature research has been criticized for lacking in performance, we applied an adapted discourse approach to identify the most relevant articles within the debate. As the fundamental manuscripts within the discourse were not obvious, we used a weighted citation-based technique to identify fundamental manuscripts and their relationships within the field of AI ethics across disciplines. Starting from an initial sample of 175 papers, we extracted and further analyzed 12 fundamental manuscripts and their citations. Although we found many similarities between traditionally curated ethical principles and the identified ethical dimensions of AI, no IS paper could be classified as fundamental to the discourse. Therefore, we derived our own ethical dimensions on AI and provided guidance for future IS research.
... Even though available methodological guidance (e.g., in the form of quality criteria or procedure models) helps to perform literature reviews, it is however still challenging to get started with the review and to formulate an appropriate search strategyespecially for students and novice researchers. We argue that this can be attributed to three main indications: First, although the review's starting point is of great importance because "[failures] have a considerable impact on the resulting review's quality" [11], (IS) researchers often face the 'dilemma at the start' [11] because they have no clear orientation where to begin searching like in other disciplines that have, for instance, established overviews of reference databases [12]. Second, although IS research is often intertwined or linked with adjacent disciplines, conventional corpus creation often focuses on parts of a phenomenon without the benefit of a larger context, which leads to deficits in terms of identifying the majority of research [13]. ...
Chapter
Full-text available
Researchers often face challenges already in the early stages of a literature review, and thus, struggle in getting started with the search and in organizing the process. This starting point is however of great relevance because design decisions such as in terms of corpus creation have impacts on the entire results of the review. By following the design science paradigm, we present the ‘Search Canvas’, a generic framework that aims at supporting the (creative) process of exploring, specifying, and visually representing a literature search strategy. In doing this, we contribute to the understanding of what components need to be considered when deriving a search strategy and provide an instrument that enables researchers to iteratively plan and communicate such strategies.
... To capture any critical IS research not citing Myers and Klein (2011) but still engaging with critical social theory, we performed two follow-up searches for critical social theory ("critical AND social AND theory") and emancipation ("emancipation OR emancipatory OR emancipate") through a library of 133 IS journals using our Scopus-based platform, www.litbaskets.io (Boell and Wang 2019). Our library of 133 journals consisted of journals included in the AIS Basket of 8, 41 AIS SIG journals, 35 IS journals listed by Chan et al. (2015), and/or the 94 IS journals listed by Stewart et al. (2017). ...
Conference Paper
Full-text available
This paper presents a literature review of critical information systems (IS) research. Specifically, it focuses on how IS researchers have responded to Myers and Klein’s (2011) call to consider critical approaches and theorists in addition Bourdieu, Foucault and Habermas. The review identifies and discusses three types of critical IS research “beyond Bourdieu, Foucault and Habermas”: work based on a) (other) critical grand social theories, b) postcolonialism and c) data-focused critical methods (i.e., Capabilities Approach, Critical Discourse Analysis, Critical Heuristics and Design, Frame Analysis and Phronetic Enquiry). Based on the literature review, the paper maps the landscape of critical approaches and theories and identify their origins. This analysis is helpful for IS researchers interested in conducting critical IS research by charting the range of critical research approaches beyond Bourdieu, Foucault and Habermas.
Article
Full-text available
Despite the ongoing expansion of agile practices beyond software firms, agile adoption remains risky, challenging, and poorly understood. Although agile practices emphasize self-organization and customer collaboration, we know little about how customers influence agile adoption within self-organizing teams. Here, we analyze how a commissioned software team engaged in customer collaboration during agile adoption at a Danish IT service provider. Our case study shows that the software team's transition to self-organized teamwork practices, agile planning routines, and active customer engagement was mutually dependent on the customer's trust in the software team and flexible collaborative routines. As a result, we advance a theoretical perspective of customer influence on agile adoption within commissioned software teams, implying that both software teams and customers need to navigate a contradictory tension between self-organization and collaboration to become agile together.
Article
Full-text available
Artificial intelligence (AI) is beginning to transform traditional research practices in many areas. In this context, literature reviews stand out because they operate on large and rapidly growing volumes of documents, that is, partially structured (meta)data, and pervade almost every type of paper published in information systems research or related social science disciplines. To familiarize researchers with some of the recent trends in this area, we outline how AI can expedite individual steps of the literature review process. Considering that the use of AI in this context is in an early stage of development, we propose a comprehensive research agenda for AI-based literature reviews (AILRs) in our field. With this agenda, we would like to encourage design science research and a broader constructive discourse on shaping the future of AILRs in research.
Article
Full-text available
The predominant means by which research becomes visible and accessible to the research community is through publication. Generally, publication requires careful framing of the research in relation to existing knowledge. As a contribution to knowledge cannot be self-evident, authors must persuade, through argumentation, the editors, reviewers, and the research community that their work offers a contribution. In Information Systems, the discussion of argumentation is often limited to the logic dimensions of argumentation, namely deductive, inductive, and abductive reasoning. In this paper, we demonstrate that argumentation requires the consideration of three additional dimensions of argumentation: rhetoric, dialectic, and social-institutional. Kuhn’s concept of the disciplinary matrix is introduced as the background toward which a cogent argument is directed and against which contribution is evaluated. We then illustrate the role of argumentation through the example of the seminal paper by Orlikowski and Iacono on the role of IT in Information Systems research. Understanding the importance of argumentation in framing one’s research contribution is critical to authors, editors, and reviewers alike within and beyond Information Systems and its reference disciplines.
Conference Paper
Full-text available
Conversational agents (CA), i.e. software that interacts with its users through natural language, are becoming increasingly prevalent in everyday life as technological advances continue to significantly drive their capabilities. CA exhibit the potential to support and collaborate with humans in a multitude of tasks and can be used for innovation and automation across a variety of business functions, such as customer service or marketing and sales. Parallel to the increasing popularity in practice, IS researchers have engaged in studying a variety of aspects related to CA in the last few years, applying different research methods and producing different types of theories. In this paper, we review 36 studies to assess the status quo of CA research in IS, identify gaps regarding both the studied aspects as well as applied methods and theoretical approaches, and propose directions for future work in this research area.
Conference Paper
Full-text available
The decision rights construct deserves increasing attention in the Information Systems (IS) field, yet integrative knowledge is sparse. This review first establishes two key distinctions from managerial theory regarding decision rights classes (decision input, control, and management) and manifestations (formal versus practiced) and then analyzes the current state of the IS literature based on 53 articles published in nine leading IS outlets between 1992 and 2018. Our review identifies six themes of IS research that have used the decision rights construct on various levels, with differing conceptualizations, and in different nomological roles. We find that the majority of studies mix decision control with decision management rights (38%) and focus on formal manifestations only (68%). We close by putting forward five recommendations, arguing that a more nuanced conceptualization of decision rights will aid future IS research in addressing contemporary IS phenomena.
Article
Full-text available
The goal of a review article is to present the current state of knowledge in a research area. Two important initial steps in writing a review article are boundary identification (identifying a body of potentially relevant past research) and corpus construction (selecting research manuscripts to include in the review). We present a theory-as-discourse approach which a) creates a theory ecosystem of potentially relevant prior research using a citation-network approach to boundary identification; and b) identifies manuscripts for consideration using machine learning or random selection. We demonstrate an instantiation of the theory as discourse approach through a proof-of-concept, which we call the Automated Detection of Implicit Theory (ADIT) technique. ADIT improves performance over the conventional approach as practiced in past Technology Acceptance Model reviews (i.e., keyword search, sometimes manual citation chaining); it identifies a set of research manuscripts that is more comprehensive and at least as precise. Our analysis shows that the conventional approach failed to identify a majority of past research. Like the three blind men examining the elephant, the conventional approach distorts the totality of the phenomenon. ADIT also enables researchers to statistically estimate the number of relevant manuscripts which were excluded from the resulting review article, thus enabling an assessment of the review article's representativeness.
Conference Paper
Full-text available
Contributing to cumulative knowledge development is a central goal in Information Systems design science research (IS-DSR). IS design theories (ISDTs) have been recognized as a particularly important building block for the accumulation of design-oriented knowledge. Yet, there are limited insights into how follow-up research builds on IS design theories in terms of testing and extending. To address these issues, we present results from the first empirical analysis of how ISDTs have been tested and extended within and beyond IS journals. Our qualitative analysis of papers citing ISDTs uncovers an alarming paucity of follow-up research that builds on these ISDTs. Specifically, the overall number of papers testing and extending any of the selected ISDTs ranges in the single digits. To propose an actionable path forward, we formulate four specific guidelines on how the IS(-DSR) community can facilitate the cumulative extension of the IS knowledge base.
Conference Paper
Full-text available
Although it is accepted that to make a contribution, Information Systems (IS) research must be novel and provide utility, these characteristics are insufficient to account for the cogency or persuasion that research makes a contribution. Novelty and utility are not self-evident and authors need to articulate an argument that their paper makes a contribution in relation to the discourse to which it seeks to contribute. We use the Habermasian ideal of argumentation to explicate the rhetorical, dialectic, logical and socio-institutional dimensions of cogent argument. We use two examples of published research to illustrate how these elements can be extended to frame research as a contribution in relation to the wider disciplinary matrix. We outline next steps of the research and the utility of the framework for researchers, reviewers and editors.
Article
Exploratory search is a specialised form of information searching behaviour that is open-ended, dynamic and multi-faceted in nature. The study elaborates the picture of exploratory search by concentrating on two theoretical frameworks for studying searching of this type, that is, the berrypicking model and the information foraging theory. The study focuses on two major issues: the motivators for exploratory search and the nature of search processes. The findings draw on the conceptual analysis of 30 key studies on this topic. The berrypicking model posits changing information needs as the main motivator for information searching, while the information foraging theory emphasises the role need for performing a task as a main trigger of information seeking. The berrypicking model devotes more attention to focused searching as a key constituent of exploratory search, while the information foraging theory concentrates on the characterisation of exploratory browsing.
Article
Diversity has attracted much attention within the information systems (IS) field, with literature concentrating on diversity in topics and methods. These constitute two of three identified areas of research field diversity; the little-investigated third area includes demographic and social diversity of researchers. This study explores this gap for researchers comprising the editorial advisory boards (EABs) of 52 IS journals and links the underexplored types of diversity to topic diversity. The journals are categorized into seven intellectual communities, using topic affinity of journal content, and a social network of EAB members constructed from board interlocks. The network structure appears to reflect the topic-based community links. Journal communities are aggregated into two components of the social network: a business-school-related core set of journals and a more diverse computing- and engineering-related periphery. The strong ties at the network center do not necessarily reflect journal status. The observed combination of focus and diversity is consistent with a polycentric view of the IS field. Findings suggest low demographic diversity in the field and that demographic diversity correlates with other types of diversity. The field’s separation into business core and computing periphery is highlighted as potentially challenging to the IS field’s identity.
Article
There has been a flowering of scholarly interest in the literature review as a research method in the information systems discipline. We feel privileged to contribute to this conversation and introduce the work of the authors represented in this special issue. Some of the highlights include three new methods for conducting literature analysis and guidelines, tutorials, and approaches for coping with some of the challenges involved in carrying out a literature review. Of the three “new method” papers, one (ontological meta-analysis and synthesis) is entirely new, and two (stylized facts and critical discourse analysis) are novel in the information systems context. The other four paper address more general issues: the challenges of effective search strategies when confronted with the burgeoning volume of research available, a detailed tool-supported approach for conducting a rigorous review, a detailed tutorial for conducting a qualitative literature review, and a discussion of quality issues. Collectively, the papers place emphasis beyond the traditional “narrative synthesis” on the importance of selecting the appropriate approach for the research context and the importance of attention to quality and transparency at all stages of the process, regardless of which approach is adopted.