ArticlePDF Available

How to Perform a Literature Review with Free and Open Source Software

Authors:

Abstract and Figures

As it provides a firm foundation for advancing knowledge, a solid literature review is a critical feature of any academic investigation. Yet, there are several challenges in performing literature reviews including: i) lack of access to the literature because of costs, ii) fracturing of the literature into many sources, lack of access and comprehensive coverage in many databases and search engines, and iii) the use of proprietary software lock-in strategies for bibliographic software, which can make porting literature reviews between organizations cumbersome and costly. These challenges often result in poor quality literature reviews completed by a single researcher unfamiliar with the approaches to the same research in other sub-fields and static reviews that are often lost to the scientific community. In this paper, an open source approach will be expanded to the application of improving the quality of literature reviews by providing best practices. Although there are many types and goals of literature reviews, it is found that all of them can be improved using a tool chain of free and open source software (FOSS) and methods. Specifically, this paper will provide a clear framework for i) comprehensive searching and obtaining access to the literature, ii) the use of FOSS for all steps including browsing, bibliographic software, and writing and iii) documenting a literature review to encourage collaboration of a dynamic document that lives into the future. This approach solves the current challenges of literature reviews and provides benefits of lower labor and economic costs, improved researcher control, and increased potential for collaboration. Finally, the challenges of using this approach and methods to overcome them are reviewed and future work is described.
Content may be subject to copyright.
A peer-reviewed electronic journal.
Copyright is retained by the first or sole author, who grants right of first publication to Practical Assessment, Research & Evaluation. Permission
is granted to distribute this article for nonprofit, educational purposes if it is copied in its entirety and the journal is credited. PARE has the
right to authorize third party reproduction of this article in print, electronic and database forms.
Volume 23 Number 8, May 2018 ISSN 1531-7714
How to Perform a Literature Review with Free and Open
Source Software
Joshua M. Pearce, Michigan Technological University and Aalto University
As it provides a firm foundation for advancing knowledge, a solid literature review is a critical feature
of any academic investigation. Yet, there are several challenges in performing literature reviews
including: i) lack of access to the literature because of costs, ii) fracturing of the literature into many
sources, lack of access and comprehensive coverage in many databases and search engines, and iii)
the use of proprietary software lock-in strategies for bibliographic software, which can make porting
literature reviews between organizations cumbersome and costly. These challenges often result in
poor quality literature reviews completed by a single researcher unfamiliar with the approaches to the
same research in other sub-fields and static reviews that are often lost to the scientific community.
In this paper, an open source approach will be expanded to the application of improving the quality
of literature reviews by providing best practices. Although there are many types and goals of literature
reviews, it is found that all of them can be improved using a tool chain of free and open source
software (FOSS) and methods. Specifically, this paper will provide a clear framework for i)
comprehensive searching and obtaining access to the literature, ii) the use of FOSS for all steps
including browsing, bibliographic software, and writing and iii) documenting a literature review to
encourage collaboration of a dynamic document that lives into the future. This approach solves the
current challenges of literature reviews and provides benefits of lower labor and economic costs,
improved researcher control, and increased potential for collaboration. Finally, the challenges of using
this approach and methods to overcome them are reviewed and future work is described.
A solid review of the prior relevant literature is a
critical feature of any academic investigation as it
provides a firm foundation for advancing knowledge
(Webster & Watson, 2012). Without a good literature
review, a researcher cannot perform significant research
in any field (Boote & Beile, 2005) whether it is for a
research article, a critical review for coursework (Jesson
& Lacey, 2006) or a dissertation (Randolph, 2009). Levy
and Ellis argue that an effective literature review should:
i) methodologically analyze and synthesize quality
literature, ii) provide a foundation to a research topic and
methodology, and iii) demonstrate that the proposed
research would advance the research field’s knowledge-
base (2006). Literature reviews help researchers avoid
redundancy, establish a context and significance of their
research topic, identify primary methodologies in the
field and determine what needs to be done to solve
research problems (Hart, 1998). A well-executed
literature review can also be a legitimate and publishable
document and can be some of the most valuable (and
cited) type of scholarly work if it is a systematic literature
review (Cooper, 1988a; Bettany-Saltikov, 2012; Boote &
Beile, 2005).
There are several challenges in performing literature
reviews. First, some scholars have difficulty gaining
access to all of the literature itself due to the collateral
damage of intellectual property (Boldrin & Levine,
2008). The unintended consequences of copyright
restrictions and paywalls narrow access to the peer-
reviewed literature (Lewis, 2012) to the point that even
Harvard University is challenged to pay for it (Sample,
2012). Researchers simply cannot review what they
cannot read. Secondly, there is also a fracturing of the
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 2
Pearce, Perform a Literature Review with Free and Open Source Software
literature into many topical journals, interdisciplinary
journals, and various society proceedings, which makes
it more challenging to perform a comprehensive
literature review. This is a particular problem in some
fields such as in engineering, which has historically
suffered from weak literature reviews (Shaw, 1995) and
can often be artificially arranged in academic and
research society silos. For example, electric battery-
related research is published in IEEE, ACS, MRS, and
ASME societies as well as conventional journals. Few
scholars are members of all of the relevant organizations.
Coupled to this there are also challenges related to the
lack of comprehensive coverage of the literature in many
databases, lack of access to all databases, the poor quality
of search engines and the difficulty of getting consistent
results from them (Budgen & Brereton, 2006). Finally,
the use of proprietary software lock-in strategies (Zhu &
Zhou, 2012) for bibliographic software such as
EndNote, Reference Manager, Papers, etc. raises costs
and can make porting literature reviews between
universities, between firms, and from universities to
industry cumbersome and costly. For example, a social
science Ph.D. student using a ‘free’ version of a
proprietary software tool while at university A may
create a detailed literature review database on her thesis
topic. However, she may not be able to access upon
graduation even if employed as a professor at university
B, without paying for the software costs or subscriptions
not supported by her new employer. These three
challenges often result in poor quality literature reviews
completed by a single researcher unfamiliar with the
approaches to the same research in other sub-fields and
static reviews that are often lost to the scientific
community as only components of a review are used in
a standard research article.
In this paper an open source approach will be
described to overcome many of these challenges in
reviewing the literature. An open source approach is
well-established to provide improved product
innovation over proprietary techniques of technical
development (Deek, & McHugh, 2007; DiBona &
Ockman, 1999; Lakhani & Von Hippel, 2003; Raymond,
1999; Söderberg, 2015). This has been proven most
effective at software development because free and open
source software (FOSS) provides open innovation and
diversification (Colombo, et al., 2014; Henkel, et al.,
2014; Dodourova & Bevis, 2014), organizational
innovation (Alexy, et al., 2013), cumulative innovation
(Boudreau, & Lakhani, 2016), development efficiency
(Hienerth, et al, 2014), avoids redundant work (Årdal &
Røttingen, 2016), the quality of code is superior
(Söderberg, 2015), and it encourages more creativity
(Martinez, 2015). The open source approach is now also
gaining traction in free and open source hardware most
notably for scientific tools (Baden et al., 2015; Coakley
& Hurt, 2016; Pearce, 2013). Regardless of the
technology, open source is cost effectiveness for use
(Pearce, 2013; Petch, et al., 2014; Riehle, 2007;
Wittbrodt, et al., 2013) and development (Hawkins,
2004; Pearce, 2015a; 2015b; 2017). In this paper, the
open source approach will be expanded to the
application of improving the quality of literature reviews.
Although there are many types and goals of literature
reviews (Cooper, 1988b), all of them can be improved
using a tool chain of free and open source software and
methods. Specifically, this paper will provide a clear
framework for i) comprehensive searching and obtaining
access to the literature, ii) the use of FOSS for all steps
including browsing, bibliographic software, and writing
and iii) documenting a literature review to encourage
collaboration of a dynamic document that lives into the
future.
Comprehensive Searching and Gaining Access to
the Literature
Previously researchers were dependent on having
paid access (normally through their institution) to one or
more of the major proprietary repositories such as Web
of Science, Engineering Village, Academic Onefile,
ProQuest, Ulrichsweb, Scopus, Science Direct
(Elsevier), Wiley Online Library, JSTOR, IEEE Xplore,
ACS Publications, and EBSCO among others. Although
costly these databases were often incomplete and biased.
For example, Web of Science is biased towards science
(instead of engineering), English language journals,
against particular fields and disciplines and has extremely
limited coverage of non-journal sources (Harzing, 2017).
Today there a number of no cost academic search
engines, which are summarized in Table 1.
As can be seen in Table 1, a number of the search
engines specifically focus on open access articles, which
are on the rise and constitute at least 28% of the
literature (Piwowar, et al., 2017). In addition to these
dedicated repositories and search engines shown in
Table 1, the Directory of Open Access Journals also
provides links to the open access publisher’s websites
that all have searchable and freely available open access
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 3
Pearce, Perform a Literature Review with Free and Open Source Software
Table 1. No cost academic search engines for published papers and preprints.
Name Website Description
Comprehensive
Academia www.academia.edu Aplatformforacademicstoshareopenaccessresearch
papersandpreprints.Over61millionacademicsusethesite.
BielefeldUniversity’sAcademic
SearchEngine(BASE)
www.base‐search.net Morethan100milliondocumentswithopenaccessfor60%of
theindexeddocuments.
DirectoryofOpenAccess
Journals
doaj.org Acommunity‐curatedonlinedirectorythatindexesand
providesaccesstohighquality,openaccess,peer‐reviewed
journals.67millionarticlesareindexed.
GoogleScholar scholar.google.com Reasonablycomprehensive,indexesacademicinformation
fromvariousonlinewebresources
MicrosoftAcademicResearch academic.microsoft.com Reasonablycomprehensive,172millionpublications indexed
OSFPrePrints(CenterforOpen
Science)
osf.io/preprints Searchablepreprintsoverawiderangeofdisciplines(contains
2.1million).
Preprints(MDPI) www.preprints.org Aplatformdedicatedtomakingearlyversionsofresearch
outputsavailable,includingoriginalresearcharticlesand
reviews.
ScienceOpen www.scienceopen.com Professionalnetworkingplatformforscholarstoenhance
theirresearchintheopenwith40millionrecords.
SemanticScholar(Allen
Institute)
www.semanticscholar.org Anacademicsearchenginethatutilizesartificialintelligence
methodstoprovidehighlyrelevantresultsandnoveltoolsto
filterthemwithease.
ResearchGate www.researchgate.net Asocialnetworkingwebsiteforresearchers.Containsmore
than100millionpublicationpages.
Specialty
BiolineInternational www.bioline.org.br Focusedonpublichealth,foodandnutritionalsecurity,food
andmedicineandbiodiversity
BioOne www.bioone.org Full‐textdatabaseofmorethan200leadingjournalsinthe
biological,ecological,andenvironmentalsciences.
CERNDocumentServer cdsweb.cern.ch Articles,reportsandmultimediacontentinhighenergy
physics
Cornell’sArXiv arxiv.org Openaccessto1.3millione‐printsinphysics,mathematics,
computerscience,quantitativebiology,quantitativefinance,
statistics,electricalengineeringandsystemsscience,and
economics.
Educational
ResourcesInformationCenter
(ERIC)
eric.ed.gov Coverseducationresearchandinformationforeducators,
researchers,andthegeneralpublic.Enablessearchingfor
peerreviewonlyandfulltextavailability.
Mendeley(Elsevier) www.mendeley.com/research‐
papers
Afreereferencemanagerandacademicsocialnetwork.
OrganicEprints orgprints.org Openaccessarchiveforpapersandprojectsrelatedto
researchinorganicfoodandfarming.
PennState’sCiteSeerX citeseerx.ist.psu.edu Focusedprimarilyontheliteratureincomputerand
informationscience.
PubMed www.ncbi.nlm.nih.gov/pubmed Morethan28millioncitationsforbiomedicalliteraturefrom
MEDLINE,lifesciencejournals,andonlinebooks.
SocialScienceResearch
Network
www.ssrn.com/en SSRN´seLibrary provides0.7millionresearchpapersacross30
disciplines.
U.S.government’sScience.gov www.science.gov Covers60databasesandover2,200scientificwebsitesto
providefullaccesstomorethan200millionpagesofU.S.
federalscienceinformation.
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 4
Pearce, Perform a Literature Review with Free and Open Source Software
content. Scholars without access to pay walled literature
also can:
1) directly request copies of papers from the
authors via email,
2) request from others in the academic community
that do have access using the Twitter hashtag
#icanhazpdf,
3) acquire articles through various torrents, Library
Genesis, openaccessbutton.org, or on the
anonymous website Reddit/r/scholar.
4) attempt interlibrary loans (at some institutions
pdf scans are provided).
Providing legal open access preprints authors can
take advantage of the often dramatic increases in
citations for open access articles (Antelman, 2004;
Harnad & Brody, 2004, Niyazov, et al., 2016). Thus,
there is already an incentive to encourage scholars to
share their work. Online open access journals provide
wide exposure for authors, which are using publishing
there at an increasing rate. For example, researchers
access and download articles from Practical Assessment,
Research & Evaluation (PARE) from all over the world
daily (see data available on http://pareonline.net/). Not
all of the literature, however, is open access nor
accessible in the databases and by the techniques listed
above. In some cases to access seminal papers, which
must be read, reviewed and referenced for a complete
literature review, scholars must gain access through paid
databases. This can be particularly challenging for
researchers not affiliated with a large organization, from
developing countries and those that conduct research at
teaching universities. The latter, for example, often
conduct research on teaching, learning and pedagogy,
but lack access to the full literature through their schools
and dedicated funding for purchasing access.
A neuroscience scholar from Kazakhstan in this
situation without adequate funding, was frustrated with
lack of access and founded Sci-Hub, the world's largest
pirate site for academic papers (at least 50 million)
(Bohannon, 2016). Sci-Hub has been sued a number of
times and continues to change its url. Interestingly,
critics of Sci-Hub in Science have complained that many
users can access the same papers through their libraries
legally, but turn to Sci-Hub for convenience (Bohannon,
2016). Clearly, there is a need for a legal and convenient
method to gain full access to all of the literature for all
scholars. Fortunately, funding agencies are now also
demanding open access posting and there has been a call
for such organizations to begin directly sponsoring and
publishing open access articles (Pearce, 2016). For
example, since 2009, the NIH has demanded that all
research they fund “submit or have submitted for them to the
National Library of Medicine's PubMed Central an electronic
version of their final, peer-reviewed manuscripts upon acceptance for
publication, to be made publicly available no later than 12 months
after the official date of publication.”(NIH, 2018). India’s
(Chawla, 2014), Portugal’s (Carvalho, et al., 2017),
Denmark’s and some of Europe’s (DTU, 2018) major
science funders have done the same. The rational
funding agencies use for this demand is that the public
has already paid for the research through tax funding
should be able to access the results for free. Private
foundations are also moving in this direction of
requiring open access such as the Gates Foundation
(Van Noorden, 2014). Most of the major conventional
publishers also maintain some legal and allowed form of
preprint publishing so it is within reason to assume that
in the near future the majority of literature will be
available open access and discoverable by the databases
shown in Table 1.
Literature Review Steps
Managing the literature can be done completely
electronically using FOSS and will be covered in detail in
the next section. Although the core process is of
performing a literature review this way is the same as the
historical method accomplished largely in hardcopy
format the conventional process is enhanced. With
FOSS literature reviews can be executed faster, be more
complete, portable and potentially collaborative.
This section will briefly cover the steps in
performing a high-quality literature review. First, using
the academic search engines shown in Table 1 the
literature review can occur. For researchers new to a
field, one of the best ways to get started is to find recent
review articles or major studies on the subject of interest.
Next the researcher should read them and then 1) read
the papers in the reference list from the review papers
backwards in time and 2) read articles that cite the
reviews forward in time, to populate their own literature
review. Care should be taken to ensure that any
information taken from review articles is not secondary
(i.e. only primary sources should be cited after having
read them). In addition, these literature review article are
normally out of date (because of the rapid rate of
technical progress) as well as they might have been
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 5
Pearce, Perform a Literature Review with Free and Open Source Software
biased or poorly done. Thus, the researcher can continue
by taking a systematic approach to searching the
literature both backwards and forwards in time. Going
backwards not only should the papers referenced be
reviewed, but the author’s names and the keywords in
those papers can also be used. This can be repeated for
a second level backwards (e.g. looking at the references,
referenced in the papers originally referenced in the
review article) or more depending on the depth
necessary for the review (Levy & Ellis, 2006). Similarly,
going forward if new authors cite the review articles then
individual searches can be run on these authors. To
accelerate the process the corresponding authors names
on a collaborative paper are the most valuable to search
as they normally represent the senior researcher and the
one with the longest literature record.
Although all sources should be covered, researchers
should focus primarily on the peer-reviewed literature.
Often there may be key information that is not available
yet in the peer-reviewed literature (e.g. new commercial
technologies) and such sources can be used, but caution
is necessary to ensure bias is not introduced (and it
should be noted that corporate sponsorship can impact
any literature). Thus, the use of non-peer-reviewed
articles (e.g. technical journals, trade magazines, blogs,
or newspapers) should be restricted to factual
information. In addition, for some fields it may also be
important to search through the patent databases (e.g.
USPTO patent database, Espacenet, or Google Patents).
In addition, researchers may want to limit their searches
to an open source database and website that provides
only inactive patents, which are in the public domain
(Nilsiam & Pearce, 2016). This type of search can be
accessed at http://freeip.mtu.edu/ made available by
Michigan Technological University. Researchers must be
extremely careful with the use of patents in literature
reviews, as there is both a clear economic conflict of
interest and patent applications do not go through the
rigor of peer review. They are essentially legal documents
made up of claims reviewed by the patent office for
novelty and non-obviousness. Thus, all the writing in a
patent that is not a claim does not necessarily have
academic merit (e.g. similar to a blog post, anyone can
write anything in the text of a patent). Literature of
quality levels in between these two types of sources such
as non-peer reviewed conference proceedings can also
be used with caution with the reviewers themselves
needing to judge carefully the quality and reliability of
the work. Finally, all sources including those published
in the peer-reviewed literature should be critically
analyzed before accepting another researcher’s findings
and interpretations as valid to be included in the review.
If errors are made at this stage of gathering the
relevant literature with the choice of keywords the
results can be of poor quality or irrelevant. Thus,
researchers should strive to use as many synonyms and
alternative ways of expressing keyword or key phrases
when using the search engines in Table 1. It is also useful
to search with quotation marks around short phrases to
ensure the most relevant literature is provided first. In
general, broader and more generalizable terms will
capture a wider range of the literature and should be used
initially. In addition, care must be taken to look beyond
temporarily popular buzzwords that have a short lifetime
in the literature. The theories underlying the buzzwords
have a longer life (Robey et al. 2000). Gathering all of
the previous literature relevant to a specific area of
scholarship is challenging. To determine if the literature
is relevant, the entire electronic record does not need to
be initially read. Randolph recommends eliminating
most likely irrelevant studies using just the title and
abstract (2009). Finally, the end of the search is indicated
when additional searches provide no new citations and
articles cited in the most-recently-discovered literature
have already been included in the review.
After all of the literature has been found, it must be
read and the data contained in the articles processed into
information that can serve as a foundation for new
research (Bem, 1995). Following the guidelines for
organizing the literature review on a wiki (detailed
below) entails making a bullet point notes of useful
information for the project from each article included in
the review. The bullet list can include any information
that either summarizes the article or is useful for the
researcher. As this information is being posted on the
open web, it is recommended that incomplete sentences
be used to record the thoughts while protecting against
self-plagiarism when using the thoughts in writing a new
manuscript as many publishers now use automatic
plagiarism detection software.
FOSS Tool Chain for Literature Reviews
In order to reduce the time and economic
investment needed to do a modern digital literature
review all proprietary (closed source) and costly software
can be avoided. A complete FOSS tool chain can be used
for creating literature reviews in which case the
researchers maintain complete control over their data
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 6
Pearce, Perform a Literature Review with Free and Open Source Software
and always have access to the software needed to access,
alter or build upon it. This FOSS software has zero
capital costs and is made up of: 1) an operating system,
2) web browser, 3) reference organizer/database, 4)
open access enabler software, 5) text editor/typesetter,
and 6) sharing and collaboration software.
The most popular type of FOSS operating system is
the GNU Linux operating systems (Stallman, 1997),
which support all architectures, are highly reliable and
have a zero cost. There are more than 100 distributions,
with the most popular being Mint, Manjaro, Ubuntu,
Debian, Solus, Elementary, Antergos and Fedora
(Distrowatch, 2018), all of which can be downloaded
and installed for free on existing computers of any type.
Linux distributions generally come with a FOSS
web browser and the most popular is Firefox (Devčić,
2015). Firefox (Mozilla, 2018) has two available plugins,
which are particularly useful for literature reviews. First,
unpaywall (2018) is a Firefox plugin that enables
researcher to click a green tab on the side of the browser
and skip the paywall on millions of peer-reviewed
journal articles. This makes finding accessible copies of
articles much more rapid that searching each database
individually. Unpaywall is fast, free, and legal as it
accesses many of the open access sites that are listed in
Table 1. Second, Zotero (2018) operates as an Android
App (and iPad/iPhone app), desktop program and a
Firefox plugin. It is a free, easy-to-use tool to help
researchers collect, organize, cite, and share research. It
replaces the functionality of proprietary packages such
as RefWorks, Endnote and Papers for zero cost. Thus,
Zotero can auto-add bibliographic information directly
from websites, or publishers pages. In addition, it can
scrape bibliographic data off of pdf files. Notes can be
easily added on each reference. Then finally, it can
import and export the bibliography databases in all of
the publishers’ various formats. Researchers can thus
export bibliographic information to paste into a
document editor for a paper or thesis as well as wiki for
dynamic collaborative literature reviews (discussed in the
next section).
Finally, academic articles can be written
conventionally with the free office suite LibreOffice
(2018), which operates similarly to Microsoft’s Office
products. Zotero has a word processor plugin to
integrate directly with LibreOffice. LibreOffice is more
than adequate for the vast majority of academic paper
writing. In addition, researchers can take the writing of
their papers one step further with LaTeX (2018), a high-
quality typesetting system specifically designed for the
production of technical and scientific documentation.
Zotero libraries can be directly exported to BibTeX files
for use with LaTeX.
Lastly, a wiki can be used to facilitate a dynamic
collaborative literature review. A wiki is a website that
allows anyone to add, delete, or revise content directly
using a web browser. MediaWiki (2018a) is a free
software open source wiki package written in open
source PHP (2018). MediaWiki is available for anyone to
set up their own wikis. Researchers can (in decreasing
order of complexity) 1) set up their own research group
wiki with MediaWiki (2018b), 2) utilize wikis already
established at their universities (e.g. wiki.aalto.fi at Aalto
University in Finland or ist.mit.edu/wikis at MIT in the
U.S.) or 3) use wikis dedicated to areas that they research.
For example, several university research groups that
focus on sustainability research use Appropedia (2018),
which is setup for collaborative solutions in
sustainability, appropriate technology, poverty
reduction, and permaculture. Utilizing a wiki makes it
easy for anyone in the group to keep track of the status
of and update literature reviews both current and from
prior years or researchers. It also easily enables multiple
members of the group to collaborate on the literature
review asynchronously. Most importantly, it also
facilitates others outside of the research group to assist
in making a literature review more complete, accurate
and up-to-date.
Thus, the entire tool chain for literature reviews can
be covered with the FOSS shown in Table 2.
Table 2. Free and open source software useful for
literature reviews
FreeandOpen
Source
Software
DownloadURL
GNULinux
Distributions
https://distrowatch.com/
FireFox https://www.mozilla.org/en‐
US/firefox/new/
Unpaywall https://unpaywall.org/
Zotero https://www.zotero.org/
LibreOffice https://www.libreoffice.org/
LaTeX https://www.latex‐project.org/
MediaWiki https://www.mediawiki.org/wiki/MediaWiki
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 7
Pearce, Perform a Literature Review with Free and Open Source Software
Encouraging Collaboration With Dynamic Wiki-
Based Literature Reviews
Historically performing a literature review has been
a solitary process, where a single researcher scours
familiar databases and resources to maintain a list of
references and perhaps comments in a single document.
This has some obvious flaws including: i) no individual
is intimately familiar with all of the areas of inquiry that
may pertain to a topic, nor do they have access to all of
the literature, and ii) previously mentioned fracturing of
the literature and inability of any search engine to
comprehensively search the literature easily. An obvious
solution to such problems is to encourage other
researchers to collaborate on relevant literature reviews.
This can be done by using an open source approach by
developing literature reviews on wikis.
Wikis are well known to foster collaboration in
general (Leuf & Cunningham, 2001; McAfee, 2006;
Wagner, 2004) and in education in particular
(Abdekhodaee, et al., 2017; Biasutti, 2017; Cilliers, L.
2017; Cole, 2009; Pearce, 2009; Wang & Turner, 2004).
Wikis are easy to edit with a relatively shallow learning
curve (e.g. a typical research student can master the
basics of wiki markup in under 30 minutes), which
enables new researchers to quickly begin making real
contributions to the effort of the group. In past
experiments using wikis for literature reviews (Pearce,
2012), multiple researchers were able to edit and thus
improve a literature review at the same time or from
different locations (e.g. different labs and offices). The
collaborative advantages to this were useful as
researchers could telecommute (e.g. work from home in
a remote community or while traveling) and thus reduce
commuting time (Denkenberger, et al., 2015) while still
contributing as full members of the group. In addition,
the use of an open edit wiki encourages others to assist
in the research outside of the group. Examples routinely
observed included of assistance from wiki users not
affiliated with the research group: i) giving helpful
comments on group research on the discussion tab of
pages, ii) making grammar and spelling corrections, iii)
adding content to the group’s literature reviews, iv)
adding categories and hyperlinking either to or within
work that are group has made, which adds to the value
and discover-ability of the work (Pearce, 2012). Open
literature reviews arranged in this way are extremely
effective for passing on knowledge to the next
generation of student researchers, keeping up to date on
background research and finding references to read
when writing a paper. However, it should be noted that
publishing a literature review on a wiki does not give
academics (who need peer reviewed articles for tenure)
credit for doing the review. The wiki is merely a means
of facilitating collaboration towards a normally
published review. For example, Figure 1 and 2 shows
several screenshots of a wiki literature review for an
economic review paper on the levelized cost of
electricity of solar cells. Figure 1 shows a typical
literature review page for a wiki. The FOSS wiki software
automatically generates a table of contents when wiki
markup is used. The webpage can be edited by clicking
the “Edit” hyperlink directly in the web browser. Best
practices encourages other researchers to assist with the
literature review. The goals and basic definitions of the
literature review are explained in the beginning and an
entry is made for each article reviewed.
Note that in Figure 2, each reviewed article was
placed in its own level thus both enabling a wiki
generated table of contents for the top of the page, but
also an individual “edit” hyperlink for each entry. The
latter makes it easy for others to add short notes as bullet
points. The entry includes full bibliographic detail along
with a bulleted list of useful information found in the
article. Bullet point summaries avoid self-plagiarism
when writing the full peer-reviewed article.
The wiki example shown in Figures 1 and 2 was
accessed tens of thousands of times directly on the
website and was edited by researchers outside of the
founding pages research group. This improved the
quality of the literature review and it became a major
contribution to the field. According to Google Scholar,
the resultant peer-reviewed article generated from the
literature review has been cited over 800 times. In
addition, this dynamic literature review continues to be
improved so that the next time a researcher needs a
review of the subject it will provide a valuable tool for
extending the work further.
Although using a dedicated group wiki provides
more control over it, using an established wiki to host a
group’s literature reviews has several other advantages.
First, it is easy to get started as creating a free account is
less technically challenging than setting up a wiki on a
researcher-owned or subscription-based server. Second,
using an existing wiki provides immediate exposure to
the existing wiki community as well as high rankings for
search results. For example, as Appropedia is currently
the largest wiki dedicated to appropriate technology and
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 8
Pearce, Perform a Literature Review with Free and Open Source Software
Figure 1. Typical literature review page on a wiki showing the top and table of contents.
Figure 2. An example short entry for an article on a wiki.
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 9
Pearce, Perform a Literature Review with Free and Open Source Software
sustainability, the research group’s that use it benefit
from existing widespread global readership. This
exposure can directly lead to funding opportunities
(Pearce et al., 2012). In addition, to targeted wikis on a
specific research topic, there are also generic open
source research wikis that can provide the needs for
nearly any discipline such as the semantic wiki for the
sciences Open Research (2018).
Benefits to the Open Source Approach
The open source approach to developing literature
reviews shown here has several benefits. First it reduces
investment both in terms of researcher time and
economic costs (both capital and operations). By using a
FOSS tool chain the costs of proprietary software are
eliminated, while at the same time using FOSS ensures
that the literature databases created are not subject to
changes in licensing agreements or discontinuing of
software or support from a proprietary business. The
risk of this occurring can be significant and FOSS
protects researchers from this loss for any reason
including bankruptcy, business decisions, sales or
mergers, or the loss of key technical staff from
proprietary vendors. Researchers can store a copy of the
code themselves (or access it from freely available 3rd
party repositories like GitHub, Bitbucket, Lauchpad,
SourceForge, GitLab, GNU Savannah or OSDN). Thus,
researchers have intellectual property control over the
software used in every stage of their literature review as
well as the database and output and can thus either solve
any future problem with the FOSS themselves or pay
others to do it. Researchers simply do not have these
freedoms with proprietary software. Using, the list of
freely accessible search engines and repositories as well
as legal methods to access the peer-reviewed literature
discussed above can save researchers significant time
while also ensuring a comprehensive review of the
literature. Finally, and perhaps most importantly,
maintaining literature reviews in an open edit wiki
enables others in the field to collaboratively improve
them for both contemporary as well as future scholars.
Overcoming Problems with This Approach
This approach has numerous advantages besides
low costs as discussed above. However, this approach
also comes with challenges. This section will review
challenges for each part of the open source approach
and discuss method to overcome them.
Technical Computer Competency
Not all researchers are comfortable setting up Linux
on their computers because of the technical challenges.
First, it should be pointed out that this has become
significantly easier over the last few years and that most
computer users would be comfortable with the vast
majority of the use of one of the easier distributions like
Ubuntu. Second, most popular Linux distributions come
with a ‘try-before-you-buy’ feature. For example,
Ubuntu can be used from a bootable USB stick (Ubuntu,
2017). The USB stick allows researchers to test run the
Ubuntu desktop experience without interfering in any
way with their PC configuration. If the researcher likes
the experience then the stick can be used to install
Ubuntu on to their machine permanently. Researchers
that previously purchased proprietary software they
need (e.g. a simulation package, graphing package, etc.)
can see if there is an open source alternative, using for
example, osalt.com. If there is not an adequate
alternative then it is possible to run the applications in
Linux using either a remote Windows system, a virtual
machine or Wine (2018). Wine, a recursive acronym for
“Wine Is Not an Emulator”, is a compatibility layer
capable of running Windows applications on Linux.
Even with these options moving to an open source
operating system may be too difficult for some
researchers. For them, a dual boot system that can use
both Linux and Windows is recommended. Finally, it is
not imperative to use Linux in order to be able to take
advantage of the remainder of the software listed in
Table 2. Thus, for example, Firefox and Zotero can be
installed on a Windows machine. Installing and using all
the remainder of the software is straightforward and
researchers can get the help of their organization’s IT
staff for assistance. Because all of the software listed in
Table 2 is zero cost, it should not represent a significant
barrier for IT to install it on all of the computers at a
university, nonprofit or company.
One Stop for Comprehensive Search and Access
Although the free (zero cost) academic search engines
for published papers and preprints social networks have
expanded significantly (Table 1), there is still a need for
a one stop location for comprehensive academic article
searching and open access finding. Håklev (2013) has
argued that there is still a need for an open alternative to
Google Scholar, although this does not preclude the
current use of the many databases in Table 1. Future
work is needed to reduce the time expenditure to utilize
all of these resources.
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 10
Pearce, Perform a Literature Review with Free and Open Source Software
Vandalism
A common fear among researchers about putting their
work on an open wiki that anyone can edit is that it
would be vandalized either maliciously or simply
corrupted by well-intentioned users that alter their
writing. Fortunately, there are several ways around this
potential problem. First spam and actual vandalism on
the major wikis, are normally removed by the vigilant
team of administrators for the wiki without any effort
from researchers. For smaller wikis (e.g. a research
group’s own wiki with only modest internet traffic) this
is less of a problem because there is less of an incentive
for bot-based advertising-funded spamming that makes
up the vast majority of vandalism. Thus, small wikis can
be passively defended. Researchers can add their
literature review pages to their watchlists so that they can
choose to be notified if it is edited by someone other
than themselves. Vandalism can be easily removed with
1 or 2 clicks of “undo” from the standard revision “view
history” menu of any Wikimedia-based page.
Critical Mass for Collaboration on Literature
Reviews
To have the free user to user assistance seen in the
success of FOSS (Lakhani & Von Hippel, 2003) there
must be a critical mass of users. Thus, the primary
challenge with the wiki collaboration approach is there
must be a critical mass or other researchers external to a
lab that use the same approach (e.g. are willing to
contribute to a shared literature review) to obtain the full
list of benefits. In a vibrant and well-populated open
research community, which has reached critical mass on
any research topic, maintaining a literature review would
become a community affair and not the primary work of
a single group. If the majority of researchers simply
added their own work in order to garner more attention
to it, this approach would be significantly improved.
Future work is needed to determine the most effective
methods for encouraging an entire research community
to adopt the open source approach.
Conclusions
This paper has shown best practices for using a tool
chain of free and open source software and methods to
perform literature reviews. Resources for
comprehensive searching and obtaining access to the
literature were provided that enable researchers from
underfunded labs and countries with low-levels of
scientific financing to participate in the scientific
enterprise. Free and open source software, was provided
for all steps including browsing, bibliographic software,
and writing. Finally, a clear framework was provided for
documenting a literature review to encourage
collaboration of a dynamic document that lives into the
future for the benefit of all researchers in a given field.
This approach was shown how to solve the current
challenges of literature reviews in general and reduce
time expenditures and costs. Finally, the challenges of
using this approach are discussed and either overcome
with existing methods or future work is provided. It is
concluded, that although there are many types and goals
of literature reviews, all of them can be improved using
a tool chain of free and open source software and
methods.
References
Abdekhodaee, A., Chase, A. M., & Ross, B. (2017). Wikis for
group work: Encouraging transparency, benchmarking,
and feedback. Australasian Journal of Educational
Technology, 33(5).
Alexy, O., Henkel, J., & Wallin, M. W. (2013). From closed
to open: Job role changes, individual predispositions,
and the adoption of commercial open source software
development. Research Policy, 42(8), 1325-1340.
Antelman, K. (2004). Do open-access articles have a greater
research impact?. College & Research Libraries, 65(5), 372-
382.
Appropedia. (2018). Retrieved April 5, 2018, from
http://www.appropedia.org/Welcome_to_Appropedia
Årdal, C., & Røttingen, J. A. (2016). Financing and
collaboration on research and development for
nodding syndrome. Health Research Policy and Systems,
14(1), 19.
Baden, T., Chagas, A. M., Gage, G., Marzullo, T., Prieto-
Godino, L. L., & Euler, T. (2015). Open Labware: 3-D
printing your own lab equipment. PLoS Biology, 13(3),
e1002086.
Devčić, I. I. (2015, March 30). Top 10 Best Web Browsers
For Linux. Retrieved April 6, 2018, from
https://beebom.com/best-web-browsers-for-linux/
Bem, D. J. (1995). Writing a review article for psychological
bulletin. Psychological Bulletin, 118(2), 172-177.
Bettany-Saltikov, J. (2012). How to do a systematic literature
review in nursing: a step-by-step guide. McGraw-Hill
Education (UK).
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 11
Pearce, Perform a Literature Review with Free and Open Source Software
Biasutti, M. (2017). A comparative analysis of forums and
wikis as tools for online collaborative learning.
Computers & Education, 111, 158-171.
Bohannon, J. (2016). Who's downloading pirated papers?
Everyone. Science, 352(6285), 508-512.
Boldrin, M., & Levine, D. K. (2008). Against intellectual
monopoly (Vol. 78). Cambridge: Cambridge University
Press.
Boote, D. N., & Beile, P. (2005). Scholars before
researchers: On the centrality of the dissertation
literature review in research preparation. Educational
Researcher, 34(6), 3-15.
Boudreau, K. J., & Lakhani, K. R. (2016). Innovation
Experiments: Researching Technical Advance,
Knowledge Production, and the Design of Supporting
Institutions. Innovation Policy and the Economy, 16(1), 135-
167.
Budgen, D., & Brereton, P. (2006). Performing systematic
literature reviews in software engineering. In
Proceedings of the 28th international conference on
Software engineering (pp. 1051-1052). ACM.
Carvalho, J., Laranjeira, C., Vaz, V., & Moreira, J. M. (2017).
Monitoring a national open access funder mandate.
Procedia Computer Science, 106, 283-290.
Chawla, D.M. (2014) India’s major science funders join
open-access push. Science. Retrieved April 6, 2018,
from
http://www.sciencemag.org/news/2014/12/india-s-
major-science-funders-join-open-access-push
Cilliers, L. (2017). Wiki acceptance by university students to
improve collaboration in Higher Education. Innovations
in Education and Teaching International, 54(5), 485-493.
Coakley, M., & Hurt, D. E. (2016). 3D Printing in the
laboratory: maximize time and funds with customized
and open-source labware. Journal of Laboratory
Automation, 21(4), 489-495.
Cole, M. (2009). Using Wiki technology to support student
engagement: Lessons from the trenches. Computers &
Education, 52(1), 141-146.
Colombo, M. G., Piva, E., & Rossi-Lamastra, C. (2014).
Open innovation and within-industry diversification in
small and medium enterprises: The case of open source
software firms. Research Policy, 43(5), 891-902.
Cooper, H. M. (1988a). The integrative research review: A
systematic approach (No. 2). Sage Publications.
Cooper, H. M. (1988b). Organizing knowledge syntheses: A
taxonomy of literature reviews. Knowledge in Society, 1(1),
104.
Deek, F. P., & McHugh, J. A. (2007). Open source:
Technology and policy. Cambridge University Press.
Denkenberger, D., Way, J. and Pearce, J.M. Educational
pathways to remote employment in isolated
communities. Journal of Human Security 11(1) (2015): 34.
DiBona, C., & Ockman, S. (1999). Open sources: Voices
from the open source revolution. " O'Reilly Media,
Inc.".
Distrowatch. (2018) DistroWatch.com: Put the fun back
into computing. Use Linux, BSD. Retrieved April 6,
2018, from https://distrowatch.com/
Dodourova, M., & Bevis, K. (2014). Networking innovation
in the European car industry: Does the Open
Innovation model fit?. Transportation Research Part A:
Policy and Practice, 69, 252-271.
DTU (2018). Requirements from funders concerning Open
Access publishing - DTU Library. Retrieved April 6,
2018, from
http://www.bibliotek.dtu.dk/english/servicemenu/pu
blish/openaccess/Requirements
Harnad, S., & Brody, T. (2004). Comparing the impact of
open access (OA) vs. non-OA articles in the same
journals. D-lib Magazine, 10(6).
Hart, C. (1998). Doing a literature review: Releasing the social
science research imagination. Sage.
Harzing, A. (2017). Google Scholar is a serious alternative to
Web of Science. Retrieved April 6, 2018, from
https://harzing.com/blog/2017/02/google-scholar-is-
a-serious-alternative-to-web-of-science
Håklev, S. (2013). An open alternative to Google Scholar.
Retrieved April 5, 2018, from
https://www.force11.org/presentation/open-
alternative-google-scholar
Hawkins, R. E. (2004). The economics of open source
software for a competitive firm. NETNOMICS:
Economic Research and Electronic Networking, 6(2), 103-117.
Henkel, J., Schöberl, S., & Alexy, O. (2014). The emergence
of openness: How and why firms adopt selective
revealing in open innovation. Research Policy, 43(5), 879-
890.
Hienerth, C., Von Hippel, E., & Jensen, M. B. (2014). User
community vs. producer innovation development
efficiency: A first empirical study. Research Policy, 43(1),
190-201.
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 12
Pearce, Perform a Literature Review with Free and Open Source Software
Jesson, J., & Lacey, F. (2006). How to do (or not to do) a
critical literature review. Pharmacy Education, 6.
Lakhani, K. R., & Von Hippel, E. (2003). How open source
software works:“free” user-to-user assistance. Research
Policy, 32(6), 923-943.
LaTeX (2018) LaTeX - A document preparation system.
Retrieved April 6, 2018, from https://www.latex-
project.org/
Leuf, B., & Cunningham, W. (2001). The Wiki way: quick
collaboration on the Web.
Levy, Y., & Ellis, T. J. (2006). A systems approach to
conduct an effective literature review in support of
information systems research. Informing Science, 9.
Lewis, D. W. (2012). The inevitability of open access. College
& Research Libraries, 73(5), 493-506.
LibreOffice (2018). Home | LibreOffice - Free Office Suite
- Fun Project - Fantastic People. Retrieved April 6,
2018, from https://www.libreoffice.org/
Martinez, M. G. (2015). Solver engagement in knowledge
sharing in crowdsourcing communities: Exploring the
link to creativity. Research Policy, 44(8), 1419-1430.
McAfee, A. P. (2006). Enterprise 2.0: The dawn of emergent
collaboration. MIT Sloan Management Review, 47(3), 21.
MediaWiki. (2018a). Retrieved April 5, 2018, from
https://www.mediawiki.org/wiki/MediaWiki
MediaWiki. (2018b) Manual:Installation guide - MediaWiki.
(n.d.). Retrieved April 6, 2018, from
https://www.mediawiki.org/wiki/Manual:Installation_
guide
Mozilla (2018) Download the fastest Firefox ever. Retrieved
April 6, 2018, from https://www.mozilla.org/en-
US/firefox/new/
NIH (2018). NIH Public Access Policy Details |
publicaccess.nih.gov. Retrieved April 6, 2018, from
https://publicaccess.nih.gov/policy.htm
Nilsiam, Y., & Pearce, J. M. (2016). Open source database
and website to provide free and open access to inactive
US patents in the public domain. Inventions, 1(4), 24.
doi:10.3390/inventions1040024
Niyazov, Y., Vogel, C., Price, R., Lund, B., Judd, D., Akil,
A., ... & Shron, M. (2016). Open access meets
discoverability: Citations to articles posted to
Academia. edu. PloS One, 11(2), e0148257.
OpenResearch. (2018). Retrieved April 6, 2018, from
http://openresearch.org/Main_Page
Pearce, J. M. (2009). Appropedia as a tool for service
learning in sustainable development. Journal of Education
for Sustainable Development, 3(1), 45-53.
Pearce, J. M. (2012). Open source research in sustainability.
Sustainability: The Journal of Record, 5(4), 238-243.
Pearce, J. M., Babasola, A., & Andrews, R. (2012). Open
solar photovoltaic systems optimization. In Proceedings of
Open, the Annual National Collegiate Inventors & Innovators
Alliance Conference.
Pearce, J. M. (2013). Open-source lab: how to build your own
hardware and reduce research costs. Elsevier.
Pearce, J.M. (2015a) Quantifying the Value of Open Source
Hardware Development. Modern Economy, 6, 1-11. doi:
10.4236/me.2015.61001
Pearce, J. M. (2015b). Return on investment for open source
scientific hardware development. Science and Public Policy,
43(2), 192-195.
Pearce, J. M. (2016). Funding Universal Open Access via
Academic Efficiency Gains from Government Funder
Sponsored Open Access Journals. Ariadne, (76).
http://www.ariadne.ac.uk/issue/76/pearce
Pearce, J.M., (2017). Emerging Business Models for Open
Source Hardware. Journal of Open Hardware. 1(1), p.2.
DOI: http://doi.org/10.5334/joh.4
Petch, A., Lightowler, C., Pattoni, L., & Watson, I. (2014).
Embedding research into practice through innovation
and creativity: a case study from social services.
Evidence & Policy: A Journal of Research, Debate and
Practice, 10(4), 555-564.
Petersen, E. E., & Pearce, J. (2017). Emergence of home
manufacturing in the developed world: Return on
investment for open-source 3-D printers. Technologies,
5(1), 7. doi:10.3390/technologies5010007
Petersen, E. E., Kidd, R. W., & Pearce, J. M. (2017). Impact
of DIY Home Manufacturing with 3D Printing on the
Toy and Game Market. Technologies, 5(3), 45; doi:
10.3390/technologies5030045
PHP: Hypertext Preprocessor. (2018). Retrieved April 5,
2018, from http://www.php.net/
Piwowar, H., Priem, J., Larivière, V., Alperin, J. P., Matthias,
L., Norlander, B., … Haustein, S. (2017). The State of
OA: A large-scale analysis of the prevalence and impact
of Open Access articles (No. e3119v1). PeerJ Inc.
https://doi.org/10.7287/peerj.preprints.3119v1
Randolph, J. J. (2009). A guide to writing the dissertation
literature review. Practical Assessment, Research &
Evaluation, 14(13), 1-13.
Practical Assessment, Research & Evaluation, Vol 23 No 8 Page 13
Pearce, Perform a Literature Review with Free and Open Source Software
Raymond, E. (1999). The cathedral and the bazaar. Knowledge,
Technology & Policy, 12(3), 23-49.
Riehle, D. (2007). The economic motivation of open source
software: Stakeholder perspectives. Computer, 40(4).
Robey, D., Boudreau, M. C., & Rose, G. M. (2000).
Information technology and organizational learning: a
review and assessment of research. Accounting,
Management and Information Technologies, 10(2), 125-
155.
Söderberg, J. (2015). Hacking capitalism: The free and open source
software movement (Vol. 9). Routledge.
Sample, I. (2012). Harvard University says it can’t afford
journal publishers’ prices. The Guardian, 24, 2012.
Shaw, D. (1995). Bibliographic database searching by
graduate students in language and literature: Search
strategies, system interfaces, and relevance judgments.
Library & Information Science Research, 17(4), 327-345.
Stallman, R. (1997). Linux and the GNU Project. Online
Document, 2006. gnu.org. Retrieved April 6, 2018,
from https://www.gnu.org/gnu/linux-and-gnu.en.html
Ubuntu (2017, July 18). Create a bootable USB stick on
Windows | Ubuntu tutorials. Retrieved April 6, 2018,
from https://tutorials.ubuntu.com/tutorial/tutorial-
create-a-usb-stick-on-windows
Unpaywall. (2018) Unpaywall. Retrieved April 6, 2018, from
https://unpaywall.org/
Van Noorden, R. (2014). Gates Foundation announces
world’s strongest policy on open access research:
Nature News blog. Retrieved April 6, 2018, from
http://blogs.nature.com/news/2014/11/gates-
foundation-announces-worlds-strongest-policy-on-
open-access-research.html
Wagner, C. (2004). Wiki: A technology for conversational
knowledge management and group collaboration. The
Communications of the Association for Information Systems,
13(1), 58.
Wang, C. M., & Turner, D. (2004). Extending the wiki
paradigm for use in the classroom. In Information
Technology: Coding and Computing, 2004. Proceedings (Vol. 1,
pp. 255-259). IEEE.
Webster, J., & Watson, R. T. (2002). Analyzing the past to
prepare for the future: Writing a literature review. MIS
Quarterly, xiii-xxiii.
WineHQ - Run Windows applications on Linux, BSD,
Solaris and macOS. (2018). Retrieved April 23, 2018,
from https://www.winehq.org/
Wittbrodt, B. T., Glover, A. G., Laureto, J., Anzalone, G. C.,
Oppliger, D., Irwin, J. L., & Pearce, J. M. (2013). Life-
cycle economic analysis of distributed manufacturing
with open-source 3-D printers. Mechatronics, 23(6), 713-
726.
Woern, A. L., & Pearce, J. M. (2017). Distributed
Manufacturing of Flexible Products: Technical
Feasibility and Economic Viability. Technologies, 5(4), 71.
Zhu, K. X., & Zhou, Z. Z. (2012). Research note—Lock-in
strategy in software competition: Open-source software
vs. proprietary software. Information Systems Research,
23(2), 536-545.
Zotero (2018). Zotero | Your personal research assistant.
Retrieved April 6, 2018, from https://www.zotero.org/
Citation:
Pearce, Joshua M. (2018). How to Perform a Literature Review with Free and Open Source Software. Practical
Assessment, Research & Evaluation, 23(9). Available online: http://pareonline.net/getvn.asp?v=23&n=8
Acknowledgment
This work was supported by Fulbright Finland.
Corresponding Author
Joshua M. Pearce, Professor
Department of Electronics & Nanoengineering, Aalto University, P.O. Box 13500, FI-00076 AALTO, Finland
Department of Materials Science & Engineering and Department of Electrical & Computer Engineering,
Michigan Technological University, 1400 Townsend Drive, Houghton, MI 49931-1295
email: pearce [at] mtu.edu
... A literature review is a detailed synthesis and critical examination of all relevant research literature on the issue under consideration. Any successful literature review should methodologically analyse and synthesise high-quality literature, offer a foundation for a study subject and technique, and show that the proposed research will contribute to the field's knowledge base [1,5]. ...
... A literature review is essential for defining research issues, avoiding ineffective approaches, providing recommendations for additional research, and developing researcher skills and analytical procedures for the study [4,7]. Furthermore, a literature review educates researchers about important scholars and research organisations and defines the researcher's study aim [1,4]. ...
... Some academics have problems accessing the required material, making conducting a complete literature study more difficult. Additional issues include a lack of access to all databases, poor search engine quality, and difficulty receiving reliable results [1,9]. Copyright limitations and subscription fees have unexpected effects limiting access to peer-reviewed material and negatively impacting effective literature reviews [10]. ...
Article
Full-text available
There is an inequality in developed and developing countries' research output in highly reputable databases. One way to reduce this inequality is to encourage researchers in developing countries to use online digital tools. This article examines the contribution of lecturers' awareness and knowledge to utilising the free online digital tools (FODT) available for literature review in the field of education. A correlation research approach involving 180 academic staff in tertiary institutions in southern Nigeria was adopted, using descriptive statistics and regression analysis. A structured questionnaire elicited the participants' awareness, knowledge, and utilisation of the free online digital tools available for literature review. The findings indicate a low level of awareness, knowledge, and utilisation of the FODT. The two regression models revealed that the lecturers' level of awareness and knowledge accounted for significant contributions to the level of the lecturers' utilisation of the FODT. These results demonstrate a link between lecturers' level of awareness and knowledge to their utilisation of the FODT. It was recommended that stakeholders in research and education create awareness of the availability of these FODTs, carry out workshops on how to use them and replicate this study in other developing countries.
... While many researchers may benefit from national or institutional subscriptions to these services, others may not. As a potential alternative, Pearce (2018) proposed a set of free open-source software (FOSS), including an elaboration on how they can be combined to perform an SLR. He also highlighted that both free and proprietary solutions have advantages and disadvantages that are worthwhile for those who do not have the required tools provided by their employers or other institutions they are members of. ...
... He also highlighted that both free and proprietary solutions have advantages and disadvantages that are worthwhile for those who do not have the required tools provided by their employers or other institutions they are members of. The same may apply to the literature databases used for the literature acquisition in Decision 5 (Pearce 2018). ...
Article
Full-text available
Systematic literature reviews (SLRs) have become a standard tool in many fields of management research but are often considerably less stringently presented than other pieces of research. The resulting lack of replicability of the research and conclusions has spurred a vital debate on the SLR process, but related guidance is scattered across a number of core references and is overly centered on the design and conduct of the SLR, while failing to guide researchers in crafting and presenting their findings in an impactful way. This paper offers an integrative review of the widely applied and most recent SLR guidelines in the management domain. The paper adopts a well-established six-step SLR process and refines it by sub-dividing the steps into 14 distinct decisions: (1) from the research question, via (2) characteristics of the primary studies, (3) to retrieving a sample of relevant literature, which is then (4) selected and (5) synthesized so that, finally (6), the results can be reported. Guided by these steps and decisions, prior SLR guidelines are critically reviewed, gaps are identified, and a synthesis is offered. This synthesis elaborates mainly on the gaps while pointing the reader toward the available guidelines. The paper thereby avoids reproducing existing guidance but critically enriches it. The 6 steps and 14 decisions provide methodological, theoretical, and practical guidelines along the SLR process, exemplifying them via best-practice examples and revealing their temporal sequence and main interrelations. The paper guides researchers in the process of designing, executing, and publishing a theory-based and impact-oriented SLR.
... Although all sources can be covered for review, researchers should give emphasis to peer-reviewed literature. (Pearce, 2018) "There may be important material that is not currently available in peer-reviewed journals (for example, new commercial technologies), and such sources can be utilized, although attention is required to avoid bias (and it should be noted that corporate sponsorship can impact any literature). As a result, non-peer-reviewed articles (such as those found in technical journals, trade magazines, blogs, or newspapers) should be avoided." ...
... As a result, non-peer-reviewed articles (such as those found in technical journals, trade magazines, blogs, or newspapers) should be avoided." (Pearce, 2018) Furthermore, in some fields, searching through patent databases may be necessary. Researcher must manage whole literature electronically using free and open source software like Zotero and mendley etc. ...
Article
This study explored best practices for discovering literature for the review of the research work. The paper discussed various difficulties faced by the investigator during literature search from popular search engines and proprietary databases. This paper presented the whole method of performing literature search and organizing it for future use by using free and open source web tools. Dos and don’ts for literature review has been thoroughly presented to help the researchers to adopt a best practice during their research process. Finally the paper proposed various new web tools such as Citation Gecko, Inciteful and Open Knowledge maps to overcome those difficulties while doing literature reviews. Each of these three tools were explored, tested and a thorough demonstration has been presented in the paper. Citation Gecko can start to search with a few “seed papers” (5 or 6), then pulls in all of their references and even worldwide citations from Crossref and Open Citations to populate the citation network. Paper Discovery service Inciteful starts working from a single relevant paper then creates a network of articles from citations, analyses the network using network analysis methods, and provides you with the information you need to quickly catch up on that topic. Using typical search engines researchers have to sort and organize the research based on different sub areas which is a tedious task. The web tool named Open Knowledge empowered with artificial intelligence solves this problem by creating maps of the literature by grouping and separating them according to their similarity and differences.
... Furthermore, when analyzing other systematic reviews, it is noticeable that other software and platforms, in addition to those employed in the present review, have been used by authors to organize and facilitate data processing. Some examples are: MAXQDA, CiteSpace, Publish or Perish, San-keyMATIc, Bibexcel, Gephi, Pajek, Ucinet, and Science of Science (Sci2) [47][48][49]. ...
Article
Full-text available
The study and application of biological knowledge favor the creation of innovative projects in several areas, so it is necessary to better understand the use of these resources specifically in the field of design. Thus, a systematic review was undertaken to identify, describe, and analyze the contributions of biomimicry to design. For this purpose, the integrative systematic review model, called the Theory of Consolidated Meta-Analytical Approach, was used, carrying out a search on the Web of Science with the descriptors “design” and “biomimicry”. For the period from 1991 to 2021, 196 publications were retrieved. The results were organized according to areas of knowledge, countries, journals, institutions, authors, and years. Citation, co-citation, and bibliographic coupling analyses were also performed. The investigation highlighted the following research emphases: the conception of products, buildings, and environments; the exploration of natural structures and systems to create materials and technologies; the use of biomimetic creative tools in product design; and projects focused on saving resources and implementing sustainability. It was noted that there was a tendency for authors to adopt a problem-based approach. It was concluded that the study of biomimicry can stimulate the development of multiple skills in design, improving creativity, and enhancing the potential integration of sustainability into production cycles.
... The benefits of open access at any scale are well established in the literature and come in two main categories. First, OA publishing is a benefit to all of science as it provides a means of reviewing and accessing to relevant literature [17,18] for making significant advancements in knowledge [19,20]. Thus, OA increases both efficiency and effectiveness of science as a whole [21]. ...
Article
Full-text available
It appears that open access (OA) academic publishing is better for science because it provides frictionless access to make significant advancements in knowledge. OA also benefits individual researchers by providing the widest possible audience and concomitant increased citation rates. OA publishing rates are growing fast as increasing numbers of funders demand it and is currently dominated by gold OA (authors pay article processing charges (APCs)). Academics with limited financial resources perceive they must choose between publishing behind pay walls or using research funds for OA publishing. Worse, many new OA journals with low APCs did not have impact factors, which reduces OA selection for tenure track professors. Such unpleasant choices may be dissolving. This article provides analysis with a free and open source python script to collate all journals with impact factors with the now more than 12,000 OA journals that are truly platinum OA (neither the author nor the readers pay for the peer-reviewed work). The results found platinum OA is growing faster than both academic publishing and OA publishing. There are now over 350 platinum OA journals with impact factors over a wide variety of academic disciplines, giving most academics options for OA with no APCs.
... The final database selection included seven academic databases, based on the database screening conducted by Pearce (2018). The IEEE and ACM databases were included because of the in-depth focus on technological research topics. ...
Conference Paper
Full-text available
Cryptocurrencies, a new class of digital asset predominantly based on blockchain technologies, have gained immense popularity in recent years. Despite of advantages over traditional monetary systems such as lower transaction cost, increased transactional security, or transparency, cryptocurrencies are not free of disadvantages. The increase in popularity has also led to an increase in market manipulation in these markets. While there has been some attempt to identify and classify various cryptocurrency market manipulation schemes, there is a distinct need for a holistic description and classification of current cryptocurrency market manipulation schemes. Therefore, based on a systematic literature review, this paper provides an overview of cryptocurrency market manipulation methods using a concept-centric approach, a characterization of these methods, and identifies market vulnerabilities. Link: https://aisel.aisnet.org/icis2021/fintech/fintech/1/
... to systematically source open access higher education journal articles. The Educational Resources Information Center (ERIC) database was also included due to its carefully curated education search terms and therefore recommended for higher education research by library and information systems professionals at U.S. doctoral research institutions (Educational Resources Information Center, n.d.; Pearce, 2018). Despite the claims of transparency and ease of use by many scholars, crawler search engines such as Google Scholar were excluded as potential search agents due to their reported lack of precision for systematic reviews (Gusenbauer & Haddaway, 2019). ...
Chapter
This chapter presents the results of a systematic review to analyze the current research since 2019 for voice dispossession as attributional accommodation among women in higher education leadership. The authors sought to quantify and categorize these attributes to better identify the verbal and nonverbal ac-commodations made by women in higher education leadership to extend prior critical review of gender parity and equity for these leaders. Study findings may inform higher educational leadership to better understand voice dispossession among female leaders and the resulting attributional accommodations made to improve gender equity and parity for leadership roles in higher education.
Article
Full-text available
aperta-accessum,which: i) harvests names and emails from adepartments faculty webpage, ii) identifies scholars’ Open Researcherand Contributor IDs (ORCID iDs), iii)obtains digital object identifiers (DOIs) of publications for each scholar, iv) checks for existingcopies in an institution’s OA repository, v) identifies the legal opportunitiesto provide OA versions of all of the articles not already in the OA repository,vi) sends authors emails requesting a simple upload of author manuscripts, andvii) adds link harvested metadata from DOIs with uploaded preprint into OArepository. The results of this study show that inthe administrative time needed to open access a single document manually, aperta-accessum can process approximatelyfive entire departments worth of peer-reviewed articles. Following best practices discussed, it is clear this open sourceOA harvester enables institutional library’s stewardship of OA knowledge on themass-scale for radically reduced costs.
Chapter
Free and open source software (FOSS) differs from proprietary software. FOSS facilitates the design of various applications per the user's requirement. Web applications are not exceptional in this way. Web-based applications are mostly based on client server architecture. This article is an analytical study of FOSS products used in web-based client server architecture. This article will provide information about FOSS product such as FireFox (web browser), Apache (web server) and MySQL (RDBMS). These reveal that various FOSS products such as Apache server covers 65% of the market share, while MySQL covers 58.7% market share and hold the top-most rank.
Preprint
Full-text available
Despite growing interest in Open Access (OA) to scholarly literature, there is an unmet need for large-scale, up-to-date, and reproducible studies assessing the prevalence and characteristics of OA. We address this need using oaDOI, an open online service that determines OA status for 67 million articles. We use three samples, each of 100,000 articles, to investigate OA in three populations: 1) all journal articles assigned a Crossref DOI, 2) recent journal articles indexed in Web of Science, and 3) articles viewed by users of Unpaywall, an open-source browser extension that lets users find OA articles using oaDOI. We estimate that at least 28% of the scholarly literature is OA (19M in total) and that this proportion is growing, driven particularly by growth in Gold and Hybrid. The most recent year analyzed (2015) also has the highest percentage of OA (45%). Because of this growth, and the fact that readers disproportionately access newer articles, we find that Unpaywall users encounter OA quite frequently: 47% of articles they view are OA. Notably, the most common mechanism for OA is not Gold, Green, or Hybrid OA, but rather an under-discussed category we dub Bronze: articles made free-to-read on the publisher website, without an explicit Open license. We also examine the citation impact of OA articles, corroborating the so-called open-access citation advantage: accounting for age and discipline, OA articles receive 18% more citations than average, an effect driven primarily by Green and Hybrid OA. We encourage further research using the free oaDOI service, as a way to inform OA policy and practice.
Article
Full-text available
Distributed manufacturing even at the household level is now well established with the combined use of open source designs and self-replicating rapid prototyper (RepRap) 3-D printers. Previous work has shown substantial economic consumer benefits for producing their own polymer products. Now flexible filaments are available at roughly 3-times the cost of more conventional 3-D printing materials. To provide some insight into the potential for flexible filament to be both technically feasible and economically viable for distributed digital manufacturing at the consumer level this study investigates 20 common flexible household products. The 3-D printed products were quantified by print time, electrical energy use and filament consumption by mass to determine the cost to fabricate with a commercial RepRap 3-D printer. Printed parts were inspected and when necessary tested for their targeted application to ensure technical feasibility. Then, the experimentally measured cost to DIY manufacturers was compared to low and high market prices for comparable commercially available products. In addition, the mark-up and potential for long-term price declines was estimated for flexible filaments by converting thermoplastic elastomer (TPE) pellets into filament and reground TPE from a local recycling center into filament using an open source recyclebot. This study found that commercial flexible filament is economically as well as technically feasible for providing a means of distributed home-scale manufacturing of flexible products. The results found a 75% savings when compared to the least expensive commercially equivalent products and 92% when compared to high market priced products. Roughly, 160 flexible objects must be substituted to recover the capital costs to print flexible materials. However, as previous work has shown the Lulzbot Mini 3-D printer used in this study would provide more than a 100% ROI printing one object a week from hard thermoplastics, the upgrade needed to provide flexible filament capabilities can be accomplished with 37 average substitution flexible prints. This, again easily provides a triple digit return on investment printing one product a week. Although these savings, which are created by printing objects at home are substantial, the results also have shown the savings could be further increased to 93% when the use of a pellet extruder and TPE pellets, and 99% if recycled TPE filament made with a recyclebot is used. The capital costs of a recyclebot can be recovered in the manufacturing of about 9 kg of TPE filament, which can be accomplished in less than a week, enabling improved environmental impact as well as a strong financial return for heavy 3-D printer users.
Article
Full-text available
The 2020 toy and game market is projected to be US135billion.Todetermineif3Dprintingcouldaffectthesemarketsifconsumersoffsetpurchasesby3Dprintingfreedesigns,thisstudyinvestigatesthe100mostpopulardownloadeddesignsatMyMiniFactoryinamonth.SavingsarequantifiedforusingaLulzbotMini3Dprinterandthreefilamenttypes:commercialfilament,pelletextrudedfilament,andpostconsumerwasteconvertedtofilamentwitharecyclebot.Casestudiesprobedthequalityof:(1)sixcommoncomplextoys;(2)Legoblocks;and(3)thecustomizabilityofopensourceboardgames.Allfilamentsanalyzedsavedtheuserover75135 billion. To determine if 3D printing could affect these markets if consumers offset purchases by 3D printing free designs, this study investigates the 100 most popular downloaded designs at MyMiniFactory in a month. Savings are quantified for using a Lulzbot Mini 3D printer and three filament types: commercial filament, pellet-extruded filament, and post-consumer waste converted to filament with a recyclebot. Case studies probed the quality of: (1) six common complex toys; (2) Lego blocks; and (3) the customizability of open source board games. All filaments analyzed saved the user over 75% of the cost of commercially available true alternative toys and over 90% for recyclebot filament. Overall, these results indicate a single 3D printing repository among dozens is saving consumers well over 60 million/year in offset purchases. The most common savings fell by 40%–90% in total savings, which came with the ability to make novel toys and games. The results of this study show consumers can generate higher value items for less money using the open source distributed manufacturing paradigm. It appears clear that consumer do-it-yourself (DIY) manufacturing is set to have a significant impact on the toy and game markets in the future.
Article
Full-text available
The rise of Free and Open Source models for software development has catalyzed the growth of Free and Open Source hardware (also known as “Libre Hardware”). Libre Hardware is gaining significant traction in the scientific hardware community, where there is evidence that open development creates both technically superior and far less expensive scientific equipment than proprietary models. In this article, the evidence is reviewed and a collection of examples of business models is developed to service scientists who have the option to manufacture their own equipment using Open Source designs. Profitable Libre Hardware business models are reviewed, which includes kit, specialty component, and calibration suppliers for makers. The results indicate that Libre Hardware businesses should target technically sophisticated customers first and, as usability matures, target expanded markets of conventional consumers.
Article
Full-text available
Through reduced 3-D printer cost, increased usability, and greater material selection, additive manufacturing has transitioned from business manufacturing to the average prosumer. This study serves as a representative model for the potential future of 3-D printing in the average American household by employing a printer operator who was relatively unfamiliar with 3-D printing and the 3-D design files of common items normally purchased by the average consumer. Twenty-six items were printed in thermoplastic and a cost analysis was performed through comparison to comparable, commercially available products at a low and high price range. When compared to the low-cost items, investment in a 3-D printer represented a return of investment of over 100% in five years. The simple payback time for the high-cost comparison was less than 6 months, and produced a 986% return. Thus, fully-assembled commercial open source 3-D printers can be highly profitable investments for American consumers. Finally, as a preliminary gauge of the effect that widespread prosumer use of 3-D printing might have on the economy, savings were calculated based on the items’ download rates from open repositories. Results indicate that printing these selected items have already saved prosumers over $4 million by substituting for purchases.
Article
Full-text available
Technology is recognised as playing a part in the changing landscape in higher education; altering delivery modes and providing flexible opportunities for learning. Research into the use of wikis has shown that they provide many opportunities for student learning and the development of twenty-first century skills, however, there has been limited success in their use for collaboration. In this exploratory research, we report on a group wiki project in an engineering management unit at an Australian university. A wiki was introduced to replace the existing group report assessment to add transparency to the task. Each group had their own wiki and students were required to provide feedback to other group wikis which enabled transparency of students’ report development and group progress. The research aim was to investigate student perceptions of using a wiki. Students found the wikis helpful for benchmarking their activity against their peers. Students stated that peer feedback had improved their work; however, much of the feedback given by peers was too brief to be constructive, and provided too late to be useful in guiding their work. This paper will be of interest to academics interested in using wikis in their teaching to develop feedback, transparency, and benchmarking.
Article
Full-text available
Although theoretically the patent system is meant to bolster innovation, the current United States Patent and Trademark Office (USPTO) is cumbersome and involves a significant time investment to locate inactive patents less than 20 years old. This article reports on the development of an open source database to find these public domain ideas. First, a search strategy is explained. Then the operation and use of free and open source software are detailed to meet the needs of open hardware innovators. Finally, a case study is presented to demonstrate the utility of the approach with 3-D printing. The results showed how the Free Inactive Patent Search enables users to search using plain language text to find public domain concepts and then provides a hyperlinked list of ideas that takes users to the USPTO database for the patent for more information. All of the source code to operate the search and the website are open source themselves and provided in the public domain for free. In the case study on 3-D printing the time to identify public domain patents was cut by a factor of more than 1500. This tool has the potential for accelerating the development of open hardware technologies to create high value for the public.
Article
Full-text available
3D printing, also known as additive manufacturing, is the computer-guided process of fabricating physical objects by depositing successive layers of material. It has transformed manufacturing across virtually every industry, bringing about incredible advances in research and medicine. The rapidly growing consumer market now includes convenient and affordable “desktop” 3D printers. These are being used in the laboratory to create custom 3D-printed equipment, and a growing community of designers are contributing open-source, cost-effective innovations that can be used by both professionals and enthusiasts. User stories from investigators at the National Institutes of Health and the biomedical research community demonstrate the power of 3D printing to save valuable time and funding. While adoption of 3D printing has been slow in the biosciences to date, the potential is vast. The market predicts that within several years, 3D printers could be commonplace within the home; with so many practical uses for 3D printing, we anticipate that the technology will also play an increasingly important role in the laboratory.
Article
The current paper presents a comparative analysis of forums and wikis as tools for online collaborative learning. The comparison was developed analyzing the data collected during a collaborative experience in an asynchronous e-learning environment. The activities lasted five weeks and consisted of forum discussions and designing a project in a wiki environment. The research method included both quantitative and qualitative analyses. A quantitative comparison of forums and wikis was developed applying the coding scheme based on the following indicators: (1) inferencing, (2) producing, (3) developing, (4) evaluating, (5) summarizing, (6) organizing, and (7) supporting. The qualitative aspects were assessed using an open-ended questionnaire for collecting the participant perspective about the functionality of the collaborative tools. Results provided evidence of the different processes during the forums and wikis activities: processes such as inferencing, evaluating, organizing and supporting characterized forum discussions while wikis induced mainly processes of producing and developing. Different purposes were also evident: forums were useful for discussing, sharing ideas while wikis were used for developing a common collaborative document. In addition, the perceived time involved in performing the activities was different: forums were easier to access than wikis, while wikis required more time and were more difficult to use than forums. As a general conclusion it is not possible to state a superiority of a tool than another because each tool has his own characteristics and could be used with different purposes. Forums and wikis could have complementary functions and should be organized to complete each other for scaffolding students’ self-regulated strategies and learning. The findings are discussed in the framework of designing collaborative virtual courses with proper tool selection.
Conference Paper
Acknowledging the benefits associated with open access (OA) to scientific literature, the Foundation for Science and Technology (Fundação para a Ciência e Tecnologia - FCT), the national funding agency for science in Portugal, defined an OA policy that aims at ensuring free, online access to publications arising from FCT-funded research. Key to the implementation of FCT OA policy is the existence of a solid and mature repository infrastructure, the Scientific Open Access Repository of Portugal – RCAAP. Furthermore, RCAAP is part of PTCRIS, an integrated research information ecosystem whose holistic vision of the research information landscape allow us to link implementation of FCT OA mandate to added value services, with great benefits for the research community. In these paper, we describe the strategy adopted to implement and monitor a funder's OA mandate in an unprecedented integrated manner, using ground-breaking technical solutions built upon an existing infrastructure to address new challenges.